In [1]:
import logging
reload(logging)
logging.basicConfig(
    format='%(asctime)-9s %(levelname)-8s: %(message)s',
    datefmt='%I:%M:%S')

# Enable logging at INFO level
logging.getLogger().setLevel(logging.INFO)

In [2]:
# Generate plots inline
%pylab inline

import json
import os

# Support to access the remote target
import devlib
from env import TestEnv

# Support to configure and run RTApp based workloads
from wlgen import RTA, Ramp, Step, Pulse, Periodic


Populating the interactive namespace from numpy and matplotlib

Test environment setup


In [3]:
# Let's use the local host as a target
te = TestEnv(
    target_conf={
        "platform": 'host',
        "username": 'put_here_your_username'
    })


11:39:33  INFO    :         Target - Using base path: /home/derkling/Code/lisa
11:39:33  INFO    :         Target - Loading custom (inline) target configuration
11:39:33  INFO    :         Target - Devlib modules to load: []
11:39:33  INFO    :         Target - Connecting host target:
11:39:33  INFO    :         Target -   username : put_here_your_username
11:39:33  INFO    :         Target -   password : 
sudo password:········
11:39:36  INFO    :         Target - Initializing target workdir:
11:39:36  INFO    :         Target -    /tmp
11:39:36  INFO    :         Target - Topology:
11:39:36  INFO    :         Target -    [[0, 1, 2, 3, 4, 5, 6, 7]]
11:39:36  WARNING :         Target - Unable to identify cluster frequencies
11:39:36  INFO    :        TestEnv - Set results folder to:
11:39:36  INFO    :        TestEnv -    /home/derkling/Code/lisa/results/20160226_113936
11:39:36  INFO    :        TestEnv - Experiment results available also in:
11:39:36  INFO    :        TestEnv -    /home/derkling/Code/lisa/results_latest

Create a new RTA workload generator object

The wlgen::RTA class is a workload generator which exposes an API to configure RTApp based workload as well as to execute them on a target.


In [4]:
# Create a new RTApp workload generator
rtapp = RTA(
    
    target=te.target, # Target execution on the local machine
    
    name='example', # This is the name of the JSON configuration file reporting
                    # the generated RTApp configuration
    
    calibration={0: 10, 1: 11, 2: 12, 3: 13} # These are a set of fake
                                             # calibration values
)


11:39:36  INFO    :          WlGen - Setup new workload example

Workload Generation Examples

Single periodic task

An RTApp workload is defined by specifying a kind, which represents the way we want to defined the behavior of each task.
The most common kind is profile, which allows to define each task using one of the predefined profile supported by the RTA base class.

The following example shows how to generate a "periodic" task


In [5]:
# Configure this RTApp instance to:
rtapp.conf(
    
    # 1. generate a "profile based" set of tasks
    kind='profile',
    
    # 2. define the "profile" of each task
    params={
        
        # 3. PERIODIC task
        # 
        # This class defines a task which load is periodic with a configured
        # period and duty-cycle.
        # 
        # This class is a specialization of the 'pulse' class since a periodic
        # load is generated as a sequence of pulse loads.
        # 
        # Args:
        #     cuty_cycle_pct  (int, [0-100]): the pulses load [%]
        #                                     default: 50[%]
        #     duration_s  (float): the duration in [s] of the entire workload
        #                          default: 1.0[s]
        #     period_ms   (float): the period used to define the load in [ms]
        #                          default: 100.0[ms]
        #     delay_s     (float): the delay in [s] before ramp start
        #                          default: 0[s]
        #     sched       (dict):  the scheduler configuration for this task
        'task_per20': Periodic(
            period_ms=100,         # period
            duty_cycle_pct=20,     # duty cycle
            duration_s=5,          # duration
            cpus=None,             # run on all CPUS
            sched={
                "policy": "FIFO",  # Run this task as a SCHED_FIFO task
            },
            delay_s=0              # start at the start of RTApp
        ).get(),
        
    },
    
    # 4. use this folder for task logfiles
    run_dir='/tmp'
    
);


11:39:36  INFO    :          RTApp - Workload duration defined by longest task
11:39:36  INFO    :          RTApp - Default policy: SCHED_OTHER
11:39:36  INFO    :          RTApp - ------------------------
11:39:36  INFO    :          RTApp - task [task_per20], sched: {'policy': 'FIFO'}
11:39:36  INFO    :          RTApp -  | calibration CPU: 0
11:39:36  INFO    :          RTApp -  | loops count: 1
11:39:36  INFO    :          RTApp - + phase_000001: duration 5.000000 [s] (50 loops)
11:39:36  INFO    :          RTApp - |  period   100000 [us], duty_cycle  20 %
11:39:36  INFO    :          RTApp - |  run_time  20000 [us], sleep_time  80000 [us]

The output of the previous cell reports the main properties of the generated tasks. Thus for example we see that the first task is configure to be:

  1. named task_per20
  2. will be executed as a SCHED_FIFO task
  3. generating a load which is calibrated with respect to the CPU 0
  4. with one single "phase" which defines a peripodic load for the duration of 5[s]
  5. that periodic load consistes of 50 cycles
  6. each cycle has a period of 100[ms] and a duty-cycle of 20%
  7. which means that the task, for every cycle, will run for 20[ms] and then sleep for 20[ms]

All these properties are translated into a JSON configuration file for RTApp.
Let see what it looks like the generated configuration file:


In [6]:
# Dump the configured JSON file for that task
with open("./example_00.json") as fh:
    rtapp_config = json.load(fh)
print json.dumps(rtapp_config, indent=4)


{
    "tasks": {
        "task_per20": {
            "policy": "SCHED_FIFO", 
            "phases": {
                "p000001": {
                    "run": 20000, 
                    "timer": {
                        "ref": "task_per20", 
                        "period": 100000
                    }, 
                    "loop": 50
                }
            }, 
            "loop": 1
        }
    }, 
    "global": {
        "duration": -1, 
        "logdir": "/tmp", 
        "default_policy": "SCHED_OTHER", 
        "calibration": 10
    }
}

Workload mix

Using the wlgen::RTA workload generator we can easily create multiple tasks, each one with different "profiles", which are executed once the rtapp application is started in the target.

In the following example we configure a workload mix composed by a RAMP task, a STEP task and a PULSE task:


In [7]:
# Configure this RTApp instance to:
rtapp.conf(
    # 1. generate a "profile based" set of tasks
    kind='profile',
    
    # 2. define the "profile" of each task
    params={
        
        # 3. RAMP task
        #
        # This class defines a task which load is a ramp with a configured number
        # of steps according to the input parameters.
        # 
        # Args:
        #     start_pct (int, [0-100]): the initial load [%], (default 0[%])
        #     end_pct   (int, [0-100]): the final load [%], (default 100[%])
        #     delta_pct (int, [0-100]): the load increase/decrease [%],
        #                               default: 10[%]
        #                               increase if start_prc < end_prc
        #                               decrease  if start_prc > end_prc
        #     time_s    (float): the duration in [s] of each load step
        #                        default: 1.0[s]
        #     period_ms (float): the period used to define the load in [ms]
        #                        default: 100.0[ms]
        #     delay_s   (float): the delay in [s] before ramp start
        #                        default: 0[s]
        #     loops     (int):   number of time to repeat the ramp, with the
        #                        specified delay in between
        #                        default: 0
        #     sched     (dict): the scheduler configuration for this task
        #     cpus      (list): the list of CPUs on which task can run
        'task_rmp20_5-60': Ramp(
            period_ms=100,         # period
            start_pct=5,           # intial load
            end_pct=65,            # end load
            delta_pct=20,          # load % increase...
            time_s=1,              # ... every 1[s]
            cpus="0"               # run just on first CPU
        ).get(),
        
        # 4. STEP task
        # 
        # This class defines a task which load is a step with a configured
        # initial and final load.
        # 
        # Args:
        # start_pct (int, [0-100]): the initial load [%]
        #                               default 0[%])
        # end_pct   (int, [0-100]): the final load [%]
        #                               default 100[%]
        # time_s    (float): the duration in [s] of the start and end load
        #                        default: 1.0[s]
        # period_ms (float): the period used to define the load in [ms]
        #                        default 100.0[ms]
        # delay_s   (float): the delay in [s] before ramp start
        #                        default 0[s]
        # loops     (int):   number of time to repeat the ramp, with the
        #                        specified delay in between
        #                        default: 0
        # sched     (dict): the scheduler configuration for this task
        # cpus      (list): the list of CPUs on which task can run
        'task_stp10-50': Step(
            period_ms=100,         # period
            start_pct=0,           # intial load
            end_pct=50,            # end load
            time_s=1,              # ... every 1[s]
            delay_s=0.5            # start .5[s] after the start of RTApp
        ).get(),
        
        # 5. PULSE task
        #
        # This class defines a task which load is a pulse with a configured
        # initial and final load.
        # 
        # The main difference with the 'step' class is that a pulse workload is
        # by definition a 'step down', i.e. the workload switch from an finial
        # load to a final one which is always lower than the initial one.
        # Moreover, a pulse load does not generate a sleep phase in case of 0[%]
        # load, i.e. the task ends as soon as the non null initial load has
        # completed.
        # 
        # Args:
        #     start_pct (int, [0-100]): the initial load [%]
        #                               default: 0[%]
        #     end_pct   (int, [0-100]): the final load [%]
        #                               default: 100[%]
        #               NOTE: must be lower than start_pct value
        #     time_s    (float): the duration in [s] of the start and end load
        #                        default: 1.0[s]
        #                        NOTE: if end_pct is 0, the task end after the
        #                        start_pct period completed
        #     period_ms (float): the period used to define the load in [ms]
        #                        default: 100.0[ms]
        #     delay_s   (float): the delay in [s] before ramp start
        #                        default: 0[s]
        #     loops     (int):   number of time to repeat the ramp, with the
        #                        specified delay in between
        #                        default: 0
        #     sched     (dict):  the scheduler configuration for this task
        #     cpus      (list):  the list of CPUs on which task can run
        'task_pls5-80': Pulse(
            period_ms=100,         # period
            start_pct=65,          # intial load
            end_pct=5,             # end load
            time_s=1,              # ... every 1[s]
            delay_s=0.5            # start .5[s] after the start of RTApp
        ).get(),
        
        
    },
    
    # 6. use this folder for task logfiles
    run_dir='/tmp'
    
);


11:39:37  INFO    :          RTApp - Workload duration defined by longest task
11:39:37  INFO    :          RTApp - Default policy: SCHED_OTHER
11:39:37  INFO    :          RTApp - ------------------------
11:39:37  INFO    :          RTApp - task [task_pls5-80], sched: using default policy
11:39:37  INFO    :          RTApp -  | start delay: 0.500000 [s]
11:39:37  INFO    :          RTApp -  | calibration CPU: 0
11:39:37  INFO    :          RTApp -  | loops count: 1
11:39:37  INFO    :          RTApp - + phase_000001: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  65 %
11:39:37  INFO    :          RTApp - |  run_time  65000 [us], sleep_time  35000 [us]
11:39:37  INFO    :          RTApp - + phase_000002: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle   5 %
11:39:37  INFO    :          RTApp - |  run_time   5000 [us], sleep_time  95000 [us]
11:39:37  INFO    :          RTApp - ------------------------
11:39:37  INFO    :          RTApp - task [task_rmp20_5-60], sched: using default policy
11:39:37  INFO    :          RTApp -  | calibration CPU: 0
11:39:37  INFO    :          RTApp -  | loops count: 1
11:39:37  INFO    :          RTApp -  | CPUs affinity: 0
11:39:37  INFO    :          RTApp - + phase_000001: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle   5 %
11:39:37  INFO    :          RTApp - |  run_time   5000 [us], sleep_time  95000 [us]
11:39:37  INFO    :          RTApp - + phase_000002: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  25 %
11:39:37  INFO    :          RTApp - |  run_time  25000 [us], sleep_time  75000 [us]
11:39:37  INFO    :          RTApp - + phase_000003: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  45 %
11:39:37  INFO    :          RTApp - |  run_time  45000 [us], sleep_time  55000 [us]
11:39:37  INFO    :          RTApp - + phase_000004: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  65 %
11:39:37  INFO    :          RTApp - |  run_time  65000 [us], sleep_time  35000 [us]
11:39:37  INFO    :          RTApp - ------------------------
11:39:37  INFO    :          RTApp - task [task_stp10-50], sched: using default policy
11:39:37  INFO    :          RTApp -  | start delay: 0.500000 [s]
11:39:37  INFO    :          RTApp -  | calibration CPU: 0
11:39:37  INFO    :          RTApp -  | loops count: 1
11:39:37  INFO    :          RTApp -  + phase_000001: sleep 1.000000 [s]
11:39:37  INFO    :          RTApp - + phase_000002: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  50 %
11:39:37  INFO    :          RTApp - |  run_time  50000 [us], sleep_time  50000 [us]

In [8]:
# Dump the configured JSON file for that task
with open("./example_00.json") as fh:
    rtapp_config = json.load(fh)
print json.dumps(rtapp_config, indent=4)


{
    "tasks": {
        "task_rmp20_5-60": {
            "policy": "SCHED_OTHER", 
            "phases": {
                "p000004": {
                    "run": 65000, 
                    "timer": {
                        "ref": "task_rmp20_5-60", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000003": {
                    "run": 45000, 
                    "timer": {
                        "ref": "task_rmp20_5-60", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000002": {
                    "run": 25000, 
                    "timer": {
                        "ref": "task_rmp20_5-60", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000001": {
                    "run": 5000, 
                    "timer": {
                        "ref": "task_rmp20_5-60", 
                        "period": 100000
                    }, 
                    "loop": 10
                }
            }, 
            "cpus": [
                0
            ], 
            "loop": 1
        }, 
        "task_pls5-80": {
            "policy": "SCHED_OTHER", 
            "phases": {
                "p000002": {
                    "run": 5000, 
                    "timer": {
                        "ref": "task_pls5-80", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000001": {
                    "run": 65000, 
                    "timer": {
                        "ref": "task_pls5-80", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000000": {
                    "delay": 500000
                }
            }, 
            "loop": 1
        }, 
        "task_stp10-50": {
            "policy": "SCHED_OTHER", 
            "phases": {
                "p000002": {
                    "run": 50000, 
                    "timer": {
                        "ref": "task_stp10-50", 
                        "period": 100000
                    }, 
                    "loop": 10
                }, 
                "p000001": {
                    "sleep": 1000000, 
                    "loop": 1
                }, 
                "p000000": {
                    "delay": 500000
                }
            }, 
            "loop": 1
        }
    }, 
    "global": {
        "duration": -1, 
        "logdir": "/tmp", 
        "default_policy": "SCHED_OTHER", 
        "calibration": 10
    }
}

Workload composition


In [9]:
# Initial phase and pinning parameters
ramp = Ramp(period_ms=100, start_pct=5, end_pct=65, delta_pct=20, time_s=1,
            cpus="0")

# Following phases
medium_slow = Periodic(duty_cycle_pct=10, duration_s=5, period_ms=100)
high_fast   = Periodic(duty_cycle_pct=60, duration_s=5, period_ms=10)
medium_fast = Periodic(duty_cycle_pct=10, duration_s=5, period_ms=1)
high_slow   = Periodic(duty_cycle_pct=60, duration_s=5, period_ms=100)

#Compose the task
complex_task = ramp + medium_slow + high_fast + medium_fast + high_slow

In [10]:
# Configure this RTApp instance to:
rtapp.conf(
    # 1. generate a "profile based" set of tasks
    kind='profile',
    
    # 2. define the "profile" of each task
    params={
        'complex' : complex_task.get()
    },

    # 6. use this folder for task logfiles
    run_dir='/tmp'
)


11:39:37  INFO    :          RTApp - Workload duration defined by longest task
11:39:37  INFO    :          RTApp - Default policy: SCHED_OTHER
11:39:37  INFO    :          RTApp - ------------------------
11:39:37  INFO    :          RTApp - task [complex], sched: using default policy
11:39:37  INFO    :          RTApp -  | calibration CPU: 0
11:39:37  INFO    :          RTApp -  | loops count: 1
11:39:37  INFO    :          RTApp -  | CPUs affinity: 0
11:39:37  INFO    :          RTApp - + phase_000001: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle   5 %
11:39:37  INFO    :          RTApp - |  run_time   5000 [us], sleep_time  95000 [us]
11:39:37  INFO    :          RTApp - + phase_000002: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  25 %
11:39:37  INFO    :          RTApp - |  run_time  25000 [us], sleep_time  75000 [us]
11:39:37  INFO    :          RTApp - + phase_000003: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  45 %
11:39:37  INFO    :          RTApp - |  run_time  45000 [us], sleep_time  55000 [us]
11:39:37  INFO    :          RTApp - + phase_000004: duration 1.000000 [s] (10 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  65 %
11:39:37  INFO    :          RTApp - |  run_time  65000 [us], sleep_time  35000 [us]
11:39:37  INFO    :          RTApp - + phase_000005: duration 5.000000 [s] (50 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  10 %
11:39:37  INFO    :          RTApp - |  run_time  10000 [us], sleep_time  90000 [us]
11:39:37  INFO    :          RTApp - + phase_000006: duration 5.000000 [s] (500 loops)
11:39:37  INFO    :          RTApp - |  period    10000 [us], duty_cycle  60 %
11:39:37  INFO    :          RTApp - |  run_time   6000 [us], sleep_time   4000 [us]
11:39:37  INFO    :          RTApp - + phase_000007: duration 5.000000 [s] (5000 loops)
11:39:37  INFO    :          RTApp - |  period     1000 [us], duty_cycle  10 %
11:39:37  INFO    :          RTApp - |  run_time    100 [us], sleep_time    900 [us]
11:39:37  INFO    :          RTApp - + phase_000008: duration 5.000000 [s] (50 loops)
11:39:37  INFO    :          RTApp - |  period   100000 [us], duty_cycle  60 %
11:39:37  INFO    :          RTApp - |  run_time  60000 [us], sleep_time  40000 [us]
Out[10]:
'example_00'