ReleaseNotes_v16.12


Target Connectivity

Configurable logging system

All LISA modules have been updated to use a more consistent logging which can be configured using a single configuraton file:


In [1]:
!head -n12 $LISA_HOME/logging.conf


################################################################################
### Root Logger
################################################################################

[loggers]
keys=root

[logger_root]
level=INFO
handlers=consoleHandler,fileHandler
propagate=0

Each module has a unique name which can be used to assign a priority level for messages generated by that module.


In [2]:
!head -n30 $LISA_HOME/logging.conf | tail -n5


# [logger_TestEnv]
# qualname=TestEnv
# level=DEBUG
# handlers=consoleHandler,fileHandler
# propagate=0

The default logging level for a notebook can also be easily configured using this few lines


In [3]:
import logging
from conf import LisaLogging
LisaLogging.setup(level=logging.INFO)


2016-12-15 17:15:08,209 INFO    : root         : Using LISA logging configuration:
2016-12-15 17:15:08,209 INFO    : root         :   /home/patbel01/Code/lisa/logging.conf

Removed Juno/Juno2 distinction

Juno R0 and Juno R2 boards are now accessible by specifying "juno" in the target configuration. The previous distinction was required because of a different way for the two boards to report HWMON channels. This distinction is not there anymore and thus Juno boards can now be connected using the same platform data.


In [6]:
from env import TestEnv

te = TestEnv({
        'platform'     : 'linux',
        'board'        : 'juno',
        'host'         : '10.1.210.45',
        'username'     : 'root'
    })
target = te.target


2016-12-15 16:49:03,716 INFO    : TestEnv      : Using base path: /home/patbel01/Code/lisa
2016-12-15 16:49:03,716 INFO    : TestEnv      : Loading custom (inline) target configuration
2016-12-15 16:49:03,717 INFO    : TestEnv      : Devlib modules to load: ['bl', 'hwmon', 'cpufreq']
2016-12-15 16:49:03,717 INFO    : TestEnv      : Connecting linux target:
2016-12-15 16:49:03,717 INFO    : TestEnv      :   username : root
2016-12-15 16:49:03,718 INFO    : TestEnv      :       host : 10.1.210.45
2016-12-15 16:49:03,718 INFO    : TestEnv      :   password : 
2016-12-15 16:49:03,718 INFO    : TestEnv      : Connection settings:
2016-12-15 16:49:03,718 INFO    : TestEnv      :    {'username': 'root', 'host': '10.1.210.45', 'password': ''}
2016-12-15 16:49:20,157 INFO    : TestEnv      : Initializing target workdir:
2016-12-15 16:49:20,157 INFO    : TestEnv      :    /root/devlib-target
2016-12-15 16:49:21,948 INFO    : TestEnv      : Topology:
2016-12-15 16:49:21,949 INFO    : TestEnv      :    [[0, 3, 4, 5], [1, 2]]
2016-12-15 16:49:23,594 INFO    : TestEnv      : Loading default EM:
2016-12-15 16:49:23,594 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/juno.json
2016-12-15 16:49:23,602 INFO    : EnergyMeter  : Scanning for HWMON channels, may take some time...
2016-12-15 16:49:23,603 INFO    : EnergyMeter  : Channels selected for energy sampling:
2016-12-15 16:49:23,603 INFO    : EnergyMeter  :    BOARDBIG_energy
2016-12-15 16:49:23,604 INFO    : EnergyMeter  :    BOARDLITTLE_energy
2016-12-15 16:49:23,604 INFO    : TestEnv      : Set results folder to:
2016-12-15 16:49:23,604 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results/20161215_164923
2016-12-15 16:49:23,604 INFO    : TestEnv      : Experiment results available also in:
2016-12-15 16:49:23,605 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results_latest

Executor Module

Simplified tests definition using in-code configurations

Automated LISA tests previously configured the Executor using JSON files. This is still possible, but the existing tests now use Python dictionaries directly in the code. In the short term, this allows de-duplicating configuration elements that are shared between multiple tests. It will later allow more flexible test configuration.

See tests/eas/acceptance.py for an example of how this is currently used.

Support to write files from Executor configuration

https://github.com/ARM-software/lisa/pull/209

A new "files" attribute can be added to Executor configurations which allows to specify a list files (e.g. sysfs and procfs) and values to be written to that files.

For example, the following test configuration:


In [9]:
tests_conf = {
    "confs" : [
        {
            "tag"            : "base",
            "flags"          : "ftrace",
            "sched_features" : "NO_ENERGY_AWARE",
            "cpufreq"        : {
                "governor" : "performance",
            },
            "files"          : {
                '/proc/sys/kernel/sched_is_big_little'           : '0',
                '!/proc/sys/kernel/sched_migration_cost_ns'      : '500000'
            },
        }
    ]
}

can be used to run a test where the platform is configured to

  • disable the "sched_is_big_little" flag (if present)
  • set to 50ms the "sched_migration_cost_ns"

Nortice that a value written in a file is verified only if the file path is prefixed by a '/'. Otherwise, the write never fails, e.g. if the file does not exists.

Support to freeze user-space across a test

https://github.com/ARM-software/lisa/pull/227

Executor learned the "freeze_userspace" conf flag. When this flag is present, LISA uses the devlib freezer to freeze as much of userspace as possible while the experiment workload is executing, in order to reduce system noise.

The Executor example notebook:

https://github.com/ARM-software/lisa/blob/master/ipynb/examples/utils/executor_example.ipynb

gives an example of using this feature.

Trace module

Tasks name pre-loading

When the Trace module is initialized, by default all the tasks in that trace are identified and exposed via the usual getTask() method:


In [15]:
from trace import Trace
import json

with open('/home/patbel01/Code/lisa/results/LisaInANutshell_Backup/platform.json', 'r') as fh:
    platform = json.load(fh)

trace = Trace('/home/patbel01/Code/lisa/results/LisaInANutshell_Backup/trace.dat',
    ['sched_switch'], platform
))


2016-12-15 17:04:14,988 INFO    : Trace        : Parsing FTrace format...
2016-12-15 17:04:15,162 INFO    : Trace        : Collected events spans a 11.783 [s] time interval
2016-12-15 17:04:15,162 INFO    : Trace        : Set plots time range to (0.000000, 11.782618)[s]
2016-12-15 17:04:15,163 INFO    : Analysis     : Registering trace analysis modules:
2016-12-15 17:04:15,163 INFO    : Analysis     :    latency
2016-12-15 17:04:15,193 INFO    : Analysis     :    eas
2016-12-15 17:04:15,197 INFO    : Analysis     :    tasks
2016-12-15 17:04:15,205 INFO    : Analysis     :    cpus
2016-12-15 17:04:15,209 INFO    : Analysis     :    functions
2016-12-15 17:04:15,212 INFO    : Analysis     :    status
2016-12-15 17:04:15,214 INFO    : Analysis     :    idle
2016-12-15 17:04:15,218 INFO    : Analysis     :    stune
2016-12-15 17:04:15,224 INFO    : Analysis     :    frequency

In [17]:
logging.info("%d tasks loaded from trace", len(trace.getTasks()))


2016-12-15 17:05:03,760 INFO    : root         : 45 tasks loaded from trace

In [19]:
logging.info("The rt-app task in this trace has these PIDs:")
logging.info("  %s", trace.getTasks()['rt-app'])


2016-12-15 17:05:49,699 INFO    : root         : The rt-app task in this trace has this PID:
2016-12-15 17:05:49,699 INFO    : root         :   {'pid': [1388, 1387]}

Android Support

Added support for Pixel Phones

A new platform definition file has been added which allows to easily setup a connection with an Pixel device:


In [33]:
!cat $LISA_HOME/libs/utils/platforms/pixel.json


{
    "board" : {
        "cores" : [
            "Silver", "Silver", "Gold", "Gold"
        ],
        "big_core" : "Gold",
        "modules" : ["bl", "cpufreq"]
    },
    "nrg_model" : {
        "little" : {
            "cpu" : {
                "nrg_max" :  925,
                "cap_max" :  763,
            },
            "cluster" : {
                "nrg_max" :  52
            }
        },
        "big": {
            "cpu" : {
                "nrg_max" :  1715,
                "cap_max" :  1024
            },
            "cluster" : {
                "nrg_max" :  96
            }
        }
    }
}

In [4]:
from env import TestEnv

te = TestEnv({
        'platform'     : 'android',
        'board'        : 'pixel',
        'ANDROID_HOME' : '/home/patbel01/Code/lisa/tools/android-sdk-linux/'
    }, force_new=True)
target = te.target


2016-12-15 17:15:15,670 INFO    : TestEnv      : Using base path: /home/patbel01/Code/lisa
2016-12-15 17:15:15,671 INFO    : TestEnv      : Loading custom (inline) target configuration
2016-12-15 17:15:15,671 INFO    : TestEnv      : External tools using:
2016-12-15 17:15:15,671 INFO    : TestEnv      :    ANDROID_HOME: /home/patbel01/Code/lisa/tools/android-sdk-linux/
2016-12-15 17:15:15,672 INFO    : TestEnv      :    CATAPULT_HOME: /home/patbel01/Code/lisa/tools/catapult
2016-12-15 17:15:15,672 INFO    : TestEnv      : Loading board:
2016-12-15 17:15:15,672 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/pixel.json
2016-12-15 17:15:15,673 INFO    : TestEnv      : Devlib modules to load: [u'bl', u'cpufreq']
2016-12-15 17:15:15,674 INFO    : TestEnv      : Connecting Android target [DEFAULT]
2016-12-15 17:15:15,674 INFO    : TestEnv      : Connection settings:
2016-12-15 17:15:15,674 INFO    : TestEnv      :    None
2016-12-15 17:15:15,733 INFO    : android      : ls command is set to ls
2016-12-15 17:15:16,068 INFO    : TestEnv      : Initializing target workdir:
2016-12-15 17:15:16,068 INFO    : TestEnv      :    /data/local/tmp/devlib-target
2016-12-15 17:15:16,555 INFO    : TestEnv      : Topology:
2016-12-15 17:15:16,556 INFO    : TestEnv      :    [[0, 1], [2, 3]]
2016-12-15 17:15:16,635 INFO    : TestEnv      : Loading default EM:
2016-12-15 17:15:16,636 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/pixel.json
2016-12-15 17:15:16,637 INFO    : TestEnv      : Set results folder to:
2016-12-15 17:15:16,637 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results/20161215_171516
2016-12-15 17:15:16,638 INFO    : TestEnv      : Experiment results available also in:
2016-12-15 17:15:16,638 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results_latest

Added UiBench workload

A new Android benchmark has been added to run UiBench provided tests. Here is a notebook which provides an example of how to run this test on your android target:

https://github.com/ARM-software/lisa/blob/master/ipynb/examples/android/benchmarks/Android_UiBench.ipynb

Tests

Intial version of the preliminary tests

Preliminary tests aim at verifying some basic support required for a complete functional EAS solution.

A initial version of these preliminary tests is now available:

https://github.com/ARM-software/lisa/blob/master/tests/eas/preliminary.py

and it will be extended in the future to include more and more tests.

Capacity capping test

A new test has been added to verify that capacity capping is working as expected:

https://github.com/ARM-software/lisa/blob/master/tests/eas/capacity_capping.py

Acceptance tests reworked

The EAS acceptace test collects a set of platform independent tests to verify basic EAS beahviours.

This test has been cleaned up and it's now avaiable with a detailed documentation:

https://github.com/ARM-software/lisa/blob/master/tests/eas/acceptance.py

Notebooks

Added scratchpad notebooks

A new scratchpad folder has been added under the ipynb folder which collects the available notebooks:


In [12]:
!tree -L 1 ~/Code/lisa/ipynb


/home/patbel01/Code/lisa/ipynb
├── chromeos
├── devlib
├── energy
├── examples
├── profiling
├── releases
├── sched_dvfs
├── sched_tune
├── scratchpad
├── server.log
├── server.pid
├── thermal
├── tutorial
└── wlgen

12 directories, 2 files

This folder is configured to be ignored by git, thus it's the best place to place your work-in-progress notebooks.

Example notebook restructoring

Example notebooks has been consolidated and better organized by topic:


In [13]:
!tree -L 1 ~/Code/lisa/ipynb/examples


/home/patbel01/Code/lisa/ipynb/examples
├── android
├── devlib
├── energy_meter
├── trace_analysis
├── trappy
├── utils
└── wlgen

7 directories, 0 files

This is the folder to look into when it comes to undedrstand how a specific LISA API works.

Here is where we will provide a dedicated folder and set of notebooks for each of the main LISA modules.