This notebook shows the features provided by the idle state analysis module. It will be necessary to collect the following events:
cpu_idle, to filter out intervals of time in which the CPU is idlesched_switch, to recognise tasks on kernelsharkDetails on idle states profiling ar given in Per-CPU/Per-Cluster Idle State Residency Profiling below.
In [1]:
import logging
from conf import LisaLogging
LisaLogging.setup()
In [2]:
%matplotlib inline
import os
# Support to access the remote target
from env import TestEnv
# Support to access cpuidle information from the target
from devlib import *
# Support to configure and run RTApp based workloads
from wlgen import RTA, Ramp
# Support for trace events analysis
from trace import Trace
# DataFrame support
import pandas as pd
from pandas import DataFrame
# Trappy (plots) support
from trappy import ILinePlot
from trappy.stats.grammar import Parser
In [3]:
# Setup a target configuration
my_conf = {
# Target platform and board
"platform" : 'linux',
"board" : 'juno',
# Target board IP/MAC address
"host" : '192.168.0.1',
# Login credentials
"username" : 'root',
"password" : 'juno',
"results_dir" : "IdleAnalysis",
# RTApp calibration values (comment to let LISA do a calibration run)
#"rtapp-calib" : {
# "0": 318, "1": 125, "2": 124, "3": 318, "4": 318, "5": 319
#},
# Tools required by the experiments
"tools" : ['rt-app', 'trace-cmd'],
"modules" : ['bl', 'cpufreq', 'cpuidle'],
"exclude_modules" : ['hwmon'],
# FTrace events to collect for all the tests configuration which have
# the "ftrace" flag enabled
"ftrace" : {
"events" : [
"cpu_idle",
"sched_switch"
],
"buffsize" : 10 * 1024,
},
}
In [4]:
# Initialize a test environment
te = TestEnv(my_conf, wipe=False, force_new=True)
target = te.target
In [5]:
# We're going to run quite a heavy workload to try and create short idle periods.
# Let's set the CPU frequency to max to make sure those idle periods exist
# (otherwise at a lower frequency the workload might overload the CPU
# so it never went idle at all)
te.target.cpufreq.set_all_governors('performance')
Out[5]:
This experiment:
perturb_cpus to ensure 'cpu_idle' events are present in the trace for all CPUs
In [6]:
cpu = 1
def experiment(te):
# Create RTApp RAMP task
rtapp = RTA(te.target, 'ramp', calibration=te.calibration())
rtapp.conf(kind='profile',
params={
'ramp' : Ramp(
start_pct = 80,
end_pct = 10,
delta_pct = 5,
time_s = 0.5,
period_ms = 5,
cpus = [cpu]).get()
})
# FTrace the execution of this workload
te.ftrace.start()
# Momentarily wake all CPUs to ensure cpu_idle trace events are present from the beginning
te.target.cpuidle.perturb_cpus()
rtapp.run(out_dir=te.res_dir)
te.ftrace.stop()
# Collect and keep track of the trace
trace_file = os.path.join(te.res_dir, 'trace.dat')
te.ftrace.get_trace(trace_file)
# Dump platform descriptor
te.platform_dump(te.res_dir)
In [7]:
experiment(te)
In [8]:
# Base folder where tests folder are located
res_dir = te.res_dir
logging.info('Content of the output folder %s', res_dir)
!tree {res_dir}
In [9]:
trace = Trace(res_dir, my_conf['ftrace']['events'], te.platform)
It is possible to get the residency in each idle state of a CPU or a cluster with the following commands:
In [10]:
# Idle state residency for CPU 3
CPU=3
state_res = trace.data_frame.cpu_idle_state_residency(CPU)
state_res
Out[10]:
For the translation between the idle value and its description:
In [11]:
DataFrame(data={'value': state_res.index.values,
'name': [te.target.cpuidle.get_state(i, cpu=CPU) for i in state_res.index.values]})
Out[11]:
The IdleAnalysis module provide methods for plotting residency data:
In [12]:
ia = trace.analysis.idle
In [13]:
# Actual time spent in each idle state
ia.plotCPUIdleStateResidency([1,2])
In [14]:
# Percentage of time spent in each idle state
ia.plotCPUIdleStateResidency([1,2], pct=True)
Take a look at the target's idle states:
In [15]:
te.target.cpuidle.get_states()
Out[15]:
Now use trappy to plot the idle state of a single CPU over time. Higher is deeper: the plot is at -1 when the CPU is active, 0 for WFI, 1 for CPU sleep, etc.
We should see that as the workload ramps down and the idle periods become longer, the idle states used become deeper.
In [16]:
p = Parser(trace.ftrace, filters = {'cpu_id': cpu})
idle_df = p.solve('cpu_idle:state')
ILinePlot(idle_df, column=cpu, drawstyle='steps-post').view()
In [17]:
def get_idle_periods(df):
series = df[cpu]
series = series[series.shift() != series].dropna()
if series.iloc[0] == -1:
series = series.iloc[1:]
idles = series.iloc[0::2]
wakeups = series.iloc[1::2]
if len(idles) > len(wakeups):
idles = idles.iloc[:-1]
else:
wakeups = wakeups.iloc[:-1]
lengths = pd.Series((wakeups.index - idles.index), index=idles.index)
return pd.DataFrame({"length": lengths, "state": idles})
Make a scatter plot of the length of idle periods against the state that was entered. We should see that for long idle periods, deeper states were entered (i.e. we should see a positive corellation between the X and Y axes).
In [18]:
lengths = get_idle_periods(idle_df)
lengths.plot(kind='scatter', x='length', y='state')
Out[18]:
Draw a histogram of the length of idle periods shorter than 100ms in which the CPU entered cpuidle state 2.
In [19]:
df = lengths[(lengths['state'] == 2) & (lengths['length'] < 0.010)]
df.hist(column='length', bins=50)
Out[19]:
In [20]:
# Idle state residency for CPUs in the big cluster
trace.data_frame.cluster_idle_state_residency('big')
Out[20]:
In [21]:
# Actual time spent in each idle state for CPUs in the big and LITTLE clusters
ia.plotClusterIdleStateResidency(['big', 'LITTLE'])
In [22]:
# Percentage of time spent in each idle state for CPUs in the big and LITTLE clusters
ia.plotClusterIdleStateResidency(['big', 'LITTLE'], pct=True)