In this tutorial, we see how to look at the group structure in a project, and how to use groups to filter and aggregate metric data. For this tutorial to execute completely, you need to have:
Note: It is recommended that you look at the Getting Started notebook first to familiarize yourself with the Query class and how pandas dataframes are used to represent time series data.
If there is no default project set already, you must do so using 'set_datalab_project_id'. Note, the project you set here must be the hosting project for a Stackdriver account, otherwise you will see the following error while loading the groups in the next cell:
`BadRequest: 400 The project 'my-project-id' is not a Stackdriver account project.`
In [1]:
# set_datalab_project_id('my-project-id')
In [2]:
from google.datalab.stackdriver import monitoring as gcm
groups_dataframe = gcm.Groups().as_dataframe()
# Sort the dataframe by the group name, and reset the index.
groups_dataframe = groups_dataframe.sort_values(by='Group name').reset_index(drop=True)
groups_dataframe.head(5)
Out[2]:
Now we initialize first_group_id from the list of Stackdriver groups. Please note:
first_group_id to the ID of a group that does have an instance.
In [3]:
import sys
if groups_dataframe.empty:
sys.stderr.write('This project has no Stackdriver groups. The remaining notebook '
'will raise errors!')
else:
first_group_id = groups_dataframe['Group ID'][0]
print('First group ID: %s' % first_group_id)
Load the CPU Utilization for last 2 hours for the group with the ID first_group_id. The time series is further aggregated as follows:
aligned to 5 minute intervals using the 'ALIGN_MEAN' method.zone and instance_name pair is combined or reduced into a single time series. This is useful to combine the time series of instances that might be restarted / redeployed continuously.
In [4]:
# Initialize the query for the CPU Utilization metric over the last 2 hours.
query_group = gcm.Query('compute.googleapis.com/instance/cpu/utilization', hours=2)
# Filter the instances to the members of the first group.
query_group = query_group.select_group(first_group_id)
# Aggregate the time series.
query_group = query_group.align(gcm.Aligner.ALIGN_MEAN, minutes=5)
query_group = query_group.reduce(gcm.Reducer.REDUCE_MEAN, 'resource.zone', 'metric.instance_name')
# Create a dataframe with zone and instance name in the headers.
cpu_group_dataframe = query_group.as_dataframe(labels=['zone', 'instance_name'])
cpu_group_dataframe.tail(5)
Out[4]:
In [5]:
cpu_group_dataframe_per_zone = cpu_group_dataframe.groupby(level=0, axis=1).mean()
_ = cpu_group_dataframe_per_zone.plot().legend(loc='center left', bbox_to_anchor=(1.0, 0.8))
In [6]:
# Find all unique zones and sort them.
all_zones = sorted(set(cpu_group_dataframe.columns.get_level_values('zone')))
# Find the global min and max so we can set the same range for all y-axes.
min_cpu = cpu_group_dataframe.min().min()
max_cpu = cpu_group_dataframe.max().max()
for zone in all_zones:
zone_plot = cpu_group_dataframe[zone].plot(title=zone, ylim=(min_cpu, max_cpu))
zone_plot.legend(loc='center left', bbox_to_anchor=(1.0, 0.8))