Computer Lab 2, Part I

This notebook consists of instructions, exercises and questions that form the practical part of Lab II, Part I. In this assignment, you will learn the basics of the OpenStack Python APIs that can be used to interact directly with the IaaS services Nova (compute) and Swift (Object Store). Please prepare your solution and answers to questions directly in this notebook, and export it to PDF. Upload that PDF as to the student portal to complete Part I of the Lab.



In [1]:

    
import os
import swiftclient.client









    



---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-bfd001a53f06> in <module>()
      1 import os
----> 2 import swiftclient.client

ImportError: No module named swiftclient.client

To establish a client connection, we will need to pass a dictionary with information about the tenant, user, credentials and the API Identity endpoint. Here, I have sourced the "openrc.sh file" obtained from the Horizon dashboard in the underlying shell prior to starting the notebook. Hence, in order to actually run the code below, you would need to do the same with your own credentials.



In [2]:

    
config = {'user':os.environ['OS_USERNAME'], 
          'key':os.environ['OS_PASSWORD'],
          'tenant_name':os.environ['OS_TENANT_NAME'],
          'authurl':os.environ['OS_AUTH_URL']}

First, we obtain a client connection to Swift (we are usign the v2 APIs)



In [3]:

    
conn = swiftclient.client.Connection(auth_version=2, **config)



In [4]:

    
# Create a container, use a UUID to make sure it has a globally unique name. 
import uuid
bucket_name = "lab2_{0}".format(str(uuid.uuid4()))
conn.put_container(bucket_name)

Question 1:

What does it mean that the object store has a global, flat namespace? What is the practical consequence for you when using it?



In [ ]:

    
# List containers
(response, bucket_list) = conn.get_account()
for bucket in bucket_list:
    print bucket['name']



In [12]:

    
# Put an object in the container
object_id = conn.put_object(bucket_name, "test_object", "Hi Swift")

Excercise 1:

Try to measure the speed with which you can put and get objects to and from Swift usign the API. Conduct your experiment several times to gather statistic and plot a) A estimated distribution of the time taken (in wall clock) to put and read an object of size 10MB in your swift container and b) vary the size of the object from 10kB to 100MB and plot the put and get throughput (in MB/s) times as a function of object size (for the smaller data sizes, you might need to repeat the experiment many times and obtain a statistical average). Include the resulting graphs and a description of your experiment in the report.



In [7]:

    
# Implement you solution here. Hint, the following command
%pylab inline 
# will  make matplotlib/pylab available and plots will be displayed directly in the notebook, for example
plt.plot([0,1,2,3],[0,1,2,3])









    



Populating the interactive namespace from numpy and matplotlib






    Out[7]:





[<matplotlib.lines.Line2D at 0x10d2893d0>]

Excercise 2:



In [ ]:

    
# Obtain a list of all the object names in your container.

In the cell below, we obtain a client connection to the Nova endpoint. It can be used to for example start, stop and terminate instances.



In [55]:

    
config = {'username':os.environ['OS_USERNAME'], 
          'api_key':os.environ['OS_PASSWORD'],
          'project_id':os.environ['OS_TENANT_NAME'],
          'auth_url':os.environ['OS_AUTH_URL'],
           }
from novaclient.client import Client
nc = Client('2',**config)

Excercise 3:

Boot a new instance (hint, look client.server in the API docs) with flavor 'm1.medium' (remember to provide an ssh-key so that you can access it later). In booting the instance, use the mechanism of 'user_data' (learn about this in the openstack and 'cloud-init' documentations) to provide a startup-script to 1. Update the instance, 2. install 'git', 'cowsay' and 'flask'.



In [16]:

    
# Use paramiko to access your instance and, using ssh, start the cowsay service on your instance,
# usinf the same command as in Task 4, lab 1. 
import paramiko



In [ ]:

    
# Make a request to the cowsay REST API and display the response inline in the notebook

Question 2:

The above excersise showed a low-level way of 'contextualization' using user data 'cloud-init'. Do some reserach online and discuss alternative tools and techniques for contextualization of your VMs. Discuss the difference between instance meta-data and user-data. Some links to get you started:

http://docs.openstack.org/user-guide/cli_provide_user_data_to_instances.html http://cernvm.cern.ch/portal/contextualisation https://cloudinit.readthedocs.org/en/latest/

Aim for the equivalent of ~1/2 page of an A4 paper, 12pt font, 2cm margins.

Excersise 4:

Use the Swift and Nova APIs to terminate your instance, to delete all the objects from your bucket, and then finally to delete the container.



In [17]:

    
# Clean up container in Swift



In [18]:

    
# Terminate all your running instances