

In [ ]:

    
sudo yum -y install git
git clone https://github.com/linhbngo/Distributed-and-Cluster-Computing.git
    
Always do git pull at the beginning of the class

If you have conflicts with git pull:

    git stash save
    git pull

Important

This notebook is to be run inside Jupyter. If you see In [ ]: to the left of a cell, it means that this is an executable Jupyter cell.

To run a Jupyter cell, one of the followings can be done:

Press the Run button in the tool bar
Hit Shift-Enter
Hit Ctrl-Enter

In an executable Jupyter cell, the ! denotes a Linux command (or a sequence of commands) that will be sent to execute in the CentOS VM. All Linux commands in shell will assume a starting directory that is the current directory of the notebook.

In an executable Jupyter cell, the %% at the first line of the cell denotes a cell magic (a single configuration option that directs how the cell is executed). %%writefile is a cell magic that instruct Jupyter to not execute the remainder of the cell, but to save them in to a file whose path is specified after the cell magic.

Step 1. Set up emulab geni-lib package for CloudLab

Open a new terminal and run the following command:

$ sudo yum install -y unzip
$ conda create -n python2 python=2.7
$ source activate python2
$ conda install ipykernel
$ python -m ipykernel install --name python2 --user
$ conda install lxml

Restart your Jupyter Server
Reopen this notebook, go to Kernel, and change Kernel to Python 2



In [ ]:

    
!unzip codes/cloudlab/emulab-0.9.zip -d codes/cloudlab



In [ ]:

    
!cd codes/cloudlab/emulab-geni-lib-1baf79cf12cb/;\
    source activate python2;\
    python setup.py install --user



In [ ]:

    
!ls /home/lngo/.local/lib/python2.7/site-packages/



In [ ]:

    
!rm -Rf codes/cloudlab/emulab-geni-lib-1baf79cf12cb/

Step 2: Reload geni-lib for the first time

On the top bar of this notebook, select Kernel and then Restart



In [ ]:

    
%%writefile codes/cloudlab/xenvm.py

"""An example of constructing a profile with a single Xen VM from CloudLab.

Instructions:
Wait for the profile instance to start, and then log in to the VM via the
ssh port specified below.  (Note that in this case, you will need to access
the VM through a high port on the physical host, since we have not requested
a public IP address for the VM itself.)
"""

import geni.portal as portal
import geni.rspec.pg as rspec

# Create a Request object to start building the RSpec.
request = portal.context.makeRequestRSpec()
# Create a XenVM
node = request.XenVM("node")
node.disk_image = "urn:publicid:IDN+emulab.net+image+emulab-ops:CENTOS7-64-STD"
node.routable_control_ip = "true"

node.addService(rspec.Execute(shell="/bin/sh",
                              command="sudo yum update"))
node.addService(rspec.Execute(shell="/bin/sh",
                              command="sudo yum install -y httpd"))
node.addService(rspec.Execute(shell="/bin/sh",
                              command="sudo systemctl restart httpd.service"))
# Print the RSpec to the enclosing page.
portal.context.printRequestRSpec()

Step 3: Test emulab geni-lib installation

Executing the cell below should produce an XML element with the following content:

<rspec xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1" xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1" xmlns:jacks="http://www.protogeni.net/resources/rspec/ext/jacks/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.geni.net/resources/rspec/3" xsi:schemaLocation="http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/request.xsd" type="request">
  <rspec_tour xmlns="http://www.protogeni.net/resources/rspec/ext/apt-tour/1">
    <description type="markdown">An example of constructing a profile with a single Xen VM.</description>
    <instructions type="markdown">Wait for the profile instance to start, and then log in to the VM via the
ssh port specified below.  (Note that in this case, you will need to access
the VM through a high port on the physical host, since we have not requested
a public IP address for the VM itself.)
</instructions>
  </rspec_tour>
  <node client_id="node" exclusive="false">
    <sliver_type name="emulab-xen"/>
  </node>
</rspec>



In [ ]:

    
!source activate python2;\
  python codes/cloudlab/xenvm.py

If things fail, where to go?

/var/tmp/start-up*

The geni-lib library helps validate the syntatical correctness of your python-based experimental design
It is similar to Amazon AWS' Cloud Formation
It does not help validate the structural correctness of your experiment upon deployment
Manual modifications of design codes and experimental profiles are time-consuming
CloudLab supports linking to experimental codes located on Github, and the experimental profiles will automatically update via git's web-hooks

Fork the following github repository: https://github.com/lbstoller/my-profile into your Github account
Create a new experimental profile that points to the forked repository in your account
Setup web-hook
Instantiate an experiment using the original version of the profile

Example of designing an experimental profile for Cloud Computing infrastructure on CloudLab using geni-lib

Sets the template for future assignments in this class
Is based on CloudLab's default stack profile
https://github.com/linhbngo/cloudlab_openstack