Outline

Starting a notebook
- profile
- common options
- remote machine
Securing remote notebooks
Running notebooks on compute nodes

Starting and Running Python notebooks

ipython notebook --no-mathjax

Options relevant for remote notebooks

--no-browser: don't open the notebook in a browser after startup
--ip: The IP address the notebook server will listen on
--port: the port the notebook server will listen on

$ ssh thauser@comet.sdsc.edu
gordon$ hostname
   comet-ln3.sdsc.edu
[thauser@comet-ln3 ~]$ ipython notebook --no-browser --port=9088 --ip=*

Connecting to the notebook server

http://comet-ln3.sdsc.edu:9088

Securing the remote notebook

There are several steps to secure the notebook server on a remote machine (see http://ipython.org/ipython-doc/dev/notebook/public_server.html)

Create a custom notebook profile
Create a self signed certificate to access the remote notebook with https
Create a simple password
Edit the notebook profile to specify the security settings

There is a bug in the combination of Safari and tornado that may not allow you to run the remote notebook properly: Use a different browser

1. Creating a notebook profile

A profile allows you to manage different configurations for your notebook

comet$ ipython profile create nbserver

This will create .ipython/profile_nbserver directory



In [4]:

    
!ssh thauser@thauser@comet.sdsc.edu 'cd .ipython/profile_nbserver; ls -al'









    



total 144
drwxr-xr-x 8 thauser cob113  4096 Jul 14 03:25 .
drwxr-xr-x 6 thauser cob113  4096 Jul 13 17:42 ..
drwxr-xr-x 2 thauser cob113  4096 Jul 13 17:58 db
-rw-r--r-- 1 thauser cob113  7168 Jul 14 03:25 history.sqlite
-rw-r--r-- 1 thauser cob113 19915 Jul 13 17:39 ipython_config.py
-rw-r--r-- 1 thauser cob113 31841 Jul 13 17:39 ipython_nbconvert_config.py
-rw-r--r-- 1 thauser cob113 24504 Jul 13 17:50 ipython_notebook_config.py
-rw-r--r-- 1 thauser cob113 24679 Jul 13 17:39 ipython_qtconsole_config.py
drwxr-xr-x 2 thauser cob113  4096 Jul 13 17:39 log
drwx------ 2 thauser cob113  4096 Jul 13 17:39 pid
drwx------ 2 thauser cob113  4096 Jul 14 03:25 security
drwxr-xr-x 2 thauser cob113  4096 Jul 13 17:39 startup
drwxr-xr-x 3 thauser cob113  4096 Jul 13 17:39 static

2. Create a self-signed certificate

comet$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

3. Create a password hash

In ipython execute



In [6]:

    
from IPython.lib import passwd

passwd('test password')









    Out[6]:





'sha1:767cf154e39a:b4f204a629dc83c06d8d799505941372ee0347df'

Not recommended: Use interactive ipython and then just passwd()

4. Edit the configuration for your profile

vim .ipython/profile_nbserver/ipython_notebook_config.py



In [10]:

    
!ssh thauser@gordon.sdsc.xsede.org 'head -n 12 .ipython/profile_nbserver/ipython_notebook_config.py'









    



# Configuration file for ipython-notebook.

c = get_config()
# Kernel config
c.IPKernelApp.pylab = 'inline'  # if you want plotting support always
# Notebook config
c.NotebookApp.certfile = u'/home/thauser/.ipython/mycert.pem'
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = u'sha1:4723172689ea:003c3bcd8ba0bf4b8b4d3b0647a2be7bf7bd7216'
c.NotebookApp.port = 11111

Running the secured remote notebook

ipython notebook --profile=nbserver

connect to

https://comet-ln3.sdsc.edu:11111

you need to enter your password

Running the notebook in an interactive job on Comet

Start your interactive job
```
comet$ srun -u -t 10 /bin/bash -i
```

Start your notebook

comet-04-45$ hostname
 comet-04-45.sdsc.edu
comet-04-45$ ipython notebook --profile=nbserver

Connect to it

https://comet-04-45.sdsc.edu:11111

Port forwarding using ssh

On some machines compute nodes maybe not accessible from the outside

ssh port forwarding

On your local machine:

#ssh -L localport:nodename:remote_port -f -N comet.sdsc.edu

$ ssh -L 8088:comet-04-45:9088 -f -N comet comet.sdsc.edu

open your localbrowser at https://localhost:8088

Using SAGA to run a job on a remote node of a cluster

A light-weight access layer for distributed computing infrastructure

http://saga-project.github.io/saga-python/

Install on your local machine

$ pip install saga-python

Create a saga script, e.g



In [5]:

    
import sys
import time
import saga

# Adapted from the saga example
# Your ssh identity on the remote machine.
ctx = saga.Context("ssh")
ctx.user_id = 'thauser'

session = saga.Session()
session.add_context(ctx)

# Create a job service object that represent a remote pbs cluster.
js = saga.job.Service("slurm+ssh://comet.sdsc.edu", session=session)

# Set the parameters for this example
local_port=9988
remote_port=11111
username='thauser'
hostname='comet.sdsc.edu'

# Next, we describe the job we want to run. A complete set of job
# description attributes can be found in the API documentation.
jd = saga.job.Description()
jd.wall_time_limit   = 10 # minutes
jd.executable        = "ipython notebook --profile=nbserver"
jd.queue             = "compute"
jd.working_directory = "A"
jd.output            = "ipythonjob.out"
jd.error             = "ipythonjob.err"



In [6]:

    
touchjob = js.create_job(jd)
# Check our job's id and state
print "Job ID    : %s" % (touchjob.id)
print "Job State : %s" % (touchjob.state)

# Now we can start our job.
print "\n...starting job...\n"
touchjob.run()

print "Job ID    : %s" % (touchjob.id)
print "Job State : %s" % (touchjob.state)

# List all jobs that are known by the adaptor.
# This should show our job as well.
print "\nListing active jobs: "
for job in js.list():
    print " * %s" % job









    



Job ID    : None
Job State : New

...starting job...

Job ID    : [slurm+ssh://comet.sdsc.edu]-[714345]
Job State : Pending

Listing active jobs: 
 * [slurm+ssh://comet.sdsc.edu]-[714345]



In [7]:

    
# Now we disconnect and reconnect to our job by using the get_job()
# method and our job's id. While this doesn't make a lot of sense
# here,  disconnect / reconnect can become very important for
# long-running job.

touchjob_clone = js.get_job(touchjob.id)

print touchjob_clone.state

while touchjob_clone.state == 'Pending':
    print "...Waiting for Job to start...."
    time.sleep(30)

nodename = touchjob_clone.execution_hosts[0]
nodename = nodename[:-2]

touchjob_clone.wait()

print "Job State   : %s" % (touchjob_clone.state)
print "Exitcode    : %s" % (touchjob_clone.exit_code)
js.close()









    



Running






    



---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-7-70335a1a759a> in <module>()
     16 nodename = nodename[:-2]
     17 
---> 18 touchjob_clone.wait()
     19 
     20 print "Job State   : %s" % (touchjob_clone.state)

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/saga/job/job.pyc in wait(self, timeout, ttype)
    570             timeout = -1.0 # FIXME
    571 
--> 572         return self._adaptor.wait (timeout, ttype=ttype)
    573 
    574 

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/saga/adaptors/cpi/decorators.pyc in wrap_function(self, *args, **kwargs)
     55                 del(kwargs['_from_task'])
     56 
---> 57         return sync_function (self, *args, **kwargs)
     58 
     59     return wrap_function

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/saga/adaptors/slurm/slurm_job.pyc in wait(self, timeout)
   1090                     return True
   1091             # avoid busy poll
-> 1092             time.sleep(0.5)
   1093 
   1094             # check if we hit timeout

KeyboardInterrupt:



In [8]:

    
touchjob.id









    Out[8]:





'[slurm+ssh://comet.sdsc.edu]-[714345]'



In [9]:

    
js.get_job(touchjob.id)









    Out[9]:





<saga.job.job.Job at 0x10b2dbe90>



In [10]:

    
touchjob_clone = js.get_job(touchjob.id)



In [11]:

    
print touchjob_clone.state









    



Running



In [13]:

    
touchjob_clone.execution_hosts









    Out[13]:





'comet-16-14'



In [ ]: