

In [1]:

    
import os,sys
sys.path.insert(0, os.path.abspath('..'))

from hublib.ui import Submit
import numpy as np
import ipywidgets as w
import re, glob

import plotly.plotly as py
import plotly.graph_objs as go
from plotly.offline import iplot, init_notebook_mode
init_notebook_mode(connected=True)

Jupyter Submit Demo

The Submit widget on Jupyter works with the command-line submit command on the hubs. It adds several new features.

Output is captured on a collapsible scrolling window.
Progress bars show graphical status
Optional local caching

Parametric Run

This example shows a parametric run where submit creates a series of jobs with different values for an input parameter.

How Local Caching Works

The output of each unique run is stored in a directory in

~/data/results/.submit_cache/${cachename}/${runname}

cachename should normally be the same as the toolname for published tools. Otherwise use something unique. It is set in the Submit() call. If you don't set cachename, caching is disabled.

runname is a unique identifier for each run. If you have a simulation that has only 50 possible input values, you can use something simple like "Run01" to "Run50". But that can get complicated for many different input values of different types. make_rname() is a convenient function that takes a variable number of arguments of different types and produces a unique string output that will be the name of the cache directory.

In the example below, the start callback calles make_rname() with the values from the two input widgets.

rname = s.make_rname(vin.value, steps.value)

If caching is disabled, runname can be anything you choose. It will overwrite any previous directory with that name.



In [2]:

    
def my_start(s):
    rname = s.make_rname(vin.value, steps.value)
    submit_str = '--local -p "@@Vin=%d:%d in %d" /apps/pegtut/current/examples/capacitor_voltage/sim1.py  --Vin=@@Vin' % (vin.value[0], vin.value[1], steps.value)
    s.run(rname, submit_str)



In [3]:

    
regex = re.compile(r"--Vin=(\d*\.\d+|\d+)")

# after the run finished, plot the data

def plot_data(s, rdir):
    out.clear_output()
    with out:
        traces = []
        i = 1
        while True:
            # our simulation writes to 'out.log'
            if not os.path.isfile('%s/%02i/out.log' % (rdir, i)):
                break
            stdout = glob.glob('%s/%02i/*.stdout' % (rdir, i))[0]
            with open(stdout) as f:
                vin = re.search(regex, f.read()).group(1)
            vals = np.loadtxt('%s/%02i/out.log' % (rdir, i))
            t = go.Scatter(
                x = vals[:,0],
                y = vals[:,1],
                mode = 'lines',
                name = '%.2f volts' % float(vin)
            )            
            traces.append(t)
            i += 1
        iplot(traces)



In [4]:

    
vin = w.FloatRangeSlider(
    value=[0, 50],
    min=0,
    max=50,
    step=1,
    description='Vin:',
    readout_format='2d',
)
steps = w.BoundedIntText(
    value=11,
    min=2,
    max=50,
    step=1,
    description='Steps:',
)



In [5]:

    
# output plot will go in this cell
out = w.Output()
out



In [6]:

    
w.VBox([vin, steps])



In [7]:

    
Submit(start_func=my_start, done_func=plot_data, cachename='SubmitTest1')