Python --> IPython --> IPython Notebook (jupyter)

What, Why ?

  • Python = language interpreter --> programs, interactive sessions
  • IPython = interactive python --> enhanced interaction
    • separately installable extension for python
    • code completion --> development, debugging
    • shortcuts for os commands and (e.g. bash) scripting --> use as "extended" bash for system administrators
  • IPython notebook
    • currently part of IPython project - from next version on: separate jupyter project ("not only for python")

The IPython notebook (jupyter)

  • html frontend + protocol to interact with ipython server process(es)
  • recently language independent parts (frontend + protocol) maintained independently: jupyter project
    • different backend support: bash, python, julia, R, haskell ..

In [1]:
# Show ipython page 
from IPython.display import display, Image, HTML
HTML('<iframe src=http://ipython.org width=1000 height=400> </iframe>')


Out[1]:

IPython + Notebook interface --> "interactive computing environment"

  • "reproducable" scientific papers: code + comments + ..

Importance for us ?

Jim Gray et al. "Scientific Data Management in the coming Decade" (2005)

"The goal is a smart notebook that empowers scientists to explore the world’s data. Science data centers with computational resources to explore huge data archives will be central to enabling such notebooks. Because data is so large, and IO bandwidth is not keeping pace, moving code to data will be essential to performance. Consequently, science centers will remain the core vehicle and federations will likely be secondary. Science centers will provide both the archives and the institutional infrastructure to develop these peta-scale archives and the algorithms and tools to analyze them."


In [2]:
HTML('<iframe src=http://nbviewer.ipython.org/github/ipython/ipython-in-depth/blob/master/examples/Notebook/What%20is%20the%20IPython%20Notebook.ipynb width=1000 height=400></iframe>')


Out[2]:

Remark: Generation of presentations from notebooks:

(Thats how this presentation was generated ....)


In [ ]:
!ipython nbconvert --to slides ipython-intro.ipynb --post serve


[NbConvertApp] Converting notebook ipython-intro.ipynb to slides
[NbConvertApp] Writing 204261 bytes to ipython-intro.slides.html
[NbConvertApp] Redirecting reveal.js requests to https://cdn.jsdelivr.net/reveal.js/2.6.2
Serving your slides at http://127.0.0.1:8000/ipython-intro.slides.html
Use Control-C to stop this server

(process:9883): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
WARNING:tornado.access:404 GET /custom.css (127.0.0.1) 7.82ms
WARNING:tornado.access:404 GET /custom.css (127.0.0.1) 1.10ms
WARNING:tornado.access:404 GET /favicon.ico (127.0.0.1) 0.93ms

New paragraph

This is rich text with links, equations:

$$\hat{f}(\xi) = \int_{-\infty}^{+\infty} f(x)\, \mathrm{e}^{-i \xi x}$$

code with syntax highlighting:

print("Hello world!")

and images:


In [1]:
%lsmagic


Out[1]:
Available line magics:
%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %install_default_config  %install_ext  %install_profiles  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

In [2]:
from IPython.display import HTML, SVG
HTML('''
         <table style="border: 2px solid black;">
         ''' + 
         ''.join(['<tr>' + 
                  ''.join(['<td>{row},{col}</td>'.format(
                                 row=row, col=col
                                 ) for col in range(5)]) +
                  '</tr>' for row in range(5)]) +
         '''
         </table>
         ''')


Out[2]:
0,00,10,20,30,4
1,01,11,21,31,4
2,02,12,22,32,4
3,03,13,23,33,4
4,04,14,24,34,4

Interactive widgets: Example


In [3]:
from IPython.html.widgets import interact
import matplotlib.pyplot as plt
import networkx as nx
%matplotlib inline


:0: FutureWarning: IPython widgets are experimental and may change in the future.

In [4]:
# wrap a few graph generation functions so they have the same signature

def random_lobster(n, m, k, p):
    return nx.random_lobster(n, p, p / m)

def powerlaw_cluster(n, m, k, p):
    return nx.powerlaw_cluster_graph(n, m, p)

def erdos_renyi(n, m, k, p):
    return nx.erdos_renyi_graph(n, p)

def newman_watts_strogatz(n, m, k, p):
    return nx.newman_watts_strogatz_graph(n, k, p)

def plot_random_graph(n, m, k, p, generator):
    g = generator(n, m, k, p)
    nx.draw(g)
    plt.show()

In [5]:
interact(plot_random_graph, n=(2,30), m=(1,10), k=(1,10), p=(0.0, 1.0, 0.001),
        generator={'lobster': random_lobster,
                   'power law': powerlaw_cluster,
                   'Newman-Watts-Strogatz': newman_watts_strogatz,
                   u'Erdős-Rényi': erdos_renyi,
                   });



In [ ]: