From Jupyter Notebooks to interactive dashboards and web applications

PyData Heidelberg - 2020-01-09

Jeremy Tuloup

https://github.com/QuantStack/quantstack-talks/2020-01-09-PyData-Heidelberg

Scientific Software Engineer

- quantstack.net

- github.com/jtpio

- twitter.com/jtpio

Jupyter Notebooks

Lorenz.ipynb

Jupyter Widgets

ipywidgets - Base Widgets


In [ ]:
from ipywidgets import HBox, IntSlider, IntText, link

slider = IntSlider(min=0, max=10)
slider

In [ ]:
slider

In [ ]:
slider.value

In [ ]:
text = IntText()

s = IntSlider()

link((text, 'value'), (s, 'value'))
HBox([text, s])

Rich ecosystem of third-party widgets

- bqplot: Interactive 2-D plotting

- ipyleaflet: Interactive maps

- ipycanvas: Interactive Canvas

- ipyvolume: Interactive 3-D plotting

- ipywebrtc, ipysheet, ipytree ...

bqplot


In [ ]:
import ipywidgets as widgets
import numpy as np

from bqplot import pyplot as plt

In [ ]:
n = 2000
x = np.linspace(0.0, 10.0, n)
np.random.seed(0)
y = np.cumsum(np.random.randn(n)*10).astype(int)

In [ ]:
fig_hist = plt.figure(title='Histogram')
hist = plt.hist(y, bins=25)
fig_hist

In [ ]:
slider = widgets.IntSlider(min=1, max=100, v_model=30)
widgets.link((hist, 'bins'), (slider, 'value'));
slider

ipyleaflet


In [ ]:
import os
import json

import numpy as np
import pandas as pd

from ipywidgets import Dropdown

from bqplot import Lines, Figure, LinearScale, DateScale, Axis

from ipyleaflet import Map, GeoJSON, WidgetControl, FullScreenControl

In [ ]:
data = pd.read_json(os.path.abspath('./examples/nations.json'))

In [ ]:
def clean_data(data):
    for column in ['income', 'lifeExpectancy', 'population']:
        data = data.drop(data[data[column].apply(len) <= 4].index)
    return data

def extrap_interp(data):
    data = np.array(data)
    x_range = np.arange(1800, 2009, 1.)
    y_range = np.interp(x_range, data[:, 0], data[:, 1])
    return y_range

def extrap_data(data):
    for column in ['income', 'lifeExpectancy', 'population']:
        data[column] = data[column].apply(extrap_interp)
    return data

In [ ]:
data = clean_data(data)
data = extrap_data(data)
date_start = pd.datetime(1800, 12, 31)
date_end = pd.datetime(2009, 12, 31)
date_scale = DateScale(min=date_start, max=date_end)
date_data = pd.date_range(start=date_start, end=date_end, freq='A', normalize=True)

country_name = 'Angola'
data_name = 'income'

x_data = data[data.name == country_name][data_name].values[0]

x_scale = LinearScale()

lines = Lines(x=date_data, y=x_data, scales={'x': date_scale, 'y': x_scale})

ax_x = Axis(label='Year', scale=date_scale, num_ticks=10, tick_format='%Y')
ax_y = Axis(label=data_name.capitalize(), scale=x_scale, orientation='vertical', side='left')

figure = Figure(axes=[ax_x, ax_y], title=country_name, marks=[lines], animation_duration=500,
                layout={'height': '300px', 'width': '600px'})

In [ ]:
def update_figure(country_name, data_name):
    lines.y = data[data.name == country_name][data_name].values[0]
    ax_y.label = data_name.capitalize()
    figure.title = country_name

In [ ]:
with open('./examples/countries.geo.json') as f:
    countries = json.load(f)

In [ ]:
m = Map(zoom=3)

geo = GeoJSON(data=countries, style={'fillColor': 'white', 'weight': 0.5}, hover_style={'fillColor': '#1f77b4'}, name='Countries')
m.add_layer(geo)

m.layout.height = '800px'
m

In [ ]:
m.add_control(FullScreenControl())

In [ ]:
widget_control1 = WidgetControl(widget=figure, position='bottomright')

m.add_control(widget_control1)

def on_hover(event, feature, **kwargs):
    global country_name

    country_name = feature['properties']['name']
    update_figure(country_name, data_name)

geo.on_hover(on_hover)

In [ ]:
dropdown = Dropdown(
    options=['income', 'population', 'lifeExpectancy'],
    value=data_name,
    description='Plotting:'
)

def on_click(change):
    global data_name

    data_name = change['new']
    update_figure(country_name, data_name)

dropdown.observe(on_click, 'value')

widget_control2 = WidgetControl(widget=dropdown, position='bottomleft')

m.add_control(widget_control2)

ipycanvas

Game of Life

Foundations for interactivity in Voilà

Jupyter Widgets Tutorial: https://github.com/jupyter-widgets/tutorial

Voilà

Remix of existing components

Builds on the strong foundations of the Jupyter Ecosystem

- jupyter_server

- nbconvert

Jupyter Protocol and Standard file format

- No changes needed to go from a notebook to a dashboard

- Jupyter Kernel still at the center of execution

Started under the QuantStack org, now an official Jupyter subproject

- Blog post: https://blog.jupyter.org/voil%C3%A0-is-now-an-official-jupyter-subproject-87d659583490

Creating a Voilà dashboard

basics.ipynb

dashboard.ipynb

Lorenz.ipynb

As a standalone app with the voila CLI

voila basics.ipynb

voila --help-all

voila basics.ipynb --theme=dark

JupyterLab Preview Extension

Classic Notebook Extension

Execution model

Voila prevents arbitrary code execution

Voilà Templates

Build modern web applications with Jupyter Notebooks

Voilà Material: Material design template

https://github.com/voila-dashboards/voila-material

gradient_descent.ipynb

voila examples/gradient_descent.ipynb --template=material

Voilà Vuetify: Material design template powered by VueJS

https://github.com/voila-dashboards/voila-vuetify

bqplot_vuetify.ipynb

voila --template vuetify-default examples/bqplot_vuetify.ipynb

Voilà Gridstack: Grid spec layout for Jupyter

GitHub: https://github.com/voila-dashboards/voila-gridstack

Blog post: https://blog.jupyter.org/voila-gridstack-template-8a431c2b353e

gridstack.ipynb

voila --template=gridstack examples/gridstack.ipynb --VoilaConfiguration.resources='{"gridstack": {"show_handles": True}}'

Voilà Embed: Embed jupyter widgets in existing websites

https://github.com/mariobuikhuizen/voila-embed

Deployment

ngrok

1. ngrok http 8866

2. Share the URL

3. Done

Heroku extension for JupyterLab

Voilà Gallery

voila-gallery.org

Live at: voila-gallery.org

GitHub: https://github.com/voila-gallery/voila-gallery.github.io

Roadmap

- WYSIWYG editor for JupyterLab (drag and drop)

- JupyterLab build system and extensions

- JupyterHub integration

- Debug Voilà dashboards in JupyterLab with the visual debugger - debugger.ipynb