Copyright 2014 by Andreas Klostermann andreas.klostermann@gmail.com
In this notebook I am going to present a way to update Graphs in a notebook dynamically during the execution of a code cell. There are two ways to do this, one is with the widget framework, the other works without the widget infrastructure.
Animating graphs in python has long been possible. Also animating matplotlib graphs inside the IPython notebook has been "modestly" possible by replacing the image with a newly rendered image. But on one hand matplotlib already is a bit too slow for realtime visualization (and it is not meant to be), on the other hand transmitting rendered bitmap data or even SVG code over the notebook websocket is hardly optimal either.
Bokeh is a plotting library which was expressly designed for interactive communication between python and javascript in the frontend. It achieves this by essentially replicating an object graph across the python side and the javascript side, and by making the data on the javascript side relatively easy to modify.
Bokeh has its own web server, which is used in all of the official animation examples. The server implements a certain "session" infrastructure, where modified data is recognized and synchronized with the client.
I think this server adds unneccessary complexity to running and publishing notebooks. Therefore I worked out the following ways of doing animations without the bokeh server, just in the notebook. It would probably be possible to use the bokeh's session framework for updating the data sources, but the manual implementation
The code in this notebook is in a very experimental state. I used Bokeh's GIT revision "dab61e9" from September 13th, and IPython 2.1.0 with an Anaconda distribution. PyMC 3 is revision "1f2a639", installed locally.
From experience I expect bokeh's API to change rapidly, and I may or may not have the time to update these notebooks. Bokeh is still in an experimental state itself, while showing great promise.
In [1]:
from __future__ import division
from bokeh.plotting import output_notebook
output_notebook()
from IPython.core import display
import numpy as np
from numpy import pi, cos, sin, linspace
In [2]:
import time
In [3]:
from bokeh.plotting import *
from bokeh.protocol import serialize_json
In [4]:
def update_data_source(ds):
ds_id = ds.ref['id']
ds_model = ds.ref['type']
js = """
var ds = Bokeh.Collections('{ds_model}').get('{ds_id}');
var data = {ds_json};
ds.set(data);
ds.trigger("change");
""".format(ds_model=ds_model, ds_id=ds_id, ds_json= serialize_json(ds.vm_serialize()))
display.display_javascript(js, raw=True)
BokehJS (and IPython notebook) internally use BackboneJs. This is a so called Model-View-Controller framework designed to communicate object-oriented data between web server and web client.
Bokeh and BokehJS both have an internal graph of objects. The first line of the javascript code above retrieves the datasource model, referencing it by the model's id which was generated and stored on Python's side of the Bokeh library.
The whole snippet of javascript is then transmitted to the notebook and executed immediately. It would be possible to use script tags and insert them into the output tree with display_html, but then every frame is saved into the notebook file and upon loading the notebook, it is executed again. See the PyMC example in this repository for a reason to use inline scripts anyway.
Please note that if you are reading this notebook outside of an active IPython Notebook session, the animations will not play and they may not look like anything.
In [5]:
figure(plot_width=800, plot_height=200)
x = np.linspace(0, 4*np.pi, 100)
y = np.sin(x*4)
line_model = line(x,y, color="#0000FF")
data_sources = [r.data_source for r in line_model.renderers if hasattr(r, 'data_source')]
show()
ds = data_sources[0]
for i in range(200):
ds.data['y'] = np.sin((x+i/14)*2)
update_data_source(ds)
time.sleep(0.05)
Widgets on the client side are Backbone models, on the IPython kernel's side they are represented by a IPython.html.widgets.DOMWidget subclass. These two objects are set up in a way that allows them to exchange messages.
This transport is arranged by IPython's kernel protocol. The IPython Kernel sends realtime updates through a websocket connection to the client, and the client can listen to these messages.
In this example I am not embedding bokeh's canvas into the widget's DOM representation. This would be possible, but it is not necessary.
In [6]:
%%javascript
require(["widgets/js/widget"], function(WidgetManager){
var AnimationWidget = IPython.WidgetView.extend({
render: function(){
var html = $("<button>Stop</button>");
this.setElement(html);
this.model.on('msg:custom', $.proxy(this.handle_custom_message, this));
$(html).click($.proxy(function(){
this.send({'msg_type':'stop'});
$(html).hide()
}, this))
},
handle_custom_message: function(data){
switch (data.custom_type){
case "replace_bokeh_data_source":
var ds = Bokeh.Collections(data.ds_model).get(data.ds_id);
ds.set($.parseJSON(data.ds_json));
ds.trigger("change");
break;
}
}
});
WidgetManager.register_widget_view('AnimationWidget', AnimationWidget)
});
A lot is going on here. First the call to "require" ensures that necessary library functions and classes are already loaded and available. This is done through the requireJS library. With "WidgetManager.register_widget_view", the widget class is registered as such.
AnimatiowWidget (on the JS side) is a class which extends from IPython.WidgetView. The render function is called when the widget is rendered (duh). Here we can set up some html, for example, or connect to buttons etc.
We have to handle a custom message over the widget's model instance, and there we do the same thing as before: Find the datasource, update it, and trigger a rerender.
In [7]:
from IPython.html import widgets
from IPython.utils.traitlets import Unicode
import json
class AnimationWidget(widgets.DOMWidget):
_view_name = Unicode('AnimationWidget', sync=True)
value = Unicode(sync=True)
def __init__(self, *args, **kwargs):
widgets.DOMWidget.__init__(self,*args, **kwargs)
self.iteration = 0
self.on_msg(self._handle_custom_msg)
self.stopped = False
def replace_bokeh_data_source(self, ds):
self.send({"custom_type": "replace_bokeh_data_source",
"ds_id": ds.ref['id'], # Model Id
"ds_model":ds.ref['type'], # Collection Type
"ds_json": serialize_json(ds.vm_serialize())
})
def start(self):
self.iteration = 0
self.stopped = False
def _handle_custom_msg(self, content):
if content['msg_type']=='stop':
self.stopped = True
def running(self):
self.iteration +=1
get_ipython().kernel.do_one_iteration()
return not self.stopped
The new widget also offers a "stop" button, and a running() method to exit cleanly when this button is pressed.
One thing is critical for this to work: The loop must give the kernel enough time to "breathe" and communicate with the client. Normally, while the code in the notebook cells is executed, no events are processed, no data synchronized (except one way from server to browser) and the button just would not work.
But with get_ipython().kernel.do_one_iteration() this problem is solved easily. To my knowledge, currently, this is not mentioned in the interactive widget documentation, but it is crucial interacting with longer running computations.
In [8]:
w = AnimationWidget()
display.display(w)
figure(plot_width=800, plot_height=200)
hold(False)
x = np.linspace(0, 4*np.pi, 100)
y = np.sin(x*4)
line_model = line(x,y, color="#0000FF")
data_sources = [r.data_source for r in line_model.renderers if hasattr(r, 'data_source')]
show()
ds = data_sources[0]
w.start()
while w.running():
ds.data['y'] = np.sin((x+w.iteration/14)*2)
w.replace_bokeh_data_source(ds)
time.sleep(0.05)
In [9]:
N= 30
M = 30
img = np.empty((M,N), dtype=np.uint32)
figure(plot_width=900, plot_height=300)
view = img.view(dtype=np.uint8).reshape((M, N, 4))
image_rgba(
image=[img], x=[0], y=[0], dw=[M], dh=[N],
x_range=[0,N], y_range=[0,M])
data_sources = [r.data_source for r in curplot().renderers if hasattr(r, 'data_source')]
ds = data_sources[0]
w = AnimationWidget()
display.display(w)
show()
w.start()
while w.running():
i = np.random.randint(0, M)
j = np.random.randint(0, N)
view[i, j, 0] = int(i/N*255)
view[i, j, 1] = 158
view[i, j, 2] = int(j/N*255)
view[i, j, 3] = 255
w.replace_bokeh_data_source(ds)
time.sleep(0.05)
In [1]:
from scipy.integrate import odeint
sigma = 10
rho = 28
beta = 8.0/3
theta = 3 * np.pi / 4
def lorenz(xyz, t):
x, y, z = xyz
x_dot = sigma * (y - x)
y_dot = x * rho - x * z - y
z_dot = x * y - beta* z
return [x_dot, y_dot, z_dot]
initial = (-10, -7, 35)
t = np.arange(0, 6, 0.006)
solution = odeint(lorenz, initial, t)
figure(plot_width=900, plot_height=300)
line(solution[:,0], solution[:,1],
line_color="#6BAED6", line_alpha=0.8, line_width=2.5)
data_sources = [r.data_source for r in curplot().renderers if hasattr(r, 'data_source')]
ds = data_sources[0]
w = AnimationWidget()
display.display(w)
show()
w.start()
while w.running():
t = np.arange(0, min([w.iteration/10.0,6]) , 0.006)
solution = odeint(lorenz, solution[2, :], t)
ds.data['x'] = solution[:,0]
ds.data['y'] = solution[:,1]
w.replace_bokeh_data_source(ds)
time.sleep(0.05)
In [ ]: