Multilanguage Architecture



In [ ]:

    
from IPython.display import display, Image, HTML
from talktools import website, nbviewer

Overview

Data science and scientific computing is a multi-language activity. We love Python, but insisting that everyone write everything in Python all the time is not realistic. The reality is that most large, complex projects involve multiple languages. In fact, it is commmon for projects to use at least three classes of languages:

Low-level (C, C++, Java, Fortran)
High-level (Python, Ruby, Perl, Julia, Bash, R)
Web-based (JavaScript, CoffeeScript, HTML, CSS)

The goal of the Jupyter project is to provide a system for interactive computing that spans all of these languages.

It does this through two abstractions:

Kernel Architecture

Notebook documents

Kernel architecture

Jupyter has a powerful, open architecture for executing code interactively across networks:

Kernel: runs code in a single language and returns results over a network.
App: interactive user interface that enables the user to type code and see results. For example, the Notebook App.
Interactive Computing Protocol: JSON/WebSocket/ZeroMQ based network protocol that allows kernels and apps to talk.

The default kernel for Jupyter is the IPython kernel and runs Python code. However, there are now kernels in other languages (see below).



In [ ]:

    
Image("images/AppKernel.png")

A single app, like the Notebook, can talk to kernels in different languages:



In [ ]:

    
Image("images/MultilanguageKernels.png")

There are now Jupyter kernels for the following languages:

Python 2 and 3 (https://github/ipython)
Julia (https://github.com/JuliaLang/IJulia.jl)
R (https://github.com/takluyver/IRkernel)
Ruby (https://github.com/minrk/iruby)
Haskell (https://github.com/gibiansky/IHaskell)
Scala (https://github.com/Bridgewater/scala-notebook)
node.js (https://gist.github.com/Carreau/4279371)
Go (https://github.com/takluyver/igo)
Bash (https://github.com/takluyver/bash_kernel)
Others already and on the way

Magic commands

The standard IPython kernel also allows running code in other languages using the %%magic syntax.

Here is a bit of Ruby:



In [ ]:

    
%%ruby
puts "Hello from Ruby #{RUBY_VERSION}"

And some bash:



In [ ]:

    
%%bash
echo "Hello from $BASH"

Let's fit a linear model in R and visualize the results:



In [ ]:

    
import numpy as np
X = np.array([0,1,2,3,4])
Y = np.array([3,5,4,6,7])
%load_ext rmagic

The %%R syntax tells IPython to run the rest of the cell as R code:



In [ ]:

    
%%R -i X,Y -o XYcoef
XYlm = lm(Y~X)
XYcoef = coef(XYlm)
print(summary(XYlm))
par(mfrow=c(2,2))
plot(XYlm)

This %%language syntax is an IPython specific extension to the Python language and only works in the IPython kernel. This "magic command syntax" allows Python code to call out to a wide range of other languages (Ruby, Bash, Julia, Fortran, Perl, Octave, Matlab, etc.)

Notebook documents

Notebook documents are JSON files that store code, output, text, images, HTML, etc. The notebook as a whole and individual cells have metadata associated with them. This metadata encodes which programming language the code cells are in. All other parts of the Notebook document format (output, images, equations, etc.) are completely language neutral.

Notebooks created in any language will work with the rest of our notebook architecture:

nbconvert
nbviewer

Jupyter notebooks are an **open**, **multilanguage** format for recording and sharing an interactive computation and results.

Here is a notebook that was created with the Julia kernel and then shared on nbviewer:



In [ ]:

    
website("http://nbviewer.ipython.org/url/jdj.mit.edu/~stevenj/IJulia%20Preview.ipynb")

Here is a native R notebook based on an article at 538:



In [ ]:

    
website('http://nbviewer.ipython.org/github/ellisonbg/data/blob/r-notebook/bechdel/analyze-bechdel.ipynb')

Styling



In [ ]:

    
HTML("""
<style>
.idea-box {
  text-align: center;
  padding: 20px;
  margin: 10px;
}
</style>
""")



In [ ]: