Jupyter Notebook

Jupyter Notebook that evolved out of iPython and is aimed at providing a platform for easy sharing, interaction, and development of open-source software, standards and services. Althought primarily and originally used for phyton interactions, you can interact with multiple programming languages using Jupyter (although orignially expanded from iPython to includ Julia, Pythion, and R). Jupyter notebooks is a great way to create, test, and share live code, as well as equations and visualizations with markdown text.

To inlcude more languages, check out kernels that are supported and their installation instructions.

Installation

To get started, download and install Jupyter. Installation of Anaconda is reccomened by Jupyter becuase it will also cover your python and notebook installation at the same time. If you prefer, you can also install Jupyter with pip, but this is only suggested for advanced Python users. There is extensive documentation to help with installation and download availble on the Jupyter and Anaconda pages.

Once installed, launch the Anacdona Navigator and launch Jupter notebooks. Also open the Anaconda Prompt window. We will use this window to install extra widgets.

In your Anaconda Prompt window, type the statement below and press enter, continue by following the prompts.

conda install -c conda-forge ipyleaflet

In your Anaconda Prompt window, type the statement below and press enter, continue by following the prompts.

conda install -c conda-forge bqplot

In your Anaconda Prompt window, type the statement below and press enter, continue by following the prompts.

conda install -c conda-forge pythreejs

Check out this cheatsheet by DataCamp to help with the interface.

Help Dropdown

Spend some time going through these help features.

Markdown

Markdown can be easily used in Jupyter. Simply change the dropdown for the cell to "Markdown". If you need a refresher, here is another good cheatsheet.

Special Commands

Special commands can change shell to run different commands, for example you can run bash commands and magic commands directly within a new cell.


In [44]:
!conda list


# packages in environment at /Users/ldsfoxmac/anaconda:
#
_license                  1.1                      py36_1  
alabaster                 0.7.10                   py36_0  
anaconda                  custom                   py36_0  
anaconda-client           1.6.3                    py36_0  
anaconda-navigator        1.6.2                    py36_0  
anaconda-project          0.6.0                    py36_0  
appnope                   0.1.0                    py36_0  
appscript                 1.0.1                    py36_0  
asn1crypto                0.22.0                   py36_0  
astroid                   1.4.9                    py36_0  
astropy                   1.3.2               np112py36_0  
babel                     2.4.0                    py36_0  
backports                 1.0                      py36_0  
beautifulsoup4            4.6.0                    py36_0  
bitarray                  0.8.1                    py36_0  
blaze                     0.10.1                   py36_0  
bleach                    1.5.0                    py36_0  
bokeh                     0.12.5                   py36_1  
boto                      2.46.1                   py36_0  
bottleneck                1.2.1               np112py36_0  
cffi                      1.10.0                   py36_0  
chardet                   3.0.3                    py36_0  
click                     6.7                      py36_0  
cloudpickle               0.2.2                    py36_0  
clyent                    1.2.2                    py36_0  
colorama                  0.3.9                    py36_0  
conda                     4.3.29                   py36_0    conda-forge
conda-env                 2.6.0                         0    conda-forge
contextlib2               0.5.5                    py36_0  
cryptography              1.8.1                    py36_0  
curl                      7.52.1                        0  
cycler                    0.10.0                   py36_0  
cython                    0.25.2                   py36_0  
cytoolz                   0.8.2                    py36_0  
dask                      0.14.3                   py36_1  
datashape                 0.5.4                    py36_0  
decorator                 4.0.11                   py36_0  
distributed               1.16.3                   py36_0  
docutils                  0.13.1                   py36_0  
entrypoints               0.2.2                    py36_1  
et_xmlfile                1.0.1                    py36_0  
fastcache                 1.0.2                    py36_1  
flask                     0.12.2                   py36_0  
flask-cors                3.0.2                    py36_0  
freetype                  2.5.5                         2  
get_terminal_size         1.0.0                    py36_0  
gevent                    1.2.1                    py36_0  
greenlet                  0.4.12                   py36_0  
h5py                      2.7.0               np112py36_0  
hdf5                      1.8.17                        1  
heapdict                  1.0.0                    py36_1  
html5lib                  0.999                    py36_0  
icu                       54.1                          0  
idna                      2.5                      py36_0  
imagesize                 0.7.1                    py36_0  
ipykernel                 4.6.1                    py36_0  
ipyleaflet                0.4.0                    py36_0    conda-forge
ipython                   5.3.0                    py36_0  
ipython_genutils          0.2.0                    py36_0  
ipywidgets                7.0.1                      py_2    conda-forge
isort                     4.2.5                    py36_0  
itsdangerous              0.24                     py36_0  
jbig                      2.1                           0  
jdcal                     1.3                      py36_0  
jedi                      0.10.2                   py36_2  
jinja2                    2.9.6                    py36_0  
jpeg                      9b                            0  
jsonschema                2.6.0                    py36_0  
jupyter                   1.0.0                    py36_3  
jupyter_client            5.0.1                    py36_0  
jupyter_console           5.1.0                    py36_0  
jupyter_core              4.3.0                    py36_0  
lazy-object-proxy         1.2.2                    py36_0  
libiconv                  1.14                          0  
libpng                    1.6.27                        0  
libtiff                   4.0.6                         3  
libxml2                   2.9.4                         0  
libxslt                   1.1.29                        0  
llvmlite                  0.18.0                   py36_0  
locket                    0.2.0                    py36_1  
lxml                      3.7.3                    py36_0  
markupsafe                0.23                     py36_2  
matplotlib                2.0.2               np112py36_0  
mistune                   0.7.4                    py36_0  
mkl                       2017.0.1                      0  
mkl-service               1.1.2                    py36_3  
mpmath                    0.19                     py36_1  
msgpack-python            0.4.8                    py36_0  
multipledispatch          0.4.9                    py36_0  
navigator-updater         0.1.0                    py36_0  
nbconvert                 5.1.1                    py36_0  
nbformat                  4.3.0                    py36_0  
networkx                  1.11                     py36_0  
nltk                      3.2.3                    py36_0  
nose                      1.3.7                    py36_1  
notebook                  5.0.0                    py36_0  
numba                     0.33.0              np112py36_0  
numexpr                   2.6.2               np112py36_0  
numpy                     1.12.1                   py36_0  
numpydoc                  0.6.0                    py36_0  
odo                       0.5.0                    py36_1  
olefile                   0.44                     py36_0  
openpyxl                  2.4.7                    py36_0  
openssl                   1.0.2l                        0  
packaging                 16.8                     py36_0  
pandas                    0.20.1              np112py36_0  
pandocfilters             1.4.1                    py36_0  
partd                     0.3.8                    py36_0  
path.py                   10.3.1                   py36_0  
pathlib2                  2.2.1                    py36_0  
patsy                     0.4.1                    py36_0  
pep8                      1.7.0                    py36_0  
pexpect                   4.2.1                    py36_0  
pickleshare               0.7.4                    py36_0  
pillow                    4.1.1                    py36_0  
pip                       9.0.1                    py36_1  
ply                       3.10                     py36_0  
prompt_toolkit            1.0.14                   py36_0  
psutil                    5.2.2                    py36_0  
ptyprocess                0.5.1                    py36_0  
py                        1.4.33                   py36_0  
py-d3                     0.2.7                     <pip>
pycosat                   0.6.2                    py36_0  
pycparser                 2.17                     py36_0  
pycrypto                  2.6.1                    py36_6  
pycurl                    7.43.0                   py36_2  
pyflakes                  1.5.0                    py36_0  
pygments                  2.2.0                    py36_0  
pylint                    1.6.4                    py36_1  
pyodbc                    4.0.16                   py36_0  
pyopenssl                 17.0.0                   py36_0  
pyparsing                 2.1.4                    py36_0  
pyqt                      5.6.0                    py36_1  
pytables                  3.3.0               np112py36_0  
pytest                    3.0.7                    py36_0  
python                    3.6.1                         2  
python-dateutil           2.6.0                    py36_0  
python.app                1.2                      py36_4  
pythreejs                 0.3.0                    py36_0    conda-forge
pytz                      2017.2                   py36_0  
pywavelets                0.5.2               np112py36_0  
pyyaml                    3.12                     py36_0  
pyzmq                     16.0.2                   py36_0  
qt                        5.6.2                         2  
qtawesome                 0.4.4                    py36_0  
qtconsole                 4.3.0                    py36_0  
qtpy                      1.2.1                    py36_0  
readline                  6.2                           2  
requests                  2.14.2                   py36_0  
rope                      0.9.4                    py36_1  
ruamel_yaml               0.11.14                  py36_1  
scikit-image              0.13.0              np112py36_0  
scikit-learn              0.18.1              np112py36_1  
scipy                     0.19.0              np112py36_0  
seaborn                   0.7.1                    py36_0  
setuptools                27.2.0                   py36_0  
simplegeneric             0.8.1                    py36_1  
singledispatch            3.4.0.3                  py36_0  
sip                       4.18                     py36_0  
six                       1.10.0                   py36_0  
snowballstemmer           1.2.1                    py36_0  
sortedcollections         0.5.3                    py36_0  
sortedcontainers          1.5.7                    py36_0  
sphinx                    1.5.6                    py36_0  
spyder                    3.1.4                    py36_0  
sqlalchemy                1.1.9                    py36_0  
sqlite                    3.13.0                        0  
statsmodels               0.8.0               np112py36_0  
sympy                     1.0                      py36_0  
tblib                     1.3.2                    py36_0  
terminado                 0.6                      py36_0  
testpath                  0.3                      py36_0  
tk                        8.5.18                        0  
toolz                     0.8.2                    py36_0  
tornado                   4.5.1                    py36_0  
traitlets                 4.3.2                    py36_0  
unicodecsv                0.14.1                   py36_0  
unixodbc                  2.3.4                         0  
wcwidth                   0.1.7                    py36_0  
werkzeug                  0.12.2                   py36_0  
wheel                     0.29.0                   py36_0  
widgetsnbextension        3.0.3                    py36_2    conda-forge
wrapt                     1.10.10                  py36_0  
xlrd                      1.0.0                    py36_0  
xlsxwriter                0.9.6                    py36_0  
xlwings                   0.10.4                   py36_0  
xlwt                      1.2.0                    py36_0  
xz                        5.2.2                         1  
yaml                      0.1.6                         0  
zict                      0.1.2                    py36_0  
zlib                      1.2.8                         3  

Magic commands- Single percent sign means the cells arguments come from the same line, 2 mean the entire cell is used for the entire argument. lsmagic command can be used to list all other commands.


In [43]:
%lsmagic


Out[43]:
Available line magics:
%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

In [27]:
%matplotlib inline

Matplotlib examples can be found here.


In [29]:
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

# Fixing random state for reproducibility
np.random.seed(19680801)


matplotlib.rcParams['axes.unicode_minus'] = False
fig, ax = plt.subplots()
ax.plot(10*np.random.randn(100), 10*np.random.randn(100), 'o')
ax.set_title('Using hyphen instead of Unicode minus')
plt.show()



In [28]:
import matplotlib.pyplot as plt
from matplotlib import collections, colors, transforms
import numpy as np

nverts = 50
npts = 100

# Make some spirals
r = np.arange(nverts)
theta = np.linspace(0, 2*np.pi, nverts)
xx = r * np.sin(theta)
yy = r * np.cos(theta)
spiral = list(zip(xx, yy))

# Make some offsets
# Fixing random state for reproducibility
rs = np.random.RandomState(19680801)


xo = rs.randn(npts)
yo = rs.randn(npts)
xyo = list(zip(xo, yo))

# Make a list of colors cycling through the default series.
colors = [colors.to_rgba(c)
          for c in plt.rcParams['axes.prop_cycle'].by_key()['color']]

fig, axes = plt.subplots(2, 2)
fig.subplots_adjust(top=0.92, left=0.07, right=0.97,
                    hspace=0.3, wspace=0.3)
((ax1, ax2), (ax3, ax4)) = axes  # unpack the axes


col = collections.LineCollection([spiral], offsets=xyo,
                                 transOffset=ax1.transData)
trans = fig.dpi_scale_trans + transforms.Affine2D().scale(1.0/72.0)
col.set_transform(trans)  # the points to pixels transform
# Note: the first argument to the collection initializer
# must be a list of sequences of x,y tuples; we have only
# one sequence, but we still have to put it in a list.
ax1.add_collection(col, autolim=True)
# autolim=True enables autoscaling.  For collections with
# offsets like this, it is neither efficient nor accurate,
# but it is good enough to generate a plot that you can use
# as a starting point.  If you know beforehand the range of
# x and y that you want to show, it is better to set them
# explicitly, leave out the autolim kwarg (or set it to False),
# and omit the 'ax1.autoscale_view()' call below.

# Make a transform for the line segments such that their size is
# given in points:
col.set_color(colors)

ax1.autoscale_view()  # See comment above, after ax1.add_collection.
ax1.set_title('LineCollection using offsets')


# The same data as above, but fill the curves.
col = collections.PolyCollection([spiral], offsets=xyo,
                                 transOffset=ax2.transData)
trans = transforms.Affine2D().scale(fig.dpi/72.0)
col.set_transform(trans)  # the points to pixels transform
ax2.add_collection(col, autolim=True)
col.set_color(colors)


ax2.autoscale_view()
ax2.set_title('PolyCollection using offsets')

# 7-sided regular polygons

col = collections.RegularPolyCollection(
    7, sizes=np.abs(xx) * 10.0, offsets=xyo, transOffset=ax3.transData)
trans = transforms.Affine2D().scale(fig.dpi / 72.0)
col.set_transform(trans)  # the points to pixels transform
ax3.add_collection(col, autolim=True)
col.set_color(colors)
ax3.autoscale_view()
ax3.set_title('RegularPolyCollection using offsets')


# Simulate a series of ocean current profiles, successively
# offset by 0.1 m/s so that they form what is sometimes called
# a "waterfall" plot or a "stagger" plot.

nverts = 60
ncurves = 20
offs = (0.1, 0.0)

yy = np.linspace(0, 2*np.pi, nverts)
ym = np.max(yy)
xx = (0.2 + (ym - yy)/ym)**2 * np.cos(yy - 0.4)*0.5
segs = []
for i in range(ncurves):
    xxx = xx + 0.02*rs.randn(nverts)
    curve = list(zip(xxx, yy*100))
    segs.append(curve)

col = collections.LineCollection(segs, offsets=offs)
ax4.add_collection(col, autolim=True)
col.set_color(colors)
ax4.autoscale_view()
ax4.set_title('Successive data offsets')
ax4.set_xlabel('Zonal velocity component (m/s)')
ax4.set_ylabel('Depth (m)')
# Reverse the y-axis so depth increases downward
ax4.set_ylim(ax4.get_ylim()[::-1])


plt.show()



In [34]:
%%HTML
<iframe src="https://giphy.com/embed/3o7qE32pRVNYJYKGBO" width="480" height="269" frameBorder="0" class="giphy-embed" allowFullScreen></iframe><p><a href="https://giphy.com/gifs/veep-3o7qE32pRVNYJYKGBO">via GIPHY</a></p>




Python

To get started, let's practice some very basic python scripts. Start with trying to print "Hello World". Once you have entered your script hit Shift+Enter. This will execute your code and create or send you to the next cell. Once you have done that try checking out a Cheatsheet to help get you running with another basic code snippet.

Don't be afraid to try something new. You won't break the shell, it will simply return an error if something is wrong.

There are many cheatsheets out there if you are uncomfortable writing your own snippet. Just do a quick google search for a function and copy/paste/execute in a cell below.


In [4]:
print ('hello world');
print('this is neat')


hello world
this is neat

In [8]:
print('This is cool!')


This is cool!

In [5]:
"abc"*52


Out[5]:
'abcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabcabc'

For this next example, we will import the pandas library. Most python scripts use some sort of library or module that contains global variables, source files, and functions. These libraries or modules can be called and executed with your code.


In [9]:
import pandas as pd

df = pd.DataFrame()
a=range(11)
b=range(10,21)
c=range(20,31)
df['a']=a
df['b']=b
df['c']=c

df


Out[9]:
a b c
0 0 10 20
1 1 11 21
2 2 12 22
3 3 13 23
4 4 14 24
5 5 15 25
6 6 16 26
7 7 17 27
8 8 18 28
9 9 19 29
10 10 20 30

The math functions can be called by saying "import math" before your code. This way jupyter, and any python script for that matter, knows to use those functions in the following code.


In [8]:
import math

# Input list.
values = [0.9999999, 1, 2, 3]

# Sum values in list.
r = sum(values)
print(r)

# Sum values with fsum.
r = math.fsum(values)
print(r)


6.999999900000001
6.9999999

In [9]:
import math

value1 = 9
value2 = 16
value3 = 100

# Use sqrt method.
print(math.sqrt(value1))
print(math.sqrt(value2))
print(math.sqrt(value3))


3.0
4.0
10.0

Widgets


In [7]:
import numpy as np
import bqplot.pyplot as plt

size = 100

plt.figure(title='Scatter plot with colors')
plt.scatter(np.random.randn(size), np.random.randn(size), color=np.random.randn(size))
plt.show()



In [6]:
from ipyleaflet import Map 

Map(center=[36.146956, -86.779788], zoom=10)



In [22]:
from ipyleaflet import Map import json

Map(center=[36.146956, -86.779788], zoom=10)


---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-22-6326f91275b6> in <module>()
      8 
      9 # geojson layer with hover handler
---> 10 with open('./europe_110.geo.json') as f:
     11     data = json.load(f)
     12 for feature in data['features']:

FileNotFoundError: [Errno 2] No such file or directory: './europe_110.geo.json'

In [12]:
from pythreejs import *

f = """
function f(origu,origv) {
    // scale u and v to the ranges I want: [0, 2*pi]
    var u = 2*Math.PI*origu;
    var v = 2*Math.PI*origv;
    
    var x = Math.sin(u);
    var y = Math.cos(v);
    var z = Math.cos(u+v);
    
    return new THREE.Vector3(x,y,z)
}
"""

surf_g = ParametricGeometry(func=f);
surf = Mesh(geometry=surf_g, material=LambertMaterial(color='green', side='FrontSide'))
surf2 = Mesh(geometry=surf_g, material=LambertMaterial(color='yellow', side='BackSide'))
scene = Scene(children=[surf, surf2, AmbientLight(color='#777777')])
c = PerspectiveCamera(position=[2.5, 2.5, 2.5], up=[0, 0, 1],
                      children=[DirectionalLight(color='white',
                                                 position=[3, 5, 1],
                                                 intensity=0.6)])
Renderer(camera=c, scene=scene, controls=[OrbitControls(controlling=c)])


---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-12-0ede01a8a748> in <module>()
----> 1 from pythreejs import *
      2 
      3 f = """
      4 function f(origu,origv) {
      5     // scale u and v to the ranges I want: [0, 2*pi]

ModuleNotFoundError: No module named 'pythreejs'

Equations

You can use LaTex within Jupyter. Check out the LaTex cheatsheet!

One alternative to LaTeX is prettyPy or sympy. These are all great ways to write out equations, although the pros and cons of each need to be weighed before using. For example prettyPy does not evaluate expressions but you also don't have to initialize variables.


In [5]:
from IPython.display import display, Math, Latex
display(Math(r'F(k) = \int_{-\infty}^{\infty} f(x) e^{2\pi i k} dx'))


$$F(k) = \int_{-\infty}^{\infty} f(x) e^{2\pi i k} dx$$

In [4]:
from IPython.display import Latex
Latex(r"""\begin{eqnarray}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0 
\end{eqnarray}""")


Out[4]:
\begin{eqnarray} \nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\ \nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\ \nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\ \nabla \cdot \vec{\mathbf{B}} & = 0 \end{eqnarray}

Extra Resources

Check out these cool Jupyter notebooks.

Most resources used to build this tutorial were taken from sourced cheatsheets, documentation, and stack overflow.


In [ ]: