Familiarize Yourself with Jupyter

The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. The following links may be useful in gaining a basic understanding of Jupyter and how to work in Jupyter notebook:

  1. http://jupyter.org/about.html: general information about Jupyter
  2. https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/execute.html: starting guide
  3. http://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Notebook%20Basics.ipynb: notebook basics

If this is your first time using Jupyter, start with taking a tour of the user interface:

  • Click on the menu Help -> User Interface Tour
  • Scroll over each of the tabs in the menu to see the options. They should be fairly straightforward.

Here are some other helpful tips and shortcuts:

  • You can run the notebook document step-by-step (one cell a time) by pressing shift + enter.
  • You can run the whole notebook in a single step by clicking on the menu Cell -> Run All.
  • To restart the kernel (i.e. the computational engine), click on the menu Kernel -> Restart. This can be useful to start a computation over from scratch (e.g. variables are deleted, open files are closed, etc...).

Cell Types

Cells are designated as Code, Markdown, or Raw NBConvert (Notebook Conversion). To switch from one type to another, click on Cell -> Cell Type.

Throughout our tutorials, we will mainly be working in code cells to learn how to use Python. However, each of the tutorials include Markdown cells to provide introductions and learning objectives for each tutorial, background information, and instructions throughout. We encourage you to open these markdown cells to become familiar with the syntax, and add markdown notes as you see fit. Learning to use Markdown and Notebook Conversion cells will become a useful skill for Python scripts and notebooks you will use in your future research. Throughout this data institute, you will learn to use the full set of Jupyter Notebook tools for conducting reproducible (and publishable) research.

Code

Code cells are where we will write and execute our Python scripts and functions.

Markdown

Markdown cells are mainly used to neatly document the code that is written in Jupyter notebooks. While this course does not focus on Markdown, we encourage you to explore the Markdown capabilities, and add Markdown cells to your notebooks as you see fit. Some useful resources for learning Markdown Syntax include:

  1. Markdown Cheatsheet: http://nestacms.com/docs/creating-content/markdown-cheat-sheet
  2. https://ipython.org/ipython-doc/3/notebook/notebook.html#markdown-cells
  3. Example Markdown Notebook: http://nbviewer.jupyter.org/github/ipython/ipython/blob/1.x/examples/notebooks/Part%204%20-%20Markdown%20Cells.ipynb

NBConvert - Notebook Conversion

The NBConvert tool allows you to convert a Jupyter .ipynb noteook document file into another format. The nbconvert documentation contains a complete description of this tool's capabilities: https://media.readthedocs.org/pdf/nbconvert/latest/nbconvert.pdf. The description below, copied from the document, gives a brief overview of the tool:

Using nbconvert enables:

  • presentation of information in familiar formats, such as PDF.
  • publishing of research using LaTeX and opens the door for embedding notebooks in papers.
  • collaboration with others who may not use the notebook in their work.
  • sharing contents with many people via the web using HTML.

Overall, notebook conversion and the nbconvert tool give scientists and researchers the flexibility to deliver information in a timely way across different formats.

Primarily, the nbconvert tool allows you to convert a Jupyter .ipynb notebook document file into another static format including HTML, LaTeX, PDF, Markdown, reStructuredText, and more. nbconvert can also add productivity to your workflow when used to execute notebooks programmatically.

If used as a Python library (import nbconvert), nbconvert adds notebook conversion within a project. For example, nbconvert is used to implement the “Download as” feature within the Jupyter Notebook web application. When used as a command line tool (invoked as jupyter nbconvert ...), users can conveniently convert just one or a batch of notebook files to another format.

Objectives

In this tutorial, we will learn some useful Python commands and tips, including how to:

  1. Use the print statement
  2. Execute bash commands (eg. pwd, ls)
  3. Use magic commands (eg %who, %who_ls, %whos)
  4. Determine the data type of a variable and convert to a different type (integers, strings, floats)
  5. Import and execute commands from a python package
  6. Use the numpy (Numerical Python) package to manipulate arrays and explore indexing

This link has some useful tips that may come in handy in your research: https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

The Zen of Python


In [2]:
import this

Basic commands: print, !pwd, !ls


In [3]:
#Basic commands: print, !pwd, !ls

#Python does not automatically display results unless you use a print statement
#You can also use a print statement as a debugging tool:

#Print working directory - the ! executes bash commands from the Python platform
print('Current Working Directory:')
!pwd  

print('\nFiles in Working Directory:') #\n enters to a new line
!ls ./


Current Working Directory:
/cygdrive/y/users/bhass/RemoteSensingDataInstitute/DataInstitute2017/PythonWorkflows

Files in Working Directory:
getting_started_with_jupyter_notebooks.ipynb
hyperspectral_hdf5
Jupyter_Markdown_Cells.ipynb

Types - integers, floats, & strings


In [4]:
#Integers
x = 1;
print('x type:',type(x))

#Floating Point Numbers
x_float = float(x) #use the command float to change a number to type float
print('x_float type:',type(x_float))

#Strings
example_string = 'This is a string'
print('example_string type:',type(example_string))
#To change a variable to type string use the command str(argument), for example:
x_string = str(x)
print('x_string type:',type(x_string))


x type: <class 'int'>
x_float type: <class 'float'>
example_string type: <class 'str'>
x_string type: <class 'str'>

Importing packages: numpy

numpy (or Numerical Python) is a useful package for manipulating numbers


In [5]:
#Import Numpy and make an example array
import numpy as np 
#to run a numpy command, use the syntax np.command_name
example_array = np.array([[1,2,3],[4,5,6]]) 
print('example_array:\n',example_array)
print('example_array type:',type(example_array))


example_array:
 [[1 2 3]
 [4 5 6]]
example_array type: <class 'numpy.ndarray'>

Indexing

Key Points:

  • The index (i,j) refers to the ith row and jth column
  • Python indices start at 0, so to select to the ith column, you need to use the index (i-1)
  • Negative indices will count backwards from the end of an array

In [6]:
#extract a single value from the array:
#indexing starts at 0, so [0,0] extracts the 1st row and 1st column
#negative indices will count backwards from the end
print(example_array[0,0]) 
print(example_array[-1,-1])


1
6

Display variables with %whos

Since the Jupyter notebook doesn't have a separate window to display the variables that have been generated, we can use the commands %who, %who_ls, or %whos to show a complete list of imported modules, functions, variables, and other outputs (eg. figures). The command %who lists the variable names in row format, %who_ls lists the variable names in column format, and %whos is the most comprehensive, displaying variable name, type, and data/info. Remember this tool throughout the following tutorials if you need to check whether a module or function was already imported, a variable created, etc.


In [7]:
#Display variables & imported modules 
%whos


Variable         Type       Data/Info
-------------------------------------
example_array    ndarray    2x3: 6 elems, type `int32`, 24 bytes
example_string   str        This is a string
np               module     <module 'numpy' from 'C:\<...>ges\\numpy\\__init__.py'>
this             module     <module 'this' from 'C:\\<...>Anaconda3\\lib\\this.py'>
x                int        1
x_float          float      1.0
x_string         str        1