The IPython Shell

(Note: If you're looking at this notebook after the presentation, this portion of the talk was given at the interactive IPython console. The cells here were used as notes, and describe roughly what appeared on the screen.

Learn Your Keybinds!

These commands aren't IPython-specific, but learning them will make your command-line life much better.

Commands listed in bold also work in IPython Notebook.

CTRL-f -> Move Cursor Forward One Character.
CTRL-b -> Move Cursor Backward One Character.
ALT-f -> Move Cursor Forward One Word.
ALT-b -> Move Cursor Backward One Word.
CTRL-a -> Move Cursor to Line Start.
CTRL-e -> Move Cursor to Line End.
CTRL-k -> Cut to End of Line.
CTRL-y -> Paste to End of Line.
CTRL-r -> Reverse Incremental Search (aka search history matching what you type).
CTRL-p -> Previous History Entry.
CTRL-n -> Next History Entry.
CTRL-l -> Clear Screen.
CTRL-c -> Interrupt Current Execution.
CTRL-d -> Shutdown.

IPython Magics: Shell Emulation

IPython has various built-in magic functions that extend standard Python syntax. Many IPython magics provide aliases of common shell commands.

In [1]: %ls
Quantopian-Meetup-Talk-Notes.ipynb
Quantopian_Meetup_Pandas.ipynb
Quantopian_Meetup_Talk_IPython_Goodies.ipynb
Quantopian_Meetup_Zipline.ipynb
secret_quantopian_stuff/

In [2]: %cd secret_quantopian_stuff/
/Users/ssanderson/projects/notebooks/quanto/secret_quantopian_stuff
In [3]: %ls
kittens/         secret_plans.py

In [4]: %cat secret_plans.py
print \
"""
Step 1: Make awesome tools for Quants.
Step 2: ????????
Step 3: Profit!
"""
In [5]: ! open kittens/super_secret_kitten.jpg

More Builtin Shell Command Aliases

%cd -> Change Directory.
%ls -> List Directory Content.
%env -> List Environment Variables.
%cat -> Print Contents of File.
%mv -> Move a File/Directory.
%rm -> Remove a File/Directory.
%mkdir -> Make a directory.
%man -> View a man page.

Defining Custom Shell Commands

%alias <alias-name> <command> -> Bind an alias to a command.
! <command> -> Run , printing its output.
!! <command> -> Run , capturing its output as a list.

Example: Use git from IPython

In [1]: alias git git

In [2]: git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

Example: Use Unix Pipes to Create a Complex Shell Command

Suppose we want a list of the currently running ipython process ids. This requires a moderate amount of code to do correctly in Python, requiring us to use psutil and iterate over the output of psutil.process_iter(). Using the !! magic, however, we can write this using Unix pipes in two lines.

In [1]: ipython_pid_strings = !! ps | grep ipython | cut -d't' -f1

In [2]: map(int, ipython_pid_strings)
Out[2]: [23546, 34983, 35040, 35042, 35048, 37696, 37703, 37827, 38290, 38291, 38293]

The above pipeline says roughly:

Run ps to print the list of currently running processes.
Filter the output of ps with grep to get just the lines that have the word 'ipython' in them.
Break each line into sub-sections with cut, using the letter t as a delimiter. Then select just the first subsection of each line. (The -d flag says to use a specific delimiter, and the -f flag says which subsections to keep.)
Convert the resulting text output into a list of integers that we can use in our shell.

Editing Long Commands:

In [1]: edit
IPython will make a temporary file named: /var/folders/bunch/of/gibberish.py

Edit file in your $EDITOR of choice and quit when done.

class Foo(object):
    def bar(self, x):
           print x

foo = Foo()
foo.bar(3)

done. Executing edited code...
3
Out[1]: 'class Foo(object):\n\n    def bar(self, x):\n        print x\n\n\nfoo = Foo()\nfoo.bar(3)\n'

Convenience Features: Autocomplete

You can autocomplete any variable name currently in your namespace with TAB.

big_variable_name = 5
big_<TAB> -> Autocomplete variable name.

Autocomplete also works on attributes of objects.

import pandas
pandas.<TAB> -> Show top-level pandas attributes.

Convenience Features: Documentation

You can get information about any callable object by evaluating <obj>? or <obj>??. These are syntactic sugar for the pinfo and pinfo2 magics, respectively.

import pandas
pandas.DataFrame? -> Show docstring.
pandas.DataFrame.plot?? -> Show source.

import zipline; zipline.<TAB> -> Autocomplete to zipline.data.loader.load_bars_from_yahoo
load_bars_from_yahoo(stocks=['AAPL', 'TSLA'])

I wish someone would do a tutorial on how to use this cool-looking Panel class...

Convenience Features: History

You can view your command history with

history

You can output your history to a file with

history -f <filename>

As mentioned above, you can search through your recent command history with CTRL-r

And More...

%quickref -> Show Output