

In [1]:

    
name = '2016-09-30-scripts-and-modules'
title = 'Using Python scripts and modules '
tags = 'basics'
author = 'Denis Sergeev'



In [2]:

    
from nb_tools import connect_notebook_to_post
from IPython.core.display import HTML

html = connect_notebook_to_post(name, title, tags, author)

You can type all the instructions in the Python interpreter. But for longer sets of instructions you definitely need to change track and write the code in text files, that are usually called scripts.

Ways of running scripts

There are several ways of executing, or running, a script. If you frequently work in a command line, you would run a Python script just by typing

$ python some_script.py

where some_script is of course the name of your script.

On Unix machines, if the script starts with #!/usr/bin/env python and the script is executable, you can just type the name of the script to run it:

$ ./demo.py

If you want to run a script from a Python interpreter, you need to use execfile command:

>>> execfile('some_script.py')

The IPython console offers another way of running a script. Yes, you guessed it, it's just literally run command:

ipython
%run demo.py

In this case, not only the script was executed, but also the variables defined in the script are now available inside the interpreter's namespace.

Debugging and profiling in IPython

This section is shamelessly taken from here.

Standard Python debugger: pdb

Interrupt execution with:

%debug magic: drops you out into the most recent error stacktrace in pdb
import q; q.d(): drops you into pdb, even outside of IPython

Interrupt execution on an Exception with %pdb magic. Use pdb the Python debugger to debug inside a notebook.

Key commands for pdb are:

p: Evaluate and print Python code
w: Where in the stack trace am I?
u: Go up a frame in the stack trace.
d: Go down a frame in the stack trace.
c: Continue execution
q: Stop execution

IPython profiler

Sometimes your code is slow. See which functions are called, how many times, and how long they take! The %prun magic reports these to you right in the Jupyter notebook!

The world beyond Jupyter

Modern graphical IDEs are shipped with built-in profiling and debugging interfaces. One of the most powerful Python IDEs is PyCharm. It has tons of integrations with the normal development flow. Some of the features include:

git integration
interactive graphical debugger
flake8 linting
smart refactoring/go to

Reusing code by importing modules

If you want to write larger and better organized programs (compared to simple scripts), where some objects are defined, (variables, functions, classes) and that you want to reuse several times, you have to create your own modules.

Let us create a module demo contained in the file demo.py:

# A demo module


def show_me_a():
    """Prints a."""
    print('a')

def show_me_b():
    """Prints b."""
    print('b')

c = 2
d = 2

In this file, we defined two functions show_me_a and show_me_b. Suppose we want to call the show_me_a function from the interpreter. We could execute the file as a script, but since we just want to have access to the function show_me_a, we are rather going to import it as a module. The syntax is as follows.

ipython
In [1]: import demo


In [2]: demo.show_me_a()
a

Pythonic import statements

Good

import sys

from os import path

import statistics as stats

from custom_package import mode

from statistics import mean, median

Bad: silently overwrites previous imports

from pylab import *

Module caching

Modules are cached: if you modify demo.py and re-import it in the old session, you will get the old one.

Solution:

ipython
    In [1]: reload(demo)

In Python 3 instead reload is not builtin, so you have to import the importlib module first and then do:

In [1]: importlib.reload(demo)

Auto-reloading in IPython

ipython
%load_ext autoreload
# always reload modules marked with "%aimport"
%autoreload 1
# reload all
%autoreload 2

Let us test it out! First we import the module using the magic:



In [3]:

    
%load_ext autoreload
%autoreload 2
import demo



In [4]:

    
demo.show_me_a()

Then we change that function to so that it prints something else:

# A demo module


def show_me_a():
    """Prints a."""
    print('Something else')

def show_me_b():
    """Prints b."""
    print('b')

c = 2
d = 2

Now demo.show_me_a() prints out "Something else" instead of "a".

`'main'` and how to use it

Sometimes we want code to be executed when a module is run directly, but not when it is imported by another module. if __name__ == '__main__' allows us to check whether the module is being run directly.

So now if the script demo.py looks like this:

def show_me_a():
    """Prints a."""
    print('Something else')

def show_me_b():
    """Prints b."""
    print('b')

# show_me_b() runs on import
show_me_b()

if __name__ == '__main__':
    # show_me_a() is only executed when the module is run directly.
    show_me_a()

Using packages and creating your own modules

In order to import your local modules, you must do three things:

put the .py file in a separate folder
add an empty __init__.py file to the folder
add that folder to the Python path with sys.path.append()

If you are getting too good at writing code and it's becoming useful for other projects or people, you should consider refactoring it into a standalone package. You can then make it available online via PyPi or Anaconda. There are great templates out there. To name but a few:

Resources



In [5]:

    
HTML(html)









    Out[5]:





    
     This post was written as an IPython (Jupyter) notebook. You can view or download it using
    nbviewer.