Lecture 6: Wednesday, September 20th 2017

Towards Intermediate Python


  • Recap: How does this stuff really work?
  • Nested environments
  • Closures
  • Decorators

Nested Environments

You can nest the definitions of functions. When you do this, inner function definitions are not even evaluated until the outer function is called. These inner functions have access to the name bindings in the scope of the outer function. So below, in make_statement, both s and key will be defined. And in key, you have access to s. This sharing is called lexical scoping.

In [1]:
def make_statement(s):
    def key(k):
        c=(s, k)
        return c
    return key
k = make_statement('name: ')
#we have captured the first element of the tuple as a "kind of state"
name = k('Albert')
name2 = k('Emmy')

('name: ', 'Albert')
('name: ', 'Emmy')

We can make this a little bit more explicit. In the line k = make_statement('name: '), make_statement() has returned the inner function key and the inner function has been given the name k. Now, when we call k() the inner function returns the desired tuple.

The reason this works is that in addition to the environment in which a user-defined function is running, that function has access to a second environment: the environment in which the function was defined. Here, key has access to the environment of make_statement. In this sense the environment of make_statement is the parent of the environment of key.

This enables two things:

  1. Names inside the inner functions (or the outer ones for that matter) do not interfere with names in the global scope. Inside the outer and inner functions, the "most lexically local" names are the ones that matter
  2. An inner function can access the environment of its enclosing (outer) function


Since the inner functions can "capture" information from an outer function's environment, the inner function is sometimes called a closure.

Notice that s, once captured by the inner function, cannot now be changed: we have lost direct access to its manipulation. This process is called encapsulation, and is a cornerstone of object oriented programming.

Augmenting Functions

Since functions are first class, we might want to augment them to put out, for example, call information, time information, etc.

Example 1

In the following, timer() accepts a function f as it's argument and returns an inner function called inner.

inner accepts a variable argument list and wraps the function f with timers to time how long it takes f to execute.

Note that f is passed a variable argument list (try to recall what Python does with that).

In [25]:
# First we write our timer function
import time
def timer(f):
    def inner(*args):
        t0 = time.time()
        output = f(*args)
        elapsed = time.time() - t0
        print("Time Elapsed", elapsed)
        return output
    return inner

In [28]:
# Now we prepare to use our timer function

import numpy as np # Import numpy

# User-defined functions
def allocate1(x, N):
    return [x]*N

def allocate2(x, N):
    ones = np.ones(N)
    return np.multiply(x, ones)

x = 1.0

# Time allocation with lists
my_alloc = timer(allocate1)
l1 = my_alloc(x, 10000000)

# Time allocation with numpy array
my_alloc2 = timer(allocate2)
l2 = my_alloc2(x, 10000000)

Time Elapsed 0.025298118591308594
Time Elapsed 0.039833784103393555

That seemed pretty useful. We might want to do such things a lot (and not just for timing purposes).

Let's recap the pattern that was so useful.

Basically, we wrote a nice function to "decorate" our function of interest. In this case, we wrote a timer function whose closure wrapped up any function we gave to it in a timing construct. In order to invoke our nice decorations, we had to pass a function to the timer function and get a new, decorated function back. Then we called the decorated function.

So the idea is as follows. We have a decorator (here called timer) that sweetens up some function (call it target).

def target():
decorated_target = decorator(target)

But Python provides what's called syntactic sugar. Instead of writing all of that, we can just write:

def target():

Now target is decorated. Let's see how this all works.

In [14]:
def allocate1(x, N):
    return [x]*N

x = 2.0
allocate1(x, 10000000)

Time Elapsed 0.020927906036376953

Example 2

We'll just create a demo decorator here.

In [5]:
def decorate(f):
    print("Let's decorate!")
    d = 1.0
    def wrapper(*args):
        print("Entering function.")
        output = f(*args)
        print("Exited function.")
        if output > d :
            print("My d is bigger than yours.")
        elif output < d:
            print("Your d is bigger than mine.")
            print("Our ds are the same size.")
    return wrapper

def useful_f(a, b, c):
    d1 = np.sqrt(a * a + b * b + c * c)
    return d1

d = useful_f(1.0, 2.0, 3.0)

Let's decorate!
Entering function.
Exited function.
My d is bigger than yours.

A key thing to remmember that a decorator is run RIGHT AFTER the function is defined, not when the function is called. Thus if you had the above decorator code in a module, it would print "Let's decorate!" when importing the module. Notice that the concept of a closure is used: the state d=1 is captured into the decorated function above.