Silicon Forest Math Series
Oregon Curriculum Network
Generator functions relate to generator expressions, objects which delay exectution until pressed into service as iterators, a type with a __next__ method.
The for loop counts on an iterator for its target and implicitly applies iter(obj) to whatever it gets. Iterators save on memory as they know how to share and underlying iterable.
Some iterators also have the ability to function as objects able to pause and resume operations, without forgetting their internal state.
In [1]:
powers = (lambda x: pow(x, n) for n in range(-4,5))
phi = (1 + pow(5,0.5)) * 0.5 # golden proportion
for n, f in enumerate(powers, start=-4): # iterates through lambda expressions
print("phi ** {:2} == {:10.8f}".format(n, f(phi)))
Any object expecting to be the target of a for loop, if not already an iterator, needs to either:
In [2]:
class Any:
def __init__(self):
self.__dict__ = {0:'scissors', 1:'paper', 2:'rock'}
def __getitem__(self, n): # enough for iter() to go on
if n == len(self.__dict__):
raise StopIteration # tells for loop when to stop
return self.__dict__[n]
for thing in Any():
print(thing)
The generator function below shows what most Pythonistas consider the base case for the keyword yield: using it much like return, to provide an object to the generator function's caller, in this case a next prime number, going in sequence.
The trial-by-division algorithm requires keeping a growing sequence of successive primes, and using them to test new candidates. After taking care of the only even prime, 2, it makes sense to jump forward through the odds, screening out all the composites.
In [1]:
import pprint
def primes():
"""generate successive prime numbers (trial by division)"""
candidate = 1
_primes_so_far = [2] # first prime, only even prime
yield _primes_so_far[0] # share it!
while True:
candidate += 2 # check odds only from now on
for prev in _primes_so_far:
if prev**2 > candidate:
yield candidate # new prime!
_primes_so_far.append(candidate)
break
if not divmod(candidate, prev)[1]: # no remainder!
break # done looping
p = primes() # generator function based iterator
pp = pprint.PrettyPrinter(width=40, compact=True)
pp.pprint([next(p) for _ in range(50)]) # next 30 primes please!
Now lets take a look at pretty much the same iterator written a different way: with an explicit __iter__ and __next__.
In [4]:
class Primes:
def __init__(self):
self.candidate = 1
self._primes_so_far = [2] # first prime, only even prime
def __iter__(self):
return self
def __next__(self):
while True:
self.candidate += 2 # check odds only from now on
for prev in self._primes_so_far:
if prev**2 > self.candidate:
self._primes_so_far.append(self.candidate)
return self._primes_so_far[-2]
if not divmod(self.candidate, prev)[1]: # no remainder!
break
pp = pprint.PrettyPrinter(width=40, compact=True)
p = Primes() # class based iterator
pp.pprint([next(p) for _ in range(30)]) # n
Feeding an open-ended iterator to a for loop runs the risk of a "forever loop". The itertools module is filled with tools designed to rein in the infinite loopers. Just use islice(obj, start, stop) to keep the for loop finite:
In [5]:
from itertools import islice
p = Primes()
for n in islice(p, 0, 20):
print(n, end=", ")
In the code below, we see (yield) turned around and used not to return objects, but to receive them. The parentheses are by convention and suggest a "mouth" bringing something in from outside.
A generator function's .send() method resumes its execution inside its body with an intake of some passed-in object from the caller, the argument to send. In the above example, two callers are nested, the inner one writing a prime to a file, the outer one feeding it next primes for said catalog.
The coroutine decorator may seem a bit mysterious at first. A generator function does not run any of its code upon being instanced. No yield statement has yet been encountered, so use of .send(obj) would raise an exception were obj any object but None.
The decorator has already fed a .send(None) to the generator in question, equivalent to feeding it to next() a first time. The decorator applies a first action somewhat like cocking a pistol, putting the generator or coroutine in the firing position, positioned at a first yield.
In [6]:
# -*- coding: utf-8 -*-
"""
Created on Thu Oct 13 13:48:52 2016
@author: Kirby Urner
David Beazley:
https://youtu.be/Z_OAlIhXziw?t=23m42s
Trial by division, but this time the primes coroutine acts
more as a filter, passing qualified candidates through to
print_me, which writes to a file.
"""
import pprint
def coroutine(func):
"""
Advances decorated generator function to the first yield
"""
def start(*args, **kwargs):
cr = func(*args, **kwargs)
cr.send(None) # or next(cr) or cr.__next__()
return cr
return start
@coroutine
def print_me(file_name):
with open(file_name, 'w') as file_obj:
while True:
to_print = (yield)
file_obj.write(str(to_print)+"\n")
@coroutine
def primes(target):
_primes_so_far = [2]
target.send(2)
while True:
candidate = (yield)
for prev in _primes_so_far:
if not divmod(candidate, prev)[1]:
break
if prev**2 > candidate:
_primes_so_far.append(candidate)
target.send(candidate)
break
output = print_me("primes.txt")
p = primes(output)
for x in range(3, 200, 2): # test odds 3-199
p.send(x)
with open("primes.txt", 'r') as file_obj:
print(", ".join(file_obj.read().split("\n"))[:-2])
In the top example, the generator provides a mechanism for advancing to a next odd number for testing, whereas the coroutine version consumes the odds we feed it, passing through those which pass the test for printing, as well as accumulating them internally.
Linked Jupyter Notebooks: