Title: Concurrent Processing
Slug: concurrent_processing
Summary: Concurrent Processing in Python.
Date: 2016-01-23 12:00
Category: Python
Tags: Basics
Authors: Chris Albon

Interesting in learning more? Check out Fluent Python

Preliminaries



In [1]:

    
from concurrent import futures

Create Data



In [2]:

    
data = range(100)

Create Function



In [3]:

    
# Create some function that takes a value
def some_function(value):
    # And outputs it raised to its own power
    return value**value

Run The Function On The Data Concurrently



In [4]:

    
# With a pool of workers
with futures.ProcessPoolExecutor() as executor:
    # Map the function to the data
    result = executor.map(some_function, data)

View Results



In [5]:

    
# List the first 5 outputs
list(result)[0:5]









    Out[5]:





[1, 1, 4, 27, 256]