https://blog.dominodatalab.com/simple-parallelization/ says: "Instead of processing your items in a normal loop, you can process all your items in parallel, spreading the work across multiple cores. Here's an example using a list of numbers, and a function that squares the numbers. You would replace this with your specific data and logic, of course"
In [4]:
from joblib import Parallel, delayed
import multiprocessing
inputs = range(10)
def processInput(i):
return i*i
num_cores = multiprocessing.cpu_count()
results = Parallel(n_jobs=num_cores)(delayed(processInput)(i) for i in inputs)
In [5]:
num_cores
Out[5]:
In [6]:
results
Out[6]: