IPython.parallel

To start the cluster, you can use notebook GUI or command line $ipcluster start



In [3]:

    
from IPython import parallel
c=parallel.Client()
dview=c.direct_view()
dview.block=True

Check a number of cores



In [2]:

    
c.ids









    Out[2]:





[0, 1, 2, 3, 4, 5, 6, 7]

First the input array is initialized and distributed over the cluster



In [10]:

    
import numpy as np
x=np.arange(100)
dview.scatter('x',x)



In [13]:

    
print c[0]['x']
print c[1]['x']
print c[-1]['x']









    



[ 0  1  2  3  4  5  6  7  8  9 10 11 12]
[13 14 15 16 17 18 19 20 21 22 23 24 25]
[88 89 90 91 92 93 94 95 96 97 98 99]

Engines computes sumation of each subset and send back to the controller



In [14]:

    
dview.execute('import numpy as np; y=np.sum(x)')
ys=dview.gather('y')
total=np.sum(ys)
print total



In [ ]: