Chapter 5, example 3

In this example we show how to use MPI from IPython.

We first need to create a MPI profile with:

ipython profile create --parallel --profile=mpi

Then specify the following in the cluster configuration file:

c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'

and run the engines with:

ipcluster start -n 4 --profile=mpi

Now, we can create a MPI client.

We will compute the sum of all integers between 1 and 16 in parallel over four cores. The script psum.py contains the following code:

from mpi4py import MPI
import numpy as np

# This function will be executed on all processes.
def psum(a):
    # "a" only contains a subset of all integers.
    # They are summed locally on this process. 
    locsum = np.sum(a)

    # We allocate a variable that will contain the final result, that is,
    # the sum of all our integers.
    rcvBuf = np.array(0.0,'d')

    # We use a MPI reduce operation:
    #   * locsum is combined from all processes
    #   * these local sums are summed with the MPI.SUM operation
    #   * the result (total sum) is distributed back to all processes in
    #     the rcvBuf variable
    MPI.COMM_WORLD.Allreduce([locsum, MPI.DOUBLE],
        [rcvBuf, MPI.DOUBLE],
        op=MPI.SUM)
    return rcvBuf

We execute this script on all engines so that the function is available everywhere.

Now we distribute the array with the 16 values across the engines (each engine gets a subarray).

We compute the total sum in parallel using MPI.

Finally, the result is available on all engines thanks to the MPI Allreduce function.