SciPy - Scientific Python

SciPy is a collection of mathematical algorithms and convenience functions built on the Numpy extension of Python. It adds significant power to the interactive Python session by providing the user with high-level commands and classes for manipulating and visualizing data. With SciPy an interactive Python session becomes a data-processing and system-prototyping environment rivaling sytems such as MATLAB, IDL, Octave, R-Lab, and SciLab.

SciPy Organization

  • cluster Clustering algorithms
  • constants Physical and mathematical constants
  • fftpack Fast Fourier Transform routines
  • integrate Integration and ordinary differential equation solvers
  • interpolate Interpolation and smoothing splines
  • io Input and Output
  • linalg Linear algebra
  • ndimage N-dimensional image processing
  • odr Orthogonal distance regression
  • optimize Optimization and root-finding routines
  • signal Signal processing
  • sparse Sparse matrices and associated routines
  • spatial Spatial data structures and algorithms
  • special Special functions
  • stats Statistical distributions and functions
  • weave C/C++ integration

In [1]:
from scipy import linalg, optimize

In [2]:
import scipy as sp
sp.info(optimize.fmin)


 fmin(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None,
      full_output=0, disp=1, retall=0, callback=None, initial_simplex=None)

Minimize a function using the downhill simplex algorithm.

This algorithm only uses function values, not derivatives or second
derivatives.

Parameters
----------
func : callable func(x,*args)
    The objective function to be minimized.
x0 : ndarray
    Initial guess.
args : tuple, optional
    Extra arguments passed to func, i.e. ``f(x,*args)``.
xtol : float, optional
    Absolute error in xopt between iterations that is acceptable for
    convergence.
ftol : number, optional
    Absolute error in func(xopt) between iterations that is acceptable for
    convergence.
maxiter : int, optional
    Maximum number of iterations to perform.
maxfun : number, optional
    Maximum number of function evaluations to make.
full_output : bool, optional
    Set to True if fopt and warnflag outputs are desired.
disp : bool, optional
    Set to True to print convergence messages.
retall : bool, optional
    Set to True to return list of solutions at each iteration.
callback : callable, optional
    Called after each iteration, as callback(xk), where xk is the
    current parameter vector.
initial_simplex : array_like of shape (N + 1, N), optional
    Initial simplex. If given, overrides `x0`.
    ``initial_simplex[j,:]`` should contain the coordinates of
    the j-th vertex of the ``N+1`` vertices in the simplex, where
    ``N`` is the dimension.

Returns
-------
xopt : ndarray
    Parameter that minimizes function.
fopt : float
    Value of function at minimum: ``fopt = func(xopt)``.
iter : int
    Number of iterations performed.
funcalls : int
    Number of function calls made.
warnflag : int
    1 : Maximum number of function evaluations made.
    2 : Maximum number of iterations reached.
allvecs : list
    Solution at each iteration.

See also
--------
minimize: Interface to minimization algorithms for multivariate
    functions. See the 'Nelder-Mead' `method` in particular.

Notes
-----
Uses a Nelder-Mead simplex algorithm to find the minimum of function of
one or more variables.

This algorithm has a long history of successful use in applications.
But it will usually be slower than an algorithm that uses first or
second derivative information. In practice it can have poor
performance in high-dimensional problems and is not robust to
minimizing complicated functions. Additionally, there currently is no
complete theory describing when the algorithm will successfully
converge to the minimum, or how fast it will if it does. Both the ftol and
xtol criteria must be met for convergence.

Examples
--------
>>> def f(x):
...     return x**2

>>> from scipy import optimize

>>> minimum = optimize.fmin(f, 1)
Optimization terminated successfully.
         Current function value: 0.000000
         Iterations: 17
         Function evaluations: 34
>>> minimum[0]
-8.8817841970012523e-16

References
----------
.. [1] Nelder, J.A. and Mead, R. (1965), "A simplex method for function
       minimization", The Computer Journal, 7, pp. 308-313

.. [2] Wright, M.H. (1996), "Direct Search Methods: Once Scorned, Now
       Respectable", in Numerical Analysis 1995, Proceedings of the
       1995 Dundee Biennial Conference in Numerical Analysis, D.F.
       Griffiths and G.A. Watson (Eds.), Addison Wesley Longman,
       Harlow, UK, pp. 191-208.
/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:2: DeprecationWarning: scipy.info is deprecated and will be removed in SciPy 2.0.0, use numpy.info instead
  

Linear algebra


In [3]:
import numpy as np
from scipy import linalg
arr = np.array([[1, 2],   # square matrix
                [3, 4]])
print (linalg.det(arr))     # determinant
print (linalg.inv(arr))     # inversion


-2.0
[[-2.   1. ]
 [ 1.5 -0.5]]

In [4]:
arr = np.array([[3, 2],   # singluar matrix
                [6, 4]])
print (linalg.det(arr))


0.0

In [5]:
print (linalg.inv(arr))     # inversion of singular matrix


---------------------------------------------------------------------------
LinAlgError                               Traceback (most recent call last)
<ipython-input-5-439c6a76afb8> in <module>
----> 1 print (linalg.inv(arr))     # inversion of singular matrix

/opt/conda/lib/python3.7/site-packages/scipy/linalg/basic.py in inv(a, overwrite_a, check_finite)
    977         inv_a, info = getri(lu, piv, lwork=lwork, overwrite_lu=1)
    978     if info > 0:
--> 979         raise LinAlgError("singular matrix")
    980     if info < 0:
    981         raise ValueError('illegal value in %d-th argument of internal '

LinAlgError: singular matrix

Optimization


In [6]:
import numpy as np
from scipy import optimize
%matplotlib inline
import matplotlib.pyplot as plt

Define function to analyze


In [7]:
def f(x):
    return x**2 + 10*np.sin(x)

Find roots and minima


In [8]:
grid = (-10, 10, 0.1)
xmin_global = optimize.brute(f, (grid,))
xmin_local = optimize.fminbound(f, 0, 10)
root = optimize.fsolve(f, 1)  # our initial guess is 1
root2 = optimize.fsolve(f, -2.5)
print (xmin_global, xmin_local, root, root2)


[-1.30641113] 3.8374671194983834 [0.] [-2.47948183]

Create data and add random noise


In [9]:
xdata = np.linspace(-10, 10, num=50)
np.random.seed(1234)
ydata = f(xdata) + np.random.randn(xdata.size) * 1

Define function for fitting the synthetic data


In [10]:
def f2(x, a, b):
    return a*x**2 + b*np.sin(x)

Curve fit


In [11]:
guess = [.1, .1]
[a, b], params_covariance = optimize.curve_fit(f2, xdata, ydata, guess)
print (a, b)


1.0005005572953594 9.757627807241091

Plot


In [12]:
x = np.arange(-10, 10, 0.1)
plt.plot(x, f(x), 'b-', label="f(x)")
plt.plot(xdata, ydata, '.', label="Synthetic data")
plt.plot(x, f2(x, a, b), 'r--', label="Curve fit result")
xmins = np.array([xmin_global[0], xmin_local])
plt.plot(xmins, f(xmins), 'go', label="Minima")
roots = np.array([root, root2])
plt.plot(roots, f(roots), 'kv', label="Roots", ms=15)
plt.legend()
plt.xlabel('x')
plt.ylabel('f(x)')


Out[12]:
Text(0, 0.5, 'f(x)')

Interpolation

Create synthetic data close to a sine function with random noise


In [13]:
measured_time = np.linspace(0, 1, 10)
noise = (np.random.random(10)*2 - 1) * .3
measures = np.sin(2 * np.pi * measured_time) + noise

Perform interpolation


In [14]:
from scipy.interpolate import interp1d
computed_time = np.linspace(0, 1, 50)             # X - values

linear_interp = interp1d(measured_time,
                         measures)                # create interpolator
linear_results = linear_interp(computed_time)     # use interpolator

cubic_interp = interp1d(measured_time,
                        measures,
                        kind='cubic')             # create interpolator
cubic_results = cubic_interp(computed_time)       # use interpolator

Plot


In [15]:
plt.plot(measured_time, measures, 'o', ms=6, label='measures')
plt.plot(computed_time, linear_results, '.-', label='linear interp')
plt.plot(computed_time, cubic_results, '.-', label='cubic interp')
plt.legend()
plt.show()


Image processing


In [16]:
from scipy import ndimage
from scipy import misc

Load sample image


In [17]:
face = misc.face()[:,:,0]
plt.imshow(face, cmap='gray')
plt.show()


Perform distortion of the sample image


In [18]:
noisy_face = face + face.std()*0.5*np.random.standard_normal(face.shape)
plt.imshow(noisy_face, cmap='gray')


Out[18]:
<matplotlib.image.AxesImage at 0x7efe8db6a210>

In [19]:
blurred_face = ndimage.gaussian_filter(noisy_face, sigma=3)
plt.imshow(blurred_face, cmap='gray')


Out[19]:
<matplotlib.image.AxesImage at 0x7efe8db52cd0>

In [20]:
median_face = ndimage.median_filter(noisy_face, size=5)
plt.imshow(median_face, cmap='gray')


Out[20]:
<matplotlib.image.AxesImage at 0x7efe8dabea50>

Improve plurred image and plot


In [21]:
from scipy import signal
wiener_face = signal.wiener(noisy_face, (5,5))
plt.imshow(wiener_face, cmap='gray')
plt.axis('off')


Out[21]:
(-0.5, 1023.5, 767.5, -0.5)

Measurements on images

Let us first generate a nice synthetic binary image.


In [22]:
x, y = np.indices((100, 100))
sig = np.sin(2*np.pi*x/50.)*np.sin(2*np.pi*y/50.)*(1+x*y/50.**2)**2
plt.imshow(sig);plt.colorbar()


Out[22]:
<matplotlib.colorbar.Colorbar at 0x7efe8c106a10>

In [23]:
mask = sig > 1
plt.imshow(mask.astype(int));plt.colorbar()


Out[23]:
<matplotlib.colorbar.Colorbar at 0x7efe8c093610>

In [24]:
labels, nb = ndimage.label(mask)
plt.imshow(labels); plt.colorbar()


Out[24]:
<matplotlib.colorbar.Colorbar at 0x7efe877f77d0>

Find areas of the identified zones


In [25]:
label_ids = np.unique(labels)
print (label_ids)
print (ndimage.sum(mask, labels, label_ids))


[0 1 2 3 4 5 6 7 8]
[  0. 190.  45. 424. 278. 459. 190. 549. 424.]

Find maximum value for each object


In [26]:
print (ndimage.maximum(sig, labels, range(1, labels.max()+1)))


[ 1.80238238  1.13527605  5.51954079  2.49611818  6.71673619  1.80238238
 16.76547217  5.51954079]

Extract idetified object from original matrix


In [27]:
sl = ndimage.find_objects(labels==4)
plt.imshow(sig[sl[0]])


Out[27]:
<matplotlib.image.AxesImage at 0x7efe877007d0>

Exercises

1. Analyze the function:

\begin{equation*} y = 0.2 x^2 + 10 cos(x) \end{equation*}
  1. Find roots of the function
  2. Find local minimum of the function

2. Fit 2nd order polynom to noisy data:

x = np.linspace(-10, 10, 20)

y = 2 * x ** 2 + np.random.randn(x.size) * 10

3. Find edges on the 'ascent' image (use ndimage.sobel)

from scipy import misc

lena = misc.ascent()


In [ ]: