A primer on numerical differentiation

In order to numerically evaluate a derivative $y'(x)=dy/dx$ at point $x_0$, we approximate is by using finite differences: Therefore we find: $$\begin{eqnarray} && dx \approx \Delta x &=&x_1-x_0, \\ && dy \approx \Delta y &=&y_1-y_0 = y(x_1)-y(x_0) = y(x_0+\Delta_x)-y(x_0),\end{eqnarray}$$

Then we re-write the derivative in terms of discrete differences as: $$\frac{dy}{dx} \approx \frac{\Delta y}{\Delta x}$$

Example

Let's look at the accuracy of this approximation in terms of the interval $\Delta x$. In our first example we will evaluate the derivative of $y=x^2$ at $x=1$.


In [5]:
dx = 1.
x = 1.
while(dx > 1.e-10):
    dy = (x+dx)*(x+dx)-x*x
    d = dy / dx
    print("%6.0e %20.16f %20.16f" % (dx, d, d-2.))
    dx = dx / 10.


 1e+00   3.0000000000000000   1.0000000000000000
 1e-01   2.1000000000000019   0.1000000000000019
 1e-02   2.0100000000000007   0.0100000000000007
 1e-03   2.0009999999996975   0.0009999999996975
 1e-04   2.0000999999991720   0.0000999999991720
 1e-05   2.0000100000139298   0.0000100000139298
 1e-06   2.0000009999243669   0.0000009999243669
 1e-07   2.0000001010878061   0.0000001010878061
 1e-08   1.9999999878450576  -0.0000000121549424
 1e-09   2.0000001654807416   0.0000001654807416
 1e-10   2.0000001654807416   0.0000001654807416

Why is it that the sequence does not converge? This is due to the round-off errors in the representation of the floating point numbers. To see this, we can simply type:


In [123]:
((1.+0.0001)*(1+0.0001)-1)


Out[123]:
0.0002000099999999172

Let's try using powers of 1/2


In [10]:
dx = 1.
x = 1.
while(dx > 1.e-10):
    dy = (x+dx)*(x+dx)-x*x
    d = dy / dx
    print("%6.0e %20.16f %20.16f" % (dx, d, d-2.))
    dx = dx / 2.


 1e+00   3.0000000000000000   1.0000000000000000
 5e-01   2.5000000000000000   0.5000000000000000
 2e-01   2.2500000000000000   0.2500000000000000
 1e-01   2.1250000000000000   0.1250000000000000
 6e-02   2.0625000000000000   0.0625000000000000
 3e-02   2.0312500000000000   0.0312500000000000
 2e-02   2.0156250000000000   0.0156250000000000
 8e-03   2.0078125000000000   0.0078125000000000
 4e-03   2.0039062500000000   0.0039062500000000
 2e-03   2.0019531250000000   0.0019531250000000
 1e-03   2.0009765625000000   0.0009765625000000
 5e-04   2.0004882812500000   0.0004882812500000
 2e-04   2.0002441406250000   0.0002441406250000
 1e-04   2.0001220703125000   0.0001220703125000
 6e-05   2.0000610351562500   0.0000610351562500
 3e-05   2.0000305175781250   0.0000305175781250
 2e-05   2.0000152587890625   0.0000152587890625
 8e-06   2.0000076293945312   0.0000076293945312
 4e-06   2.0000038146972656   0.0000038146972656
 2e-06   2.0000019073486328   0.0000019073486328
 1e-06   2.0000009536743164   0.0000009536743164
 5e-07   2.0000004768371582   0.0000004768371582
 2e-07   2.0000002384185791   0.0000002384185791
 1e-07   2.0000001192092896   0.0000001192092896
 6e-08   2.0000000596046448   0.0000000596046448
 3e-08   2.0000000298023224   0.0000000298023224
 1e-08   2.0000000149011612   0.0000000149011612
 7e-09   2.0000000000000000   0.0000000000000000
 4e-09   2.0000000000000000   0.0000000000000000
 2e-09   2.0000000000000000   0.0000000000000000
 9e-10   2.0000000000000000   0.0000000000000000
 5e-10   2.0000000000000000   0.0000000000000000
 2e-10   2.0000000000000000   0.0000000000000000
 1e-10   2.0000000000000000   0.0000000000000000

In addition, one could consider the midpoint difference, defined as: $$ dy \approx \Delta y = y(x_0+\frac{\Delta_x}{2})-y(x_0-\frac{\Delta_x}{2}).$$

For a more complex function we need to import it from math. For instance, let's calculate the derivative of $sin(x)$ at $x=\pi/4$, including both the forward and midpoint differences.


In [13]:
from math import sin, sqrt, pi
dx = 1.
while(dx > 1.e-10):
    x = pi/4.
    d1 = sin(x+dx) - sin(x); #forward
    d2 = sin(x+dx*0.5) - sin(x-dx*0.5); # midpoint
    d1 = d1 / dx;
    d2 = d2 / dx;
    print("%6.0e %20.16f %20.16f %20.16f %20.16f" % (dx, d1, d1-sqrt(2.)/2., d2, d2-sqrt(2.)/2.) )
    dx = dx / 2.


 1e+00   0.2699544827129282  -0.4371522984736194   0.6780100988420897  -0.0290966823444578
 5e-01   0.5048856975964859  -0.2022210835900616   0.6997640691250939  -0.0073427120614536
 2e-01   0.6118351194488110  -0.0952716617377366   0.7052667953545546  -0.0018399858319930
 1e-01   0.6611301360648314  -0.0459766451217162   0.7066465151141266  -0.0004602660724210
 6e-02   0.6845566203276618  -0.0225501608588857   0.7069916978116613  -0.0001150833748863
 3e-02   0.6959440534591224  -0.0111627277274252   0.7070780092891873  -0.0000287718973603
 2e-02   0.7015538499518499  -0.0055529312346977   0.7070995881463560  -0.0000071930401916
 8e-03   0.7043374663312676  -0.0027693148552800   0.7071049829223881  -0.0000017982641595
 4e-03   0.7057239167465070  -0.0013828644400405   0.7071063316202526  -0.0000004495662950
 2e-03   0.7064157978737740  -0.0006909833127736   0.7071066687949497  -0.0000001123915979
 1e-03   0.7067614018394579  -0.0003453793470897   0.7071067530886239  -0.0000000280979237
 5e-04   0.7069341196006462  -0.0001726615859013   0.7071067741617298  -0.0000000070248177
 2e-04   0.7070204574165473  -0.0000863237700003   0.7071067794299779  -0.0000000017565697
 1e-04   0.7070636210573866  -0.0000431601291609   0.7071067807464715  -0.0000000004400761
 6e-05   0.7070852015604032  -0.0000215796261444   0.7071067810775276  -0.0000000001090200
 3e-05   0.7070959914854029  -0.0000107897011447   0.7071067811557441  -0.0000000000308035
 2e-05   0.7071013863605913  -0.0000053948259563   0.7071067811848479  -0.0000000000016996
 8e-06   0.7071040837909095  -0.0000026973956381   0.7071067811775720  -0.0000000000089756
 4e-06   0.7071054324915167  -0.0000013486950309   0.7071067811921239   0.0000000000055763
 2e-06   0.7071061068563722  -0.0000006743301754   0.7071067811921239   0.0000000000055763
 1e-06   0.7071064440533519  -0.0000003371331957   0.7071067811921239   0.0000000000055763
 5e-07   0.7071066126227379  -0.0000001685638097   0.7071067811921239   0.0000000000055763
 2e-07   0.7071066969074309  -0.0000000842791167   0.7071067811921239   0.0000000000055763
 1e-07   0.7071067392826080  -0.0000000419039395   0.7071067811921239   0.0000000000055763
 6e-08   0.7071067616343498  -0.0000000195521977   0.7071067802608013  -0.0000000009257463
 3e-08   0.7071067690849304  -0.0000000121016172   0.7071067802608013  -0.0000000009257463
 1e-08   0.7071067765355110  -0.0000000046510366   0.7071067690849304  -0.0000000121016172
 7e-09   0.7071067690849304  -0.0000000121016172   0.7071067690849304  -0.0000000121016172
 4e-09   0.7071067690849304  -0.0000000121016172   0.7071067988872528   0.0000000177007052
 2e-09   0.7071068286895752   0.0000000475030276   0.7071067690849304  -0.0000000121016172
 9e-10   0.7071068286895752   0.0000000475030276   0.7071067094802856  -0.0000000717062619
 5e-10   0.7071068286895752   0.0000000475030276   0.7071065902709961  -0.0000001909155515
 2e-10   0.7071065902709961  -0.0000001909155515   0.7071070671081543   0.0000002859216067
 1e-10   0.7071075439453125   0.0000007627587649   0.7071065902709961  -0.0000001909155515

A more in-depth discussion about round-off erros in numerical differentiation can be found here

Special functions in numpy

numpy provides a simple method diff() to calculate the numerical derivatives of a dataset stored in an array by forward differences. The function gradient() will calculate the derivatives by midpoint (or central) difference, that provides a more accurate result.


In [9]:
%matplotlib inline
import numpy as np
from matplotlib import pyplot

y = lambda x: x*x

x1 = np.arange(0,10,1)
x2 = np.arange(0,10,0.1)

y1 = np.gradient(y(x1), 1.)
print y1

pyplot.plot(x1,np.gradient(y(x1),1.),'r--o');
pyplot.plot(x1[:x1.size-1],np.diff(y(x1))/np.diff(x1),'b--x');


[  1.   2.   4.   6.   8.  10.  12.  14.  16.  17.]

Notice above that gradient() uses forward and backward differences at the two ends.


In [39]:
pyplot.plot(x2,np.gradient(y(x2),0.1),'b--o');


More discussion about numerical differenciation, including higher order methods with error extrapolation can be found here.

The module scipy also includes methods to accurately calculate derivatives:


In [91]:
from scipy.misc import derivative

y = lambda x: x**2

dx = 1.
x = 1.

while(dx > 1.e-10):
    d = derivative(f, x, dx, n=1, order=3)
    print("%6.0e %20.16f %20.16f" % (dx, d, d-2.))
    dx = dx / 10.


 1e+00 2.000000000000000000 0.000000000000000000
 1e-01 2.000000000000000444 0.000000000000000444
 1e-02 2.000000000000001776 0.000000000000001776
 1e-03 1.999999999999835243 -0.000000000000164757
 1e-04 1.999999999999224620 -0.000000000000775380
 1e-05 2.000000000002000178 0.000000000002000178
 1e-06 2.000000000001999734 0.000000000001999734
 1e-07 2.000000000057510885 0.000000000057510885
 1e-08 1.999999993396172737 -0.000000006603827263
 1e-09 2.000000054458439092 0.000000054458439092
 1e-10 2.000000165480741554 0.000000165480741554

One way to improve the roundoff errors is by simply using the decimal package


In [126]:
from decimal import Decimal

dx = Decimal("1.")
while(dx >= Decimal("1.e-10")):
    x = Decimal("1.")
    dy = (x+dx)*(x+dx)-x*x
    d = dy / dx
    print("%6.0e %20.16f %20.16f" % (dx, d, d-Decimal("2.")))
    dx = dx / Decimal("10.")


 1e+00   3.0000000000000000   1.0000000000000000
 1e-01   2.1000000000000001   0.1000000000000000
 1e-02   2.0099999999999998   0.0100000000000000
 1e-03   2.0009999999999999   0.0010000000000000
 1e-04   2.0001000000000002   0.0001000000000000
 1e-05   2.0000100000000001   0.0000100000000000
 1e-06   2.0000010000000001   0.0000010000000000
 1e-07   2.0000000999999998   0.0000001000000000
 1e-08   2.0000000099999999   0.0000000100000000
 1e-09   2.0000000010000001   0.0000000010000000
 1e-10   2.0000000001000000   0.0000000001000000