Taylor Approximation Theorem

If $f$ is smooth at $x=a$, then of all the polynomials of degree $N$ or less, the one which best approximates $f(x)$ for $x$ near $a$ is given by: $$P_N(x) = f(a) + f^{(1)}(a)(x-a) + \frac{f^{(2)}(a)}{2!}(x-a)^2 + \frac{f^{(3)}(a)}{3!}(x-a)^3 + \\ ... + \frac{f^{(N)}(a)}{N!}(x-a)^N$$ where $^{(N)}$ is the _Nth derivative_ of $f$.

In sigma notation: $$P_N(x) = \sum_{n=0}^{N} \frac{f^{(n)}(a)}{n!}(x-a)^n$$

Remember that (with respect to this formula):

  • $0! = 1$
  • $f^{(0)}(a)$ means the same thing as $f(a)$ (i.e. zero derivatives), and
  • $f^{(1)}(a)$ means the same thing as $f'(a)$, the first derivative of $f$ at $a$.

Nth-Order Taylor Polynomial of $f(x)$ at $x=a$

The polynomial $P_N$ is called the Nth-order Taylor polynomial of $f(x)$ at $x=a$.

Also, notice that it's called the Nth-"order" Taylor polynomial, not the Nth-"degree" Taylor polynomial.

That is becuase the degree of $P_N$ could be less than $N$; for instance, if the term $f^{(n)}(a)$ = 0, then the final term in the sum $P_N$ would disappear (i.e. be equal to zero), and the degree of $P_N$ would be at most $N-1$.

Most Important Property

The most important property of the Taylor polynomial is that: $P_N^{(n)}(a) = f^{(n)}(a)$ for all $n = 0, 1, ..., N$.

Meaning, the values of all the derivatives of $P_N$ and $f$ match when $x=a$, up to and including the Nth derivative.

But, all higher derivatives of $P_N$ must be zero everywhere.

The function $P_N$ is the distillation of all the information about $f$ which comes from its derivatives up to order $N$ at $x=a$.

These are my notes & observations from "[The Calculus Lifesaver](http://press.princeton.edu/video/banner/)," by Adrian Banner.


Example

Try this out with $f(x) = e^x$:

By the Taylor Approximation Theorem, with $N=3$ and $a=0$:

$$P_3(x) = f(0) + f'(0)(x)+\frac{f{(2)}(0)}{2!}(x^2) + \frac{f^{(3)}}{3!}(x^3)$$

Since all the derivatives of $f(x) = e^x$ (wrt $x$) are just $e^x$, $f(0), f'(0), f''(0), \text{ and } f^{(3)}(0)$ are all $e^0$. Also, $2! = 2$ and $3! = 6$, so you're left with:

$$P_3(x) = 1 + x + \left( \frac{1}{2} \right)x^2 + \left( \frac{1}{6} \right) x^3$$

In [8]:
import sympy as sp
from sympy.mpmath import exp

%matplotlib inline

# Customize figure size
plt.rcParams['figure.figsize'] = 25, 15
#plt.rcParams['lines.linewidth'] = 1
#plt.rcParams['lines.color'] = 'g'
#plt.rcParams['font.family'] = 'monospace'
plt.rcParams['font.size'] = '16.0'
plt.rcParams['font.monospace'] = 'Anonymous Pro, serif'
plt.rcParams['text.hinting'] = 'either'

f = lambda x: exp(x)

p = lambda x: 1 + x + (1/2)*(x**2)+ (1/6)*(x**3)

sp.mpmath.plot([f, p], xlim=[-5,5], ylim=[-5,15], points=500)