In one implementation of forward-mode automatic differentiation (autodiff), we use "dual numbers" to carry forward partial derivatives along with our calculation. A dual number is similar to a complex number in that it has a real valued component, and an extra component. In the case of dual numbers the extra component is an "infinitesimal" component $\epsilon$. Whereas in complex numbers, $i$ is defined by $i^2 = -1$, for dual numbers, $\epsilon$ is defined by $\epsilon^2 = 0$.
In :# create a dual number type immutable Dual value::Float64 eps::Float64 end
We'll attempt to differentiate the function$$f(x) = x^2 + x \sin(x)$$
which seems moderately interesting.
In :# define a test function: f(x) = x^2 + x * sin(x)
Out:f (generic function with 1 method)
So we'll need to know how to multiply, add and take the sine of our
In :import Base: +, *, sin
In :*(x::Dual, y::Dual) = Dual(x.value * y.value, x.value * y.eps + y.value * x.eps)
Out:* (generic function with 152 methods)
In :+(x::Dual, y::Dual) = Dual(x.value + y.value, x.eps + y.eps)
Out:+ (generic function with 164 methods)
In :sin(x::Dual) = Dual(sin(x.value), cos(x.value) * x.eps)
Out:sin (generic function with 11 methods)
Sweet! Now we'll try to run it!
In :x = Dual(3., 1.) f(x)
In :fprime(x) = 2 * x + sin(x) + x * cos(x)
Out:fprime (generic function with 1 method)