Fourtier Transform of a Data Set

20th January 2014

Neil D. Lawrence

Imagine we are given data and we wish to generalize from it. Without making further assumptions, we have no more information than the given data set. We can think of this ssomewhat like a weighted sum of Dirac delta functions. The Dirac delta function is defined to be a function with an integral of one, which is zero at all places apart from zero, where it is infinite. Given observations at particular times (or locations) $x_i$ we can think of our observations as being a function, $$ f(\mathbf{x}) = \sum_{i=1}^n y_i \delta(\mathbf{x}-\mathbf{x}_i), $$ This function is highly discontinuous, imagine if we wished to smooth it by filtering in Fourier space. The Fourier transform of a function is given by, $$ F(\boldsymbol{\omega}) = \int_{-\infty}^\infty f(\mathbf{x}) \exp\left(-i2\pi \boldsymbol{\omega}^\top \mathbf{x}\right) \text{d} \mathbf{x} $$ and since our function is a series of delta functions the the transform is easy to compute, $$ F(\boldsymbol{\omega}) = \sum_{i=1}^n y_i\exp\left(-i 2\pi \boldsymbol{\omega}^\top \mathbf{x}_i\right) $$ which has a real part given by a weighted sum of cosines and a complex part given by a weighted sum of sines.

One theorem that gives insight into covariances is Boechner's theorem. Boechner's theorem states that any positive filter in Fourier space gives rise to a valid covariance function. Further, it gives a relationship between the filter and the form of the covariance function. The form of the covariance is given by the Fourier transform of the filter, with the argument of the transform being replaced by the distance between the points.

Fourier space is a transformed space of the original function to a new basis. The transformation occurs through a convolution with a sine and cosine basis. Given a function of time $f(t)$ the Fourier transform moves it to a weighted linear sum of a sine and cosine basis, $$ F(\omega) = \int_{-\infty}^\infty f(t) \left[\cos(2\pi \omega t) - i \sin(2\pi \omega t) \right]\text{d} t $$ where is the imaginary basis, $i=\sqrt{-1}$. Through Euler's formula, $$ \exp(ix) = \cos x + i\sin x $$ we can re-express this form as $$ F(\omega) = \int_{-\infty}^\infty f(t) \exp(-i 2\pi\omega)\text{d} t $$ which is a standard form for the Fourier transform. Fourier's theorem was that the inverse transform can also be expressed in a similar form so we have $$ f(t) = \int_{-\infty}^\infty F(\omega) \exp(2\pi\omega)\text{d} \omega. $$ Although we've introduced the transform in the context of time Fourier's interest was an analytical theory of heat and the transform can be applied to a multidimensional spatial function, $f(\mathbf{x}).