Before we jump into to core contents of the course, we need to define some terminology that we will use for the rest of the course.
A signal is defined as any physical quantity carrying information that varies with one or more independent variables, such a time, space etc. Mathematically, a signal can be represented a function of independent variable(s). For example,
$$s\left(t\right) = 5t$$$$s\left(t\right) = 1.23t^2 - 5.11t +41.5$$Where, $t$ is time. These two are examples of signals 1-dimensional (1-D) signals - functions of a single independent variable. An image file is an 2-D signals that is a function of two independent spatial coordiates $x$ and $y$.
$$s\left(x,y\right) = x^2 + y^2 + 2xy$$The mathematical representation of signals will not be possible for all types of signals. For example, many of the physiological signals cannot be represented mathematically, either because the exact function is not known or is too complicated.
In [4]:
# Initialization code to load the required moduels
%pylab inline
import seaborn
In [6]:
t = np.arange(-10, 10, 0.01)
s1 = 5 * t
s2 = 1.23 * (t ** 2) - 5.11 * t + 41.5
figure(figsize=(13.5,3))
subplot2grid((1,2), (0,0), rowspan=1, colspan=1)
plot(t, s1, label="$5t$")
xlabel('Time', fontsize=15)
legend(prop={'size':20})
subplot2grid((1,2), (0,1), rowspan=1, colspan=1)
plot(t, s2, label="$1.23t^2 - 5.11t + 41.5$")
xlabel('Time', fontsize=15)
legend(prop={'size':20});
The above two signals are examples of 1D signals. Can you think of an examples of a 3-D and 4-D signals?
One type of classification of signals has already been mentioned in the previous section based on the number of independent variables associated with the signal. The rest of the sections describes some of the other classifications of signals.
Signals can take one or more values for each value(s) of its independent variables. Scalar signals take on only one value, while vector signals take more than one value. A gray scale image is an example of a scalar 2-D signal, while a RGB image is an example foa vector 2-D signals (with each point on the image taking three values).
$$ I\left(x, y\right) = i$$Where, $i \in [0, 1]$ is the intensity of the point with 0 corresponding to black and 1 corresponding to white colors. An RGB image on the other hand can be represented as the following,
$$ \mathbf{I}\left(t\right) = \begin{pmatrix}r & g & b\end{pmatrix}^{T} $$Where, $r, g, b \in [0, 1]$ correpond to the amount of red, green and blue components in each point in the image. The different combinations of these three primary colors results in the different colors in the image.
An important classification of signals is based on the nature of the values assumed by the independent variable of a singal. Continuous-time signals are ones whose independent variable can take on any value in continuous interval $\left(a, b\right)$ on the real axis, and $a$ can be $\infty$ and $b$ can be $-\infty$. For example, a function $e^{-0.1t^{2}}$ with $t \in \left(-\infty, \infty\right)$ is an example of a continuous-time signal.
On the other hand, discrete signals can take on values only for specific values of the independent variable. They can be represented in a tabular form that represents the mapping from the signals domain to its range. For example consider a discrete signal $x$ that maps members from the set $A$ to set $B$. $$ x: \begin{pmatrix} \vdots \\ a_{n} \\ a_{n+1} \\ a_{n+2} \\ \vdots \end{pmatrix} \mapsto \begin{pmatrix} \vdots \\ b_{n} \\ b_{n+1} \\ b_{n+2} \\ \vdots \end{pmatrix} \implies x\left[a_{n}\right] = b_{n} $$
In general, the values of the independent variable $a_{n}$ need not be equidistant, but in practice these are taken to be equally spaced for computational and mathematical convenience. When the values of the domain are equidistant, with the difference between two successive values $a$, then the function representation becomes, $x\left[na\right] = b_{n}$ or simple $x\left[n\right] = b_{n}$, where $n \in \mathbb{Z}$. We will use the square bracket notation for discrete-time signals, and the simple brackets for continuous-time signals. Consider the example of the continuous-time signal given above, $x\left(t\right) = e^{-0.1*t^{2}}, t \in \mathbb{R}$. The discrete-time version of this signal is $x\left[n\right] = e^{-0.1*n^{2}}, t \in \mathbb{Z}$. The plot of these two signals is shown below.
Note: Although these signals are called continuos-time and discrete-time, there is not restriction on the choice of the independent variable - it can be time, space, etc. The name 'time' is used in the naming convention as time signals are the most commonly encountered signals in practice.
In [5]:
t = np.arange(-10, 10.01, 0.01)
n = np.arange(-10, 11, 1.0)
x_t = np.exp(-0.1 * (t ** 2)) # continuous signal
x_n = np.exp(-0.1 * (n ** 2)) # discrete signal
fig = figure(figsize=(13.5,3))
plot(t, x_t, label="$e^{-0.1*t^{2}}$")
stem(n, x_n, label="$e^{-0.1*n^{2}}$", basefmt='.')
ylim(-0.1, 1.1)
xlabel('Time', fontsize=15)
legend(prop={'size':20});
In the above example, the discrete-time signal was obtained by selecting the values of the continuous-time signal at specific (equidistant) time points. This is the process of sampling, which will be dealt in details in Chapter 4.
Similar to the previous classification, this classification is based on the nature of values taken by the signal at different values of its independent variables. When a signal can take any value in a continuous interval in $\mathbb{R}$, it is called a continuous-valued signal, while a discrete-valued signal can only take on specific values from a set of values. In the case of discrete-valued signals, the set of values the signal can take are usually equidistant. An example of a continuous-valued $\left(x\left(t\right)\right)$ and discrete-valued $\left(\hat{x}\left(t\right)\right)$ signals are shown in the following figure.
In [7]:
t = np.arange(-10, 10.01, 0.01)
# number of discrete values
n_steps = 10.
# continuous-valued signal
x_c = np.exp(-0.1 * (t ** 2))
# discrete-valued signal
x_d = (1/n_steps) * np.round(n_steps * x_c)
fig = figure(figsize=(13.5,3))
plot(t, x_c, label="$x(t)$")
plot(t, x_d, label="$\hat{x}(t)$")
ylim(-0.1, 1.1)
xlabel('Time', fontsize=15)
legend(prop={'size':20});
In the above example, the $\hat{x}\left(t\right)$ is the discrete-valued version of $x\left(t\right)$, and it can only take on values in the set $\{\dotsc, -0.2, -0.1, 0, 0.1, 0.2, \dotsc\}$. The conversion of $x\left(t\right)$ to $\hat{x}\left(t\right)$ is called quantization, which will be discussed in Chapter 4.
The last two classifications can be combined to have four possible combinations of signals:
These four cases are demonstrated in the following four figures.
In [8]:
t = np.arange(-10, 10.01, 0.01)
n = np.arange(-10, 11, 0.5)
n_steps = 5.
# continuous-time continuous-valued signal
x_t_c = np.exp(-0.1 * (t ** 2))
# continuous-time discrete-valued signal
x_t_d = (1/n_steps) * np.round(n_steps * x_t_c)
# discrete-time continuous-valued signal
x_n_c = np.exp(-0.1 * (n ** 2))
# discrete-time discrete-valued signal
x_n_d = (1/n_steps) * np.round(n_steps * x_n_c)
figure(figsize=(13.5,6))
subplot2grid((2,2), (0,0), rowspan=1, colspan=1)
plot(t, x_t_c,)
ylim(-0.1, 1.1)
title("Continuous-time Continuous-valued", fontsize=15)
subplot2grid((2,2), (0,1), rowspan=1, colspan=1)
plot(t, x_t_d,)
ylim(-0.1, 1.1)
title("Continuous-time Discrete-valued", fontsize=15)
subplot2grid((2,2), (1,0), rowspan=1, colspan=1)
stem(n, x_n_c, basefmt='.')
ylim(-0.1, 1.1)
xlim(-10, 10)
title("Discrete-time Continuous-valued", fontsize=15)
subplot2grid((2,2), (1,1), rowspan=1, colspan=1)
stem(n, x_n_d, basefmt='.')
ylim(-0.1, 1.1)
xlim(-10, 10)
title("Discrete-time Discrete-valued", fontsize=15);
This classification is based on the symmerty of signals about the vertical axis. A continous-time signal $x\left(t\right)$ is called even, if
$$ x\left(t\right) = x\left(-t\right), \,\, \forall t $$The signal $x\left(t\right)$ is called odd, if
$$ x\left(t\right) = -x\left(-t\right), \,\, \forall t $$Even signals are symmertric about the vertical axis (about the point $t = 0$), while odd signals are antisymmetric about the vertical axis. This classification also applied to discrete-time signals.
An interesting property of signals is that any signal $x\left(t\right)$ (even, odd or neigther) can be decomposed into an even and odd component. Lets us supposed that $x\left(t\right)$ can be decomposed into an even $x_{e}\left(t\right)$ and an odd $x_{o}\left(t\right)$ component.
$$ x\left(t\right) = x_{e}\left(t\right) + x_{o}\left(t\right) \implies x\left(-t\right) = x_{e}\left(t\right) - x_{o}\left(t\right) $$Thus, $$ x_{e}\left(t\right) = \frac{x\left(t\right) + x\left(-t\right)}{2} \,\,\, \& \,\,\, x_{o}\left(t\right) = \frac{x\left(t\right) - x\left(-t\right)}{2} $$
A periodic signal is one that repeats itself after a fixed finite value of time (or the corresponding independent variable), i.e. if the continuous-time $x\left(\right)$ is periodic, then
$$ x\left(t\right) = x\left(t + T\right), \,\,\, \forall t $$where $T$ is a positive constant. If $T$ is the smallest positive value for which the above relationship is satisfied, then $T$ is called the fundamental period of the signal, and the following is also true.
$$ x\left(t\right) = x\left(t + T\right) = x\left(t + 2T\right) = x\left(t + 3T\right) \dotsc , \,\,\, \forall t $$The inverse of $T$ is called the fundamental frequency.
Any signal that does not satisfy the above condition is called an aperiodic or non-periodic signal.
In the case of a discrete-time signal $x\left[n\right]$, the following condition must be satisfied for the signal to be periodic,
$$ x\left[n\right] = x\left[n + N\right], \,\,\, \forall n \in \mathbb{Z} \,\,\, \& \,\,\, N \in \mathbb{Z} $$The fundamental period of a discrete-time signal must be an integer.
All the signals we seen so far have been descriveds through a mathematical expression. Such an expression allows one to completely characterize the signal on the domain over which it is defined. Such signals are termed deterministic. The representation of deterministic signals is not limited to mathematical expressions; a table of values or a well-defined rule for its calculation would also do as well. These explicit representations of the signal is known as the signal model. Through the signal model, the value of the signal can be be predicted for any value of its arguments in its domain.
A stochastic or random signal is one that cannot be represented by an explicit mathematical expression to any reasonable accuracy, or it is exceedingly complicated to do so for any practical purposes. Stochastic signals evolve in an unpredicatable fashion as a functions of its arguments. For example, the EMG signal recorded using surface electrodes from a muscle is a stochastic signal. Unlike deterministic signals, which are ammenable to classical techniques of mathematical analysis, statistical techniques are the main tools for analysing stochastic signals.
Consider a continuous-time signals $v\left(t\right)$ and $i\left(t\right)$, which represent the voltage across and the current through a resistor $R$, respectively. Then the instantaneous power dissipated in the resistor is,
$$ P\left(t\right) = \frac{v^{2}\left(t\right)}{R} = Ri^{2}\left(t\right) $$In both cases the power is proportional to the resistance $R$. When the value of the resistance in 1omh, then the instantaneous power takes the same form for both $v\left(t\right)$ and $i\left(t\right)$.
In signal processing, regardless of the type of signal represented by $x\left(t\right)$, the instantaneous power is defined as,
$$ p\left(t\right) = x^{2}\left(t\right) $$Thus the total energy in a signal $x\left(t\right), \,\,\, t \in (-\infty, \infty)$ is given by,
$$ E = \int_{-\infty}^{\infty} x^{2}\left(t\right) dt $$and the average power is defined as, $$ P = \lim_{T \to \infty} \frac{1}{T} \int_{-T/2}^{T/2} x^{2}\left(t\right) dt $$
Similar definitions can be given for discrete-time signals $\left(x\left[n\right]\right)$ as well, by replacing the integrals by sums. The total energy in $x\left[n\right]$ is,
$$ E = \sum_{n=-\infty}^{\infty} x^{2}\left[n\right] $$And the average power is, $$ P = \lim_{N \to \infty} \sum_{n=-N}^{N} x^{2}\left[n\right] $$
Based on these definitions, an energy signal is defined as any signal whose total energy $E$ is finite. i.e. $$ 0 < E < \infty $$
While a signal is called a power signal if its average power is finite. $$0 < P < \infty$$
Thus, an energy signal has zero time averaged power, and a power signal have infinite energy. Periodic and random signals are usually viewed as power signals, while deterministic and non-periodic signals are viewed as energy signals.
A system is any physical device or algorithm that performs some operation on a singal to transform it into another signal. Mathematically, systems can be thought of as functions or operators that map a signal to another signal. For example, an amplifier increases the amplitude of a signals, a filter removes components of a signals that are considered unwanted, etc. The following figure show a schematic of a system that takes a signal as its input, processes the signal and provides an output processed signal.
Systems are characterized by the nature of the operations that they perform on an input signal. The different classifications of systems, similar to that of singals, are characterized by the type of signals they deal with and the type of opertaions they perform on signals. Instead of presenting the different types of clasifications, like the one done for signals, here present some of the general characteristrics of systems which can be used as a basis for classifying systems:
Linearity is an important properpty. All systems we will study and design in this book will be systems that are linear. A system is said to be linear when it satisfies the properties of scaling and superposition. Consider a system $f$ that operates on a set of signals $x_{i}\left(t\right), i \in \{1, 2, \dotsc n\}$ and produces the outputs $y_{i}\left(t\right)$ respectively, i.e.
$$ y_{i}\left(t\right) = f\left(x_{i}\left(t\right)\right), \, i \in \{1, 2, \dotsc n\} $$Then, the system $f$ is linear if and only if the following is true.
$$ f\left(\sum_{i} a_{i}x_{i}\left(t\right)\right) = \sum_{i}a_{i}y_{i}\left(t\right) $$where, $a_i$s are some arbitrary constants. The scaling and superpostion properties are both contained in the previous statement.
Scaling refers to the multiplication of the input signals by an arbitrary constant $a$ results in the an output that is also equally scaled (multiplied by $a$), i.e.
$$ f: x_1 \mapsto y_1 \implies ax_1 \mapsto ay_1 $$The principle of superposition says that if we know the output of a linear system to a set of inputs $x_{i}, i \in \{1, 2, 3, \dotsc n\}$, then the output of the system to the sum of these inputs is simply the sum of the outputs corresponding to these individual inputs. i.e.
$$ f: x_{i} \mapsto y_{i} \implies \sum_{i} x_{i} \mapsto \sum_{i} y_{i} $$Linearity is a very important property, and all systems that we will analyse and design in the rest of this book will be based on the linearity assumption, i.e. linear systems. Any system that does not satifty the scaling and superposition properties are known as non-linear systems. Almost all real systems are non-linear, but it will be convenient to assume that they are linear (or approximately linear) in order to use the tools of signal processing and systems theory.
The property of memory is easy to understand in the context of a system that operates on time domain signals. A system is said to have memory if its behavior depends on the past (and may be future) values of its input. On the contrary, it is said to be memoryless if its behaiour only depends on the current values of its input.
The followings are the input output relations for the two systems that are shown in the above figure. The left circuit is a simple voltage divider, and the other one is a simple RC circuit.
For the voltage divider (memoryless system):
$$ v_{out}\left(t\right) = \frac{R2}{R1 + R2}v_{in}\left(t\right) = 0.5v_{in}\left(t\right) $$For the RC circuit, the input-output relationships are slightly more complicated because of the presence of the capacitor. The relationship between $v_{in}\left(t\right)$ and $v_{out}\left(t\right)$ are governed by the following differential equation.
$$ C\frac{dv_{out}\left(t\right)}{dt} + \frac{v_{out}\left(t\right)}{R} = \frac{v_{int}\left(t\right)}{R} $$Solving this equation will give us the following relationship for $v_{out}\left(t\right)$
$$ v_{out}\left(t\right) = e^{-t/RC} \left(\frac{1}{RC}\int_{0}^{\infty} {e^{t / RC}{v_{in}\left(t\right)}}dt\right) + v_{out}\left(0\right)e^{-t/RC} $$The difference in the behavior of these two systems can be understood for looking at their response to the following pulse pulse input. $$ v_{in}\left(t\right) = \left\{ \begin{array}{l l} 1, & 1 \leq t < 2 \\ 0, & otherwise \end{array} \right. $$
The output of the two systems for this input are shown in the following figure.
In [3]:
time = np.arange(0, 10., 0.001)
# input voltage
v_in = 1.0 * np.array([time >= 1.0, time < 2.0]).all(0)
# output voltage for voltage divide
v_out_vd = 0.5 * v_in
# output voltage for RC circuit
R = 10000
C = 100e-6
RC = R * C
v_out_0 = 0.
v_out_rc = np.array([0 if t < 1.0
else ((1 - np.exp(-1/RC)) * np.exp(-(t-2.0)/RC)
if t >= 2.0
else (1 - np.exp(-(t - 1.0)/RC)))
for t in time])
figure(figsize=(11,6))
ax = subplot2grid((2,1), (0,0), rowspan=1, colspan=1)
plot(time, v_in, label="$v_{in}$")
plot(time, v_out_vd, lw=3, label="$v_{out}$")
ylim(-0.1, 1.1)
# xlabel('Time (s)', fontsize=20)
title('Memoryless system', fontsize=15)
legend(prop={'size':20})
subplot2grid((2,1), (1,0), rowspan=1, colspan=1, sharex=ax)
plot(time, v_in, label="$v_{in}$")
plot(time, v_out_rc, lw=3, label="$v_{out}$")
ylim(-0.1, 1.1)
xlabel('Time (s)', fontsize=15)
title('System with memory', fontsize=15)
legend(prop={'size':20})
tight_layout();
In the above two cases, the difference between a system with or without memory should be very clear. A memoryless system can generate an output that follows the input irrespective of how abruptly the input signal varies. On the other hand, a system will memory will be more sluggish and tend to "remember" its past values of the input. In the response of the RC circuit, $v_{out}\left(t\right)$ appears to not to want to change from the value $v_{out}\left(1\right) = 0$, when the input changes from 0 to 1 abruptly at $t = 1$. Similarly when the input falls back to 0 abruptly, $v_{out}\left(t\right)$ does not appear to want to change from the value of $v_{out}\left(2\right) \approx 0.6$, and falls back to 0 only aftera while. This clearly demonstrates the effect of memory in a system.
A system is said to be causal if its output depends only on the present and past values of its inputs, and not on the future values. While, a non-causal system's output depends also on the future values of the input signal. Causality and non-causality are purely based on what is considered the present. The choice of what is the present is fixed for systems that operate in real-time. For example, a filter t that is operating in real-time is strictly causal. However, a system that works offline on stored data (nonreal time) is free to define its present, and thus use data from the future to generate its output.
Time-invariance property deals with whether or not a system changes over time. This change over time is characterized by the input-output relationship of the system. A system is said to time-invariant if its input-output realtionship does not change with time, i.e. one get the same output from the system for a given input independnt of whether the input was applied now, 10 minutes eariler or a year from now. Any system where this property does not hold is termed as time-variant. Time-invariance is another property that we will assume in the systems that we analyze and design in the upcoming chapters.
Stability refers to the property of a system to produce limited output when provided with finite input. Such systems are called stable system in the bounded-input bounded-output (BIBO) sense. Systems that do not satify the BIBO criteria are called unstable systems.
A system is said to invertible if an inverse of this system can be constructed, i.e. if a system $f$ maps $x\left(t\right)$ to $y\left(t\right)$, then the inverse of the system will map $y\left(t\right)$ to $x\left(t\right)$.
In [1]:
from IPython.core.display import HTML
def css_styling():
styles = open("../styles/custom.css", "r").read()
return HTML(styles)
css_styling()
Out[1]: