Linear algebra crash course
Roberto Di Remigio, Luca Frediani
This is a very brief and incomplete introduction to a few selected topics in linear algebra. An extremely well-written book, covering many topics in mathematics heavily used in physics is James Nearing's Mathematical Tools for Physics. The book is freely available online on the author's website. These notes were translated from the italian version, originally written by Filippo Lipparini and Benedetta Mennucci.
Vector spaces and bases
Definition: Vector space
Let $\mathbb{R}$ be the set of real numbers. Elements of $\mathbb{R}$ will be called scalars, to distinguish them from vectors. We define as a vector space $V$ on the field $\mathbb{R}$ the set of vectors such that:
- the sum of two vectors is still a vector. For any pair of vectors $\mathbf{u},\mathbf{v}\in V$ their sum $\mathbf{w} = \mathbf{u}+\mathbf{v} = \mathbf{v} + \mathbf{u}$ is still an element of $V$.
- the product of a vector and a scalar is still a vector. For any $\mathbf{v}\in V$ and for any $\alpha \in \mathbb{R}$ the product $\mathbf{w} = \alpha\mathbf{v}$ is still an element of $V$.
We will write vectors using a boldface font, while we will use normal font for scalars.
Real numbers already are an example of a vector space: the sum of two real numbers is still a real number, as is their product. Vectors in the plane are yet another example (when their starting point is the origin of axes): a sum of two vectors, given by the parallelogram rule, is still a vector in the plane; the product of a vector by a scalar is a vector with direction equal to that of the original vector and magnitude given by the product of the scalar and the magnitude of the original vector.
How big is a vector space? It is possible to get an intuitive idea of dimension for ``geometric'' vector spaces: the plane has dimension 2, the space has dimension 3, while the set of real numbers has dimension 1. Some more definitions will help in getting a more precise idea of what the dimension of a vector space is.
Definition: Linear combination
A linear combination of vectors $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$ is the vector obtained by summing these vectors, possibly multiplied by a scalar: \begin{equation} \mathbf{w} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \ldots + c_n\mathbf{v}_n = \sum_{i = 1}^n c_i\mathbf{v}_i \end{equation}
Definition: Linear dependence
Two non-zero vectors are said to be linearly dependent if and only if (iff) there exist two non-zero scalars $c_1, c_2$ such that: \begin{equation} c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0} \end{equation} where $\mathbf{0}$ is the null vector. Equivalently, two vectors are said to be linearly dependent if one can be written as the other multiplied by a scalar: \begin{equation} \mathbf{v}_2 = -\frac{c_1}{c_2} \mathbf{v}_1 \end{equation} Conversely, two vectors are said to be linearly independent when the following is true: \begin{equation} c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0} \quad\Leftrightarrow\quad c_1=c_2 = 0 \end{equation}
Consider now two linearly dependent vectors in the plane. What does this imply? As one can be expressed as a scalar multiplied by the other, they have the same direction and magnitude differing by a factor.
The notions above are easily generalized to more than a pair of vectors: vectors $\mathbf{v}_1, \mathbf{v}_2, \ldots\mathbf{v}_n$ are linearly independent iff the only linear combination that gives the null vector is the one where all the coefficients are zero: \begin{equation} \sum_{i=1}^n c_n\mathbf{v}_n = \mathbf{0} \quad\Leftrightarrow\quad c_i = 0 \forall i \end{equation}
How many linearly independent vectors are there in the plane? As our intuition told us before: only 2! Let us think about the unit vectors $\mathbf{i}$ and $\mathbf{j}$, i.e. the orthogonal vectors of unit magnitude we can draw on the Cartesian $x$ and $y$ axes. All the vectors in the plane can be written as a linear combination of those two: \begin{equation} \mathbf{r} = x\mathbf{i} + y\mathbf{j} \end{equation} We are already used to specify a vector by means of its components $x$ and $y$. So, by taking linear combinations of the two unit vectors we are able to generate all the vectors in the plane. Doesn't this look familiar? Think about the set of complex numbers $\mathbb{C}$ and the way we represent it. It's now time for some remarks:
- the orthogonal, unit vectors in the plane are linearly independent;
- they are the minimal number of vectors needed to span, i.e. to generate, all the other vectors in the plane.
Definition: Basis
Given the vector space $V$ its basis is defined as the minimal set of linearly independent vectors $\mathcal{B} = \lbrace \mathbf{e}_1,\mathbf{e}_2, \ldots,\mathbf{e}_n \rbrace$ spanning all the vectors in the space. This means that any suitable linear combination of vectors in $\mathcal{B}$ generates a vector $\mathbf{v}\in V$: \begin{equation} \mathbf{v} = c_1\mathbf{e}_1 + c_2\mathbf{e}_2 + \ldots + c_n\mathbf{e}_n = \sum_{i=1}^n c_n \mathbf{e}_n \end{equation} the coefficients in the linear combination are called components or coordinates of the vector $\mathbf{v}$ in the given basis $\mathcal{B}$.
Turning back to our vectors in the plane, the set $\lbrace \mathbf{i}, \mathbf{j} \rbrace$ is a basis for the plane and that that $x$ and $y$ are the components of the vector $\mathbf{r}$. We are now ready to answer the question: how big is a vector space?
Defintion: Dimension of a vector space
The dimension of a vector space is the number of vectors making up its basis.
Vectors using NumPy
We have already seen how to use NumPy to handle quantities that are vector-like. For example, when we needed to plot a function with matplotlib, we first generated a linearly spaced vector of $N$ points inside an interval and then evaluated the function in those points.
Vectors, matrices and objects with higher dimensions are all represented as arrays.
An array is a homogeneous grid of values that is indexed by a tuple of nonnegative integers.
There are two important terms related to NumPy arrays:
- the rank of an array is the number of dimensions,
- the shape is a tuple of nonnegative integers, giving the size of the array along each dimension.
Let's see what that means by starting with the simplest array: a one-dimensional array.
In [3]:
import numpy as np
a = np.array([1, 2, 3]) # Create a rank 1 array
print(type(a)) # Prints "<type 'numpy.ndarray'>"
print(a.shape) # Prints "(3,)"
As you can see, we can easily access all this information. One-dimensional arrays in NumPy can be used to represent vectors, while two-dimensional arrays can be used to represent matrices.
a = np.array([1, 2, 3]) is equivalent to the vector:
\begin{equation}
\mathbf{a} = \begin{pmatrix}
1 \\
2 \\
3
\end{pmatrix}
\end{equation}
The components of the vector can be accessed using so-called subscript notation, as in the following code snippet:
In [5]:
print(a[0], a[1], a[2]) # Prints "1 2 3"
a[0] = 5 # Change an element of the array
print(a) # Prints "[5, 2, 3]"
index counting in Python starts from 0 not from 1! As you can see, subscript notation can also be used to modify the value at a given index.
There a number of predefined functions to create arrays with certain properties.
We have already met linspace and here is a list of some additional functions:
In [6]:
a = np.zeros(10) # Create an array of all zeros
print(a)
b = np.ones(10) # Create an array of all ones
print(b) # Prints "[[ 1. 1.]]"
c = np.full(10, 7) # Create a constant array
print(c) # Prints "[[ 7. 7.]
# [ 7. 7.]]"
e = np.random.random(10) # Create an array filled with random values
print(e)
NumPy supports indexing arrays via slicing like Matlab does. While indexing via a single integer number will return the element at that position of the array, i.e. a scalar, slicing will return another array with fewer dimensions. For example:
In [8]:
a = np.random.random(15) # Create an array of rank 15
print(a.shape)
print(a)
# Select elements 10, 11 and 15 of the array
print(a[9], a[10], a[14])
# Now take a slice: all elements, but the first and second
print('Taking a slice!')
b = a[2:]
print(b)
# And notice how the rank changed!
print('We have removed two elements!')
print(b.shape)
print('Take another slice: all elements in between the third and tenth')
c = a[2:11]
print(c)
As you can see, we can quite easily select just parts of an array and either create new, smaller arrays or just obtain the value at a certain position. Algebraic operations and functions are also defined on arrays. The definitions for addition and subtraction are obvious. Those for multiplication, division and other mathematical functions are less obvious: in NumPy they are always performed element-wise. The following example will clarify.
In [9]:
a = np.full(10, 7.)
b = np.full(10, 4.)
print(a)
print(b)
print('Sum of a and b')
print(a + b)
print('Difference of a and b')
print(a - b)
print('Elementwise product of a and b')
print(a * b)
print('Elementwise division of a and b')
print(a / b)
print('Elementwise square root of b')
print(np.sqrt(b))
In [ ]: