Lecture 10 - eigenvalues and eigenvectors

An eigenvector $\boldsymbol{x}$ and corrsponding eigenvalue $\lambda$ of a square matrix $\boldsymbol{A}$ satisfy

$$ \boldsymbol{A} \boldsymbol{x} = \lambda \boldsymbol{x} $$

Rearranging this expression,

$$ \left( \boldsymbol{A} - \lambda \boldsymbol{I}\right) \boldsymbol{x} = \boldsymbol{0} $$

The above equation has solutions (other than $\boldsymbol{x} = \boldsymbol{0}$) if

$$ \det \left( \boldsymbol{A} - \lambda \boldsymbol{I}\right) = 0 $$

Computing the determinant of an $n \times n$ matrix requires solution of an $n$th degree polynomial. It is known how to compute roots of polynomials up to and including degree four (e.g., see http://en.wikipedia.org/wiki/Quartic_function). For matrices with $n > 4$, numerical methods must be used to compute eigenvalues and eigenvectors.

An $n \times n$ will have $n$ eigenvalue/eigenvector pairs (eigenpairs).

Computing eigenvalues with NumPy

NumPy provides a function to compute eigenvalues and eigenvectors. To demonstrate how to compute eigpairs, we first create a $5 \times 5$ symmetric matrix:


In [1]:
# Import NumPy and seed random number generator to make generated matrices deterministic
import numpy as np
np.random.seed(1)

# Create a symmetric matrix with random entries
A = np.random.rand(5, 5)
A = A + A.T
print(A)


[[0.83404401 0.81266309 0.41930889 0.97280008 0.94750046]
 [0.81266309 0.37252042 1.03078023 0.81407228 1.50707831]
 [0.41930889 1.03078023 0.4089045  1.43680726 0.34081177]
 [0.97280008 0.81407228 1.43680726 0.28077388 0.8904241 ]
 [0.94750046 1.50707831 0.34081177 0.8904241  1.7527783 ]]

We can compute the eigenvectors and eigenvalues using the NumPy function linalg.eig


In [2]:
# Compute eigenvectors of A
evalues, evectors = np.linalg.eig(A)

print("Eigenvalues: {}".format(evalues))
print("Eigenvectors: {}".format(evectors))


Eigenvalues: [ 4.49901636 -1.34808792 -0.66778843  0.21610602  0.94977508]
Eigenvectors: [[-0.40163425 -0.19049617  0.13563534 -0.88537464 -0.01076773]
 [-0.45887678  0.38587861  0.76218267  0.24145961  0.03611968]
 [-0.35255653 -0.62923828  0.03786448  0.30864498 -0.61892459]
 [-0.42177956  0.60360849 -0.53501774 -0.01546805 -0.41385451]
 [-0.57090098 -0.23350566 -0.33615908  0.24961576  0.66651049]]

The $i$th column of evectors is the $i$th eigenvector.

Symmetric matrices

Note that the above eigenvalues and the eigenvectors are real valued. This is always the case for symmetric matrices. Another features of symmetric matrices is that the eigenvectors are orthogonal. We can verify this for the above matrix:

We can also check that the second eigenpair is indeed an eigenpair (Python/NumPy use base 0, so the second eiegenpair has index 1):


In [3]:
import itertools

# Build pairs (0,0), (0,1), . . . (0, n-1), (1, 2), (1, 3), . . . 
pairs = itertools.combinations_with_replacement(range(len(evectors)), 2)

# Compute dot product of eigenvectors x_{i} \cdot x_{j}
for p in pairs:
    e0, e1 = p[0], p[1]
    print ("Dot product of eigenvectors {}, {}: {}".format(e0, e1, evectors[:, e0].dot(evectors[:, e1])))


Dot product of eigenvectors 0, 0: 0.9999999999999997
Dot product of eigenvectors 0, 1: 5.551115123125783e-17
Dot product of eigenvectors 0, 2: 5.551115123125783e-17
Dot product of eigenvectors 0, 3: -1.3877787807814457e-17
Dot product of eigenvectors 0, 4: -5.273559366969494e-16
Dot product of eigenvectors 1, 1: 1.0000000000000002
Dot product of eigenvectors 1, 2: -2.706168622523819e-16
Dot product of eigenvectors 1, 3: -4.85722573273506e-16
Dot product of eigenvectors 1, 4: 1.6653345369377348e-16
Dot product of eigenvectors 2, 2: 1.0000000000000002
Dot product of eigenvectors 2, 3: 2.220446049250313e-16
Dot product of eigenvectors 2, 4: -5.551115123125783e-17
Dot product of eigenvectors 3, 3: 1.0
Dot product of eigenvectors 3, 4: 7.979727989493313e-17
Dot product of eigenvectors 4, 4: 1.0000000000000002

In [4]:
print("Testing  Ax and (lambda)x: \n {}, \n {}".format(A.dot(evectors[:,1]), evalues[1]*evectors[:,1]))


Testing  Ax and (lambda)x: 
 [ 0.25680558 -0.5201983   0.84826852 -0.81371731  0.31478616], 
 [ 0.25680558 -0.5201983   0.84826852 -0.81371731  0.31478616]

Non-symmetric matrices

In general, the eigenvalues and eigenvectors of a non-symmetric, real-valued matrix are complex. We can see this by example:


In [5]:
B = np.random.rand(5, 5)
evalues, evectors = np.linalg.eig(B)

print("Eigenvalues: {}".format(evalues))
print("Eigenvectors: {}".format(evectors))


Eigenvalues: [ 2.43827549+0.j         -0.7356488 +0.j          0.95516424+0.j
  0.20592847+0.25009345j  0.20592847-0.25009345j]
Eigenvectors: [[-0.31712315+0.j         -0.3448319 +0.j         -0.48360642+0.j
   0.42721133-0.23453391j  0.42721133+0.23453391j]
 [-0.50795246+0.j         -0.55615526+0.j          0.29477162+0.j
  -0.25335016+0.19638993j -0.25335016-0.19638993j]
 [-0.46645352+0.j          0.01776781+0.j          0.53639466+0.j
   0.38172586-0.30291015j  0.38172586+0.30291015j]
 [-0.52413653+0.j          0.45219892+0.j         -0.62472838+0.j
  -0.55565352+0.j         -0.55565352-0.j        ]
 [-0.38615957+0.j          0.605791  +0.j          0.03506771+0.j
  -0.15322242+0.30005314j -0.15322242-0.30005314j]]

Unlike symmetric matrices, the eigenvectors are in general not orthogonal, which we can test:


In [6]:
# Compute dot product of eigenvectors x_{i} \cdot x_{j}
pairs = itertools.combinations_with_replacement(range(len(evectors)), 2)
for p in pairs:
    e0, e1 = p[0], p[1]
    print ("Dot product of eigenvectors {}, {}: {}".format(e0, e1, evectors[:, e0].dot(evectors[:, e1])))


Dot product of eigenvectors 0, 0: (0.9999999999999998+0j)
Dot product of eigenvectors 0, 1: (-0.08737921508729499+0j)
Dot product of eigenvectors 0, 2: (0.06733087339766464+0j)
Dot product of eigenvectors 0, 3: (0.16556046975206345+4.450334363795272e-05j)
Dot product of eigenvectors 0, 4: (0.16556046975206345-4.450334363795272e-05j)
Dot product of eigenvectors 1, 1: (1+0j)
Dot product of eigenvectors 1, 2: (-0.24890309336077585+0j)
Dot product of eigenvectors 1, 3: (-0.3437183199465273+0.14803892356676399j)
Dot product of eigenvectors 1, 4: (-0.3437183199465273-0.14803892356676399j)
Dot product of eigenvectors 2, 2: (0.9999999999999998+0j)
Dot product of eigenvectors 2, 3: (0.2652324961688288+0.019355072804240914j)
Dot product of eigenvectors 2, 4: (0.2652324961688288-0.019355072804240914j)
Dot product of eigenvectors 3, 3: (0.44927678773939966-0.6231089392651208j)
Dot product of eigenvectors 3, 4: (1.0000000000000004+0j)
Dot product of eigenvectors 4, 4: (0.44927678773939966+0.6231089392651208j)