Diagonalizing a Matrix

$ \mathbf{A} x_1 = \lambda_1 x_1 \\ \mathbf{A} x_2 = \lambda_2 x_2 \\ \mathbf{A} \times \begin{vmatrix} x_1 & x_2 \end{vmatrix} = \begin{vmatrix} \lambda_1 x_1 & \lambda_2 x_2 \end{vmatrix} = \begin{vmatrix} x_1 & x_2 \end{vmatrix} \times \begin{vmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{vmatrix} \\ THEN \\ \mathbf{A} \mathbf{V} = \mathbf{V} \mathbf{\Lambda} \\ SO \\ \mathbf{V}^{-1} \mathbf{A} \mathbf{V} = \mathbf{\Lambda} \\ AND \\ \mathbf{A} = \mathbf{V} \mathbf{\Lambda} \mathbf{V}^{-1} $

Powering

$ \mathbf{A}^2 = \mathbf{V} \mathbf{\Lambda} \mathbf{V}^{-1} \mathbf{V} \mathbf{\Lambda} \mathbf{V}^{-1} \\ = \mathbf{V} \mathbf{\Lambda} \mathbf{\Lambda} \mathbf{V}^{-1} \\ = \mathbf{V} \mathbf{\Lambda}^2 \mathbf{V}^{-1} \\ $

Powering to n

$ \mathbf{A}^n = \mathbf{V} \mathbf{\Lambda} \mathbf{V}^{-1} \mathbf{V} \mathbf{\Lambda} \mathbf{V}^{-1} ... \\ = \mathbf{V} \mathbf{\Lambda}^n \mathbf{V}^{-1} \\ $

Source

https://www.youtube.com/watch?v=U8R54zOTVLw

Markov Matrix

$ p_1 = \mathbf{A} p_0, p_2 = \mathbf{A} p_1 \\ p_2 = \mathbf{A} \mathbf{A} p_0 \\ p_2 = \mathbf{A}^2 p_0 \\ p_2 = \mathbf{V} \mathbf{\Lambda}^2 \mathbf{V}^{-1} p_0 $

**If $p_{n+1} = \mathbf{A} p_n$ then $p_{n} = \mathbf{A}^n p_0 = \mathbf{V} \mathbf{\Lambda}^n \mathbf{V}^{-1} p_0$**

Writing p_0 as combination of eigenvectors

$ p_0 = c_1 x_1 + c_2 x_2 ... c_n x_n => \mathbf{V}\mathbf{c} = p_0 => \mathbf{c} = \mathbf{V}^{-1} p_0\\ \mathbf{A} p_0 = p_1 = c_1 \lambda_1 x_1 + c_2 \lambda_2 x_2 ... c_k \lambda_k x_k \\ \mathbf{A}^n p_0 = p_n = c_1 \lambda_1^n x_1 + c_2 \lambda_2^n x_2 ... c_k \lambda_k^n x_k \\ = p_n = \mathbf{c} \mathbf{V} \mathbf{\Lambda}^n \\ = \mathbf{V} \mathbf{\Lambda}^n \mathbf{V}^{-1} p_0 $

Source

https://www.youtube.com/watch?v=xtMzTXHO_zA



In [12]:

    
import numpy as np
from scipy.linalg import eig, inv
from diffmaps_util import k, diag, sort_eigens



In [13]:

    
m = np.array([.8, .2, .5, .5]).reshape(2,2)
m









    Out[13]:





array([[ 0.8,  0.2],
       [ 0.5,  0.5]])



In [14]:

    
u0 = np.array([0,1])
for i in range(0,50):
    u0 = u0.dot(m)
print u0









    



[ 0.71428571  0.28571429]



In [15]:

    
w, v = eig(m)
print w.real
print v









    



[ 1.   0.3]
[[ 0.70710678 -0.37139068]
 [ 0.70710678  0.92847669]]



In [11]:

    
v.dot(inv(v).dot(u0))









    Out[11]:





array([ 0.71428571,  0.28571429])



In [24]:

    
m = np.random.randn(9).reshape(3,3)
L = k(m, .7)
D = diag(L)
m = inv(D).dot(L)
print m









    



[[ 0.94116975  0.00146505  0.05736519]
 [ 0.00155068  0.9961767   0.00227263]
 [ 0.05732609  0.00214567  0.94052823]]



In [31]:

    
w, v = eig(m)
w = w.real
print w
print v









    



[ 0.88350021  1.          0.99437447]
[[ 0.70516737 -0.57735027  0.39646056]
 [ 0.00459603 -0.57735027 -0.83158942]
 [-0.709026   -0.57735027  0.38894481]]



In [29]:

    
p0 = np.eye(len(m))

Building a diagonal eigenvalue matrix



In [33]:

    
lmbda = np.zeros((3,3))
np.fill_diagonal(lmbda, w)

$p_1 = p_0A$



In [36]:

    
p1 = p0.dot(m)
p1









    Out[36]:





array([[ 0.94116975,  0.00146505,  0.05736519],
       [ 0.00155068,  0.9961767 ,  0.00227263],
       [ 0.05732609,  0.00214567,  0.94052823]])

$p1 = \mathbf{V} \Lambda \mathbf{V}^{-1} p_0$



In [38]:

    
v.dot(lmbda).dot(inv(v)).dot(p0)









    Out[38]:





array([[ 0.94116975,  0.00146505,  0.05736519],
       [ 0.00155068,  0.9961767 ,  0.00227263],
       [ 0.05732609,  0.00214567,  0.94052823]])

$p2 = p1A$



In [39]:

    
p2 = p1.dot(m)
p2









    Out[39]:





array([[ 0.8890913 ,  0.0029614 ,  0.1079473 ],
       [ 0.00313448,  0.99237516,  0.00449036],
       [ 0.10787372,  0.00423952,  0.88788676]])

$p2 = \mathbf{V} \Lambda^2 \mathbf{V}^{-1} p_0$



In [40]:

    
v.dot(lmbda ** 2).dot(inv(v)).dot(p0)









    Out[40]:





array([[ 0.8890913 ,  0.0029614 ,  0.1079473 ],
       [ 0.00313448,  0.99237516,  0.00449036],
       [ 0.10787372,  0.00423952,  0.88788676]])