The inverse Matrix Theorem

The inverse matrix theorem is a statement about a number of equivalent conditions that we can impose on a matrix.

Given a matrix $A$ with shape $(n, n)$

$Det(A) \neq 0$ implies $A^{-1}$ exists.
We can find a Matrix B such that $A * B = B * A = np.eye(n) = I_n$
$Det(A) = \lambda _1 * \lambda _2 * ... * \lambda _n$ where $\lambda _i$ is an eigenvalue of the matrix.

These require some explanation.

Determinants.

Linear Transformations may stretch the underlying space by some magnitude (eigenvalues) along certain vectors (eigenvectors). These are the vectors and values which "characterize a matrix." -- Hence the word eigen.



In [1]:

    
import numpy as np
import numpy.linalg as la



In [2]:

    
A = np.array(range(1,5)).reshape(2,2)
determinant_A = la.det(A)

print(A)
print("Determinant is: {}".format(determinant_A)) # Notice the rounding error.









    



[[1 2]
 [3 4]]
Determinant is: -2.0000000000000004

Hmm.... is this related to anything??



In [3]:

    
# Let's check it's eigenvalues.
print("The Matrix A has eigenvalues: {}".format([x for x in la.eigvals(A)]))
print("And their product is: {}".format(np.product(la.eigvals(A))))
determinant_A - np.product(la.eigvals(A)) < 10**-5









    



The Matrix A has eigenvalues: [-0.37228132326901431, 5.3722813232690143]
And their product is: -1.9999999999999998






    Out[3]:





True



In [4]:

    
# Lets define an epsilon and check if they are the same.
eps = 10 ** -5
def check_floats_equal(float_1, float_2, eps=eps):
    return float_1 - float_2 < eps

check_floats_equal(np.product(la.eigvals(A)), la.det(A))









    Out[4]:





True

Hmm... Interesting.

The Determinant of a matrix is just the product of it's eigenvalues.



In [5]:

    
vals, vecs = la.eig(A)
print(vecs)









    



[[-0.82456484 -0.41597356]
 [ 0.56576746 -0.90937671]]



In [6]:

    
# Lets define a distance function for vectors in a matrix.
def l2_distance_cols(matrix):
    norms = []
    for row in matrix.transpose():
        dist = 0
        for val in row:
            dist += val ** 2
        
        norms.append(dist)
    
    return np.array(norms)
        
l2_distance_cols(vecs)









    Out[6]:





array([ 1.,  1.])

It looks like the eigenvectors are normalized to length 1.

Let's look at another matrix and look at its eigenvalues and eigenvectors. This one should be a bit simpler.



In [7]:

    
A = np.array([0, -1, 1, 0]).reshape(2, 2)



In [8]:

    
A









    Out[8]:





array([[ 0, -1],
       [ 1,  0]])



In [9]:

    
np.linalg.det(A)









    Out[9]:





1.0



In [10]:

    
vals, vecs = np.linalg.eig(A)
print("Values:\n{}".format(vals))
print("Vectors:\n{}".format(vecs))









    



Values:
[ 0.+1.j  0.-1.j]
Vectors:
[[ 0.70710678+0.j          0.70710678-0.j        ]
 [ 0.00000000-0.70710678j  0.00000000+0.70710678j]]

Hmm.... what is happening there, why does it have imaginary eigenvectors??



In [11]:

    
# First a quick vector making helper function.
def list_to_vector(items):
    return np.array(items).reshape(len(items), 1)



In [12]:

    
np.matmul(A, list_to_vector([1,0]))









    Out[12]:





array([[0],
       [1]])



In [13]:

    
np.matmul(A, list_to_vector([0,1]))









    Out[13]:





array([[-1],
       [ 0]])

This looks like a twist to me. It's sending $$ [1, 0]^T \mapsto [0,1]^T$$ $$ [0, 1]^T \mapsto [-1,0]^T$$

If it's a twist... how can it have a direction it stretches something in a straight line in... we'd have to imagine really hard...

Oh.

That's why it has imaginary eigenvectors -- they don't really exist. But they do symbolically and mathematically.

Well... according to the inverse matrix theorem, the fact that $Det(A) = 1 \neq 0$ implies that it has an inverse matrix. Lets check that out.



In [14]:

    
A_inv = np.linalg.inv(A)
print("Here is A^-1\n{}".format(A_inv))









    



Here is A^-1
[[ 0.  1.]
 [-1.  0.]]



In [15]:

    
np.matmul(A_inv, list_to_vector([1,0]))









    Out[15]:





array([[ 0.],
       [-1.]])



In [16]:

    
np.matmul(A_inv, list_to_vector([0,1]))









    Out[16]:





array([[ 1.],
       [ 0.]])

Hmm...

$$ [1,0]^T \mapsto [0,-1]^T \\ [0,1]^T \mapsto [1,0]^T $$

Oh!

It's a rotation in the opposite direction!! Neato gang!

What happens if we multiply the two together...?



In [17]:

    
I_2 = np.matmul(A, A_inv)
print("{} \n* \n{} \n=\n{}".format(A, A_inv, I_2))









    



[[ 0 -1]
 [ 1  0]] 
* 
[[ 0.  1.]
 [-1.  0.]] 
=
[[ 1.  0.]
 [ 0.  1.]]

We just get the identity matrix back -- just like we wanted!!

Lets take another example.

Fix a new matrix $A = 2 * I_2$



In [18]:

    
A = np.eye(2) * 2
print(A)









    



[[ 2.  0.]
 [ 0.  2.]]



In [19]:

    
vals, vecs = np.linalg.eig(A)
print("Vals:\n{}".format(vals))
print("Vecs:\n{}".format(vecs))









    



Vals:
[ 2.  2.]
Vecs:
[[ 1.  0.]
 [ 0.  1.]]

What should the inverse of $A$ be?



In [20]:

    
A_inv = np.linalg.inv(A)
print("The inverse is:\n{}".format(A_inv))









    



The inverse is:
[[ 0.5  0. ]
 [ 0.   0.5]]

Hmm... We have $$A = 2 * I_2$$ $$A^{-1} = \frac{1}{2} * I_2$$

It looks like it just stretches along the eigenvectors.



In [ ]: