A Primer in Linear Algebra

This notebook presents some basic linear algebra.

Load Packages and Extra Functions

The notebook uses the functions printmat() and printlnPs() for formatted printing of numbers.


In [1]:
using Dates

include("jlFiles/printmat.jl")


Out[1]:
printyellow (generic function with 1 method)

Adding and Multiplying: A Matrix and a Scalar

With a matrix $A$ and a scalar $c$, do

  1. A*c (textbook: $Ac$) to multiply each element of $A$ by $c$

  2. A .+ c (textbook: $A+cJ$, where $J$ is a matrix of ones) to add $c$ to each element of $A$


In [2]:
A = [1 3;3 4]
c = 10

println("A:")
printmat(A)
println("c:")
printmat(c)

println("A*c:")
printmat(A*c)

println("A .+ c:")
printmat(A .+ c)          #notice the dot in .+


A:
         1         3
         3         4

c:
        10

A*c:
        10        30
        30        40

A .+ c:
        11        13
        13        14

Adding and Multiplying: Two Matrices

With two matrices of the same dimensions ($A$ and $B$), do

A+B (textbook: $A+B$) to add them (element by element)


In [3]:
A = [10;11]                 #A and B are 2x1 matrices
B = [2;5]

println("A:")
printmat(A)
println("B:")
printmat(B)

println("A-B:")
printmat(A-B)


A:
        10
        11

B:
         2
         5

A-B:
         8
         6


In [4]:
A = [1 3;3 4]               #A and B are 2x2 matrices
B = [1 2;3 -2]
println("A:")
printmat(A)
println("B:")
printmat(B)

println("A+B:")
printmat(A+B)


A:
         1         3
         3         4

B:
         1         2
         3        -2

A+B:
         2         5
         6         2

Multiplying matrices ($A$ and $B$) of conformable dimensions

A*B (textbook: $AB$)


In [5]:
A = [1 3;3 4]           #A and B are 2x2 matrices
B = [1 2;3 -2]
println("A:")
printmat(A)
println("B:")
printmat(B)

println("A*B:")
printmat(A*B)


A:
         1         3
         3         4

B:
         1         2
         3        -2

A*B:
        10        -4
        15        -2


In [6]:
A = [1 3;3 4]           #A is 2x2, B is a vector with 2 elements
B = [2;5]
println("A:")
printmat(A)
println("B:")
printmat(B)

println("A*B:")
printmat(A*B)


A:
         1         3
         3         4

B:
         2
         5

A*B:
        17
        26

Transpose

Transpose a matrix as A' (textbook: $A'$)


In [7]:
A = [10;11]
println("A: ")
printmat(A)
println("A': ")
printmat(A')

B = [1 2 3;4 5 6]
println("B: ")
printmat(B)
println("B': ")
printmat(B')


A: 
        10
        11

A': 
        10        11

B: 
         1         2         3
         4         5         6

B': 
         1         4
         2         5
         3         6

Inner and Outer Products, Quadratic Forms

Inner and outer products of (column) vectors with $k$ elements:

x'z (textbook: $x'z$) to get a scalar and x*z' (textbook: $xz'$) to get a $k\times k$ matrix


In [8]:
x = [10;11]
z = [2;5]
println("x and z")
printmat([x z])

println("x'z: ")
printlnPs(x'z)

println("x*z':")
printmat(x*z')


x and z
        10         2
        11         5

x'z: 
        75
x*z':
        20        50
        22        55

A quadratic form ($A$ is an $n \times n$ matrix and x is an $n$ vector)

x'A*x (textbook: $x'Ax$) to get a scalar


In [9]:
A = [1 3;3 4]

println("x:")
printmat(x)
println("A:")
printmat(A)

println("x'A*x: ")
printlnPs(x'A*x)


x:
        10
        11

A:
         1         3
         3         4

x'A*x: 
      1244

Matrix Inverse

A matrix inverse of an $nxn$ matrix $A$:

inv(A) or A^(-1) (textbook: $A^{-1}$)

The inverse is such that $AA^{-1}=I$ and $A^{-1}A=I$


In [10]:
println("A:")
printmat(A)

println("inv(A):")
printmat(inv(A))

println("inv(A)*A:")
printmat(inv(A)*A)


A:
         1         3
         3         4

inv(A):
    -0.800     0.600
     0.600    -0.200

inv(A)*A:
     1.000    -0.000
     0.000     1.000

OLS Notation

$X'X$ or $\sum\nolimits_{t=1}^{T}x_{t}x_{t}^{\prime}$?

Let $x_t$ be a (column) vector with values of $K$ regressors for observation $t$. Then $x_{t}x_{t}^{\prime}$ is the outer product (a $K\times K$ matrix) and $\sum\nolimits_{t=1}^{T}x_{t}x_{t}^{\prime}$ is just the sum (of each element) across the $T$ observations.

We can calculate the same thing by (a) letting $X$ be a $T\times K$ matrix with $x_{t}^{\prime}$ in row $t$ and (b) then calculating $X'X$.


In [11]:
x₁ = [1;-1]     #(column vector)
x₂ = [1;0]
x₃ = [1;1.0]

X = [x₁';x₂';x₃']

println("X")
printmat(X)

(T,K) = (size(X,1),size(X,2))

Sxx1 = x₁*x₁' + x₂*x₂' + x₃*x₃'      #just to illustrate     

Sxx2 = zeros(K,K)
for t = 1:T
    Sxx2 = Sxx2 + X[t,:]*X[t,:]'  #yes, X[t,:] becomes a column vector
end

Sxx3 = X'X

println("sum of outer products, three versions")
printmat(Sxx1)
printmat(Sxx2)
printmat(Sxx3)


X
     1.000    -1.000
     1.000     0.000
     1.000     1.000

sum of outer products, three versions
     3.000     0.000
     0.000     2.000

     3.000     0.000
     0.000     2.000

     3.000     0.000
     0.000     2.000


In [ ]: