ECON 61001: Review of some linear algebra concepts
Alastair R. Hall
The University of Manchester
Alastair R. Hall ECON61001: Linear Algebra Review 1 / 14
Outline of this revision session
Vector spaces
Linear (in)dependence
Rank of a matrix
Quadratic forms and definiteness of matrices Spectral decomposition
Resources: Orme (2009) Linear Algebra Notes and sequence of videos, both on BB in folder “Linear Algebra Resources”.
Alastair R. Hall ECON61001: Linear Algebra Review 2 / 14
Vector spaces, Orme Chap 2
The Totality of all n × 1 vectors is called the n-dimensional vector space.
sometimes called the n-dimensional Euclidean space and denoted
Rn = x : x′ = (x1,x2,…,xn);xi ∈ R, i = 1,2,…,n.
Alastair R. Hall ECON61001: Linear Algebra Review 3 / 14
Vector spaces, Orme Chap 2
Let {ei}ni=1 denote the n-dimensional unit vectors (ith element of ei is one and all others are zero).
Then we have
x = x1e1 +x2e2 …+xnen = xiei
n i=1
Alastair R. Hall ECON61001: Linear Algebra Review 4 / 14
Vector spaces, Orme Chap 2
Note:
(i) any vector x ∈ Rn can be written in this way
(ii) e1,e2,…,en are all n×1 and none of the ei can be
expressed as a linear combination of the remaining ej, j ̸= i.
(i) → {ei}ni=1 spans Rn.
(ii) → {ei}ni=1 is a linearly independent set. (i) & (ii) → {ei}ni=1 forms a basis for Rn.
Alastair R. Hall ECON61001: Linear Algebra Review 5 / 14
Vector spaces, Orme Chap 2
A vector space, V , is a non-empty set of vectors satisfying, for a, b ∈ V :
a+b∈V;
λa ∈ V for any scalar λ.
A sub-space of Rn is a non-empty subset of Rn which is also a vector space.
Alastair R. Hall ECON61001: Linear Algebra Review 6 / 14
Linearly (in)dependent sets, Orme Chap 2
Let { xj }mj=1 be a collection of n × 1 vectors.
If there exist scalars {λj }mj=1 with at least one λj ̸= 0 such that mj=1 λjxj = 0 then {xj }mj=1 is said to form a linearly dependent set.
Conversely, if mj=1 λjxj = 0 only holds for λj = 0, j = 1,2,…,m then {xj }mj=1 is said to form a linearly independent set.
Alastair R. Hall ECON61001: Linear Algebra Review 7 / 14
Rank of a matrix, Orme Chap 3
Define
X
n×m
x1,1 x1,2 … x1,m . . .. .
xn,1 xn,2 . . . xn,m
= . . . . = [ x•,1 , x•,2 , . . . , x•,m ]
If {x•,j}mj=1 form a linearly independent set then X has full column rank that is, the column rank = # of columns.
If {x•,j}mj=1 form a linearly dependent set then X does not have full column rank and:
column rank of X = maximum # of columns of X that can form a linearly independent set.
(n×1) (n×1) (n×1)
Alastair R. Hall ECON61001: Linear Algebra Review 8 / 14
Rank of a matrix, Orme Chap 3
Similarly
X= . . . . = . .
x1,1 x1,2 … x1,m
. . .. .
x2,• (1 × m)
n×m ..
xn,1 xn,2 … xn,m xn,•
If {xi,•}ni=1 form a linearly independent set then X has full row
rank that is, the row rank = # of rows.
If {xi,•}ni=1 form a linearly dependent set then X does not have full
row rank and:
row rank of X = maximum # of rows of X that can form a linearly independent set.
x1,• (1×m)
(1×m)
Alastair R. Hall ECON61001: Linear Algebra Review 9 / 14
Rank of a matrix, Orme Chap 3
Key result: row rank of X = column rank of X.
So define rank of X = row/column rank of X, and denote by
rank (X).
Important results involving rank:
for X (n × m): rank(X) ≤ min[n, m]. rank(AB) ≤ min[rank(A), rank(B)].
if m = n then X is nonsingular (det(X) ̸= 0 and X−1 exists) if and only if rank(X) = m(= n).
Alastair R. Hall ECON61001: Linear Algebra Review 10 / 14
Quadratic forms
Let A be a n×n symmetric matrix, and x be a n×1 vector. A quadratic form in A takes the form x′Ax and is a scalar.
A is positive definite (pd) iff x′Ax > 0 for all x ̸= 0.
A is positive semi-definite (psd) iff x′Ax ≥ 0 for all x ̸= 0.
The definitions of negative definiteness and negative semi-definiteness are analogous only with direction of inequalities reversed.
Note A can also be indefinite in which case quadratic forms in A can be either positive or negative depending on x.
Alastair R. Hall ECON61001: Linear Algebra Review 11 / 14
Spectral decomposition of symmetric matrix, Orme Chap 6
Let A be a (real) symmetric n × n matrix then there exists
Λ=diag(λ1,λ2,…,λn)whereλi are(real)scalars,
X = [x•,1, . . ., x•,n] an orthogonal matrix (that is, X−1 = X′), such that
A = XΛX′ ∼ spectral decomposition of A {λi,x•,i}ni=1 known as eigenvalues and eigenvectors of A.
Alastair R. Hall ECON61001: Linear Algebra Review 12 / 14
Spectral decomposition of symmetric matrix, Orme Chap 6
As a result:
det(A) = ni=1 λi.
tr(A) =d ni=1 ai,i = ni=1 λi Connection to positive definiteness of A:
A is positive (negative) definite iff λi > 0 (< 0) for all i =1,2,...,n.
A is positive (negative) semi- definite iff λi ≥ 0 (≤ 0) for all i =1,2,...,n.
Alastair R. Hall ECON61001: Linear Algebra Review 13 / 14
Time to test your understanding
I have posted some questions on Blackboard that test your understanding. Please try to do these.
The solutions are also posted for your convenience but please do contact me if you have any questions about this material.
Alastair R. Hall ECON61001: Linear Algebra Review 14 / 14