CS代写 CSC 311) Spring 2020

Linear Algebra Review
(Adapted from ’s slides)
Introduction to Machine Learning (CSC 311) Spring 2020
University of Toronto

Copyright By PowCoder代写 加微信 powcoder

Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 1 / 28

A scalar is a number.
A vector is a 1-D array of numbers. The set of vectors of length n with real elements is denoted by Rn.
Vectos can be multiplied by a scalar.
Vector can be added together if dimensions match.
A matrix is a 2-D array of numbers. The set of m ⇥ n matrices with real elements is denoted by Rm⇥n.
Matrices can be added together or multiplied by a scalar. We can multiply Matrices to a vector if dimensions match.
In the rest we denote scalars with lowercase letters like a, vectors with bold lowercase v, and matrices with bold uppercase A.
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 2 / 28

Norms measure how “large” a vector is. They can be defined for matrices too.
The `p-norm for a vector x:
kxkp = |xi|p . i
The `2-norm is known as the Euclidean norm. P The `1-norm is known as the Manhattan norm, i.e., kxk1 =
The `1 is the max (or supremum) norm, i.e., kxk1 = maxi |xi|.
i |xi|. Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 3 / 28

Dot Product
Dot product is defined as v · u = v>u = Pi uivi.
The `2 norm can be written in terms of dot product: kuk2 = pu.u.
Dot product of two vectors can be written in terms of their `2 norms and the angle ✓ between them:
a>b = kak2 kbk2 cos(✓).
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 4 / 28

Cosine Similarity
Cosine between two vectors is a measure of their similarity: cos(✓)= a·b .
Orthogonal Vectors: Two vectors a and b are orthogonal to
each other if a · b = 0.
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 5 / 28

Vector Projection
Given two vectors a and b, let bˆ = b be the unit vector in the direction of b. kbk
Then a1 = a1 · bˆ is the orthogonal projection of a onto a straight line parallel to b, where
a1=kakcos(✓)=a·bˆ=a· b kbk
Image taken from wikipedia.
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 6 / 28

Trace is the sum of all the diagonal elements of a matrix, i.e., Tr(A) = XAi,i.
Cyclic property:
Tr(ABC) = Tr(CAB) = Tr(BCA).
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 7 / 28

Multiplication
Matrix-vector multiplication is a linear transformation. In other words,
M(v1+av2)=Mv1+aMv2 =)(Mv)i=XMi,jvj. j
Matrix-matrix multiplication is the composition of linear transformations, i.e., P
(AB)v = A(Bv) =) (AB)i,j = k Ai,kBk,j.
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 8 / 28 Image taken from wikipedia.

Invertibality
I denotes the identity matrix which is a square matrix of zeros with ones along the diagonal. It has the property IA = A
(BI = B) and Iv = v
A square matrix A is invertible if A1 exists such that A1A = AA1 = I.
Not all non-zero matrices are invertible, e.g., the following matrix
is not invertible:
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 9 / 28

Transposition
Transposition is an operation on matrices (and vectors) that interchange rows with columns. (A>)i,j = Aj,i.
(AB)> = B>A>.
A is called symmetric when A = A>.
A is called orthogonal when AA> = A>A = I or A1 = A>.
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 10 / 28

Diagonal Matrix
A diagonal matrix has all entries equal to zero except the diagonal entries which might or might not be zero, e.g. identity matrix.
A square diagonal matrix with diagonal enteries given by entries of vector v is denoted by diag(v).
Multiplying vector x by a diagonal matrix is ecient: diag(v)x = vx,
where is the entrywise product.
Inverting a square diagonal matrix is ecient
diag(v)1 = diag⇣[ 1 ,…, 1 ]>⌘. v1 vn
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 11 / 28

Determinant
Determinant of a square matrix is a mapping to scalars. det(A) or |A|
Measures how much multiplication by the matrix expands or contracts the space.
Determinant of product is the product of determinants: det(AB) = det(A)det(B)
a b = ad bc cd
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 12 / 28

List of Equivalencies
Assuming that A is a square matrix, the following statements are equivalent
Ax = b has a unique solution (for every b with correct dimension).
Ax = 0 has a unique, trivial solution: x = 0. Columns of A are linearly independent.
A is invertible, i.e. A1 exists.
det(A) 6= 0
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 13 / 28

Zero Determinant
If det(A) = 0, then:
A is linearly dependent.
Ax = b has infinitely many solutions or no solution. These cases correspond to when b is in the span of columns of A or out of it.
Ax = 0 has a non-zero solution. (since every scalar multiple of one solution is a solution and there is a non-zero solution we get infinitely many solutions.)
Intro ML (UofT) CSC311 – Tut 2 – Linear Algebra 14 / 28

程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com