LinearRegression-Slides
Vectors and Matrices in Python¶
We can create a vector by:
In [1]:
import numpy as np
a = np.array([1,0,2])
b = np.array([3,2,1])
a,b
Out[1]:
(array([1, 0, 2]), array([3, 2, 1]))
A vector only has one dimension in python (its not directly treated as matrix). We can see that as the second dimension (calling shape) is left free
In [2]:
a.shape
Out[2]:
(3,)
We can add vectors and compute the inner product with the dot function
In [3]:
c = a + b
d = a.dot(c)
d
Out[3]:
10
A matrix is created by:
In [4]:
A = np.array([[2, 1, 3], [1, 1 ,2]])
B = np.array([[2, 1], [1, 2], [5 ,2]])
A.shape, B.shape
Out[4]:
((2, 3), (3, 2))
where we have created a $2×3$ and a $3×2$ matrix
Or vectors can be stacked into matrices
In [5]:
X = np.column_stack((a,b))
Y = np.row_stack((a,b))
X,Y
Out[5]:
(array([[1, 3],
[0, 2],
[2, 1]]), array([[1, 0, 2],
[3, 2, 1]]))
We can add, transpose and multiply matrices
In [6]:
C = A + B.transpose()
D = C.dot(A.transpose()) # matrix product C * A
C,D
Out[6]:
(array([[4, 2, 8],
[2, 3, 4]]), array([[34, 22],
[19, 13]]))
And we can multiply matrices with vectors
In [7]:
e = A.dot(a) # this corresponds to A * a
f = a.dot(B) # this corresponds to a^T * B
e, f
Out[7]:
(array([8, 5]), array([12, 5]))
The inverse of a matrix can be computed by
In [8]:
import numpy.linalg as linalg
AA = A.dot(A.transpose()) # A * A^T … we can only invert quadratic matrices
AAinv = linalg.inv(AA)
AA, AAinv
Out[8]:
(array([[14, 9],
[ 9, 6]]), array([[ 2. , -3. ],
[-3. , 4.66666667]]))
Multiplying with the inverse needs to result in the Identity matrix (from both sides)
In [9]:
AA.dot(AAinv), AAinv.dot(AA)
Out[9]:
(array([[ 1.00000000e+00, 0.00000000e+00],
[ 3.55271368e-15, 1.00000000e+00]]),
array([[ 1.00000000e+00, 0.00000000e+00],
[ 7.10542736e-15, 1.00000000e+00]]))
Note: Computing the inverse of a matrix is tricky and it is hard to get a numerically accurate solution. Whenever we need to compute the inverse of a matrix times another matrix ($\boldsymbol{A}^{-1}\boldsymbol{B}$, then it is better to use specifically tailored methods for this which are numerically more stable.
In [10]:
import numpy.linalg as linalg
# compute A^-1*b in a more stable way using linalg.solve.
b = np.array([1, 2])
out1 = linalg.solve(AA, b)
out1
Out[10]:
array([-4. , 6.33333333])
Excercise 1:¶
Compute:
$(\boldsymbol{A} \boldsymbol a – \boldsymbol b)^T(\boldsymbol A \boldsymbol a – \boldsymbol b)$,
$(\boldsymbol{C} \boldsymbol b)^T\boldsymbol C$
$(\boldsymbol{C}^T \boldsymbol C)^{-1}\boldsymbol C^T \boldsymbol a$,
where the matrices are defined below. Check your result also in terms of the dimensionalities of the resulting matrices. Thats an easy way of spotting an error. Always use the linalg.solve method instead of the linalg.inv method if possible.
Linear Regression in Matrix Form¶
We want to find a linear function (line) that best fits the data
Post navigation