程序代写代做代考 algorithm python chain LinearRegression-Slides

LinearRegression-Slides

Vectors and Matrices in Python¶

We can create a vector by:

In [1]:

import numpy as np
a = np.array([1,0,2])
b = np.array([3,2,1])

a,b

Out[1]:

(array([1, 0, 2]), array([3, 2, 1]))

A vector only has one dimension in python (its not directly treated as matrix). We can see that as the second dimension (calling shape) is left free

In [2]:

a.shape

Out[2]:

(3,)

We can add vectors and compute the inner product with the dot function

In [3]:

c = a + b
d = a.dot(c)
d

Out[3]:

10

A matrix is created by:

In [4]:

A = np.array([[2, 1, 3], [1, 1 ,2]])
B = np.array([[2, 1], [1, 2], [5 ,2]])

A.shape, B.shape

Out[4]:

((2, 3), (3, 2))

where we have created a $2×3$ and a $3×2$ matrix

Or vectors can be stacked into matrices

In [5]:

X = np.column_stack((a,b))
Y = np.row_stack((a,b))

X,Y

Out[5]:

(array([[1, 3],
[0, 2],
[2, 1]]), array([[1, 0, 2],
[3, 2, 1]]))

We can add, transpose and multiply matrices

In [6]:

C = A + B.transpose()
D = C.dot(A.transpose()) # matrix product C * A
C,D

Out[6]:

(array([[4, 2, 8],
[2, 3, 4]]), array([[34, 22],
[19, 13]]))

And we can multiply matrices with vectors

In [7]:

e = A.dot(a) # this corresponds to A * a
f = a.dot(B) # this corresponds to a^T * B
e, f

Out[7]:

(array([8, 5]), array([12, 5]))

The inverse of a matrix can be computed by

In [8]:

import numpy.linalg as linalg

AA = A.dot(A.transpose()) # A * A^T … we can only invert quadratic matrices
AAinv = linalg.inv(AA)

AA, AAinv

Out[8]:

(array([[14, 9],
[ 9, 6]]), array([[ 2. , -3. ],
[-3. , 4.66666667]]))

Multiplying with the inverse needs to result in the Identity matrix (from both sides)

In [9]:

AA.dot(AAinv), AAinv.dot(AA)

Out[9]:

(array([[ 1.00000000e+00, 0.00000000e+00],
[ 3.55271368e-15, 1.00000000e+00]]),
array([[ 1.00000000e+00, 0.00000000e+00],
[ 7.10542736e-15, 1.00000000e+00]]))

Note: Computing the inverse of a matrix is tricky and it is hard to get a numerically accurate solution. Whenever we need to compute the inverse of a matrix times another matrix ($\boldsymbol{A}^{-1}\boldsymbol{B}$, then it is better to use specifically tailored methods for this which are numerically more stable.

In [10]:

import numpy.linalg as linalg
# compute A^-1*b in a more stable way using linalg.solve.
b = np.array([1, 2])
out1 = linalg.solve(AA, b)

out1

Out[10]:

array([-4. , 6.33333333])

Excercise 1:¶
Compute:

$(\boldsymbol{A} \boldsymbol a – \boldsymbol b)^T(\boldsymbol A \boldsymbol a – \boldsymbol b)$,
$(\boldsymbol{C} \boldsymbol b)^T\boldsymbol C$
$(\boldsymbol{C}^T \boldsymbol C)^{-1}\boldsymbol C^T \boldsymbol a$,

where the matrices are defined below. Check your result also in terms of the dimensionalities of the resulting matrices. Thats an easy way of spotting an error. Always use the linalg.solve method instead of the linalg.inv method if possible.

Linear Regression in Matrix Form¶

We want to find a linear function (line) that best fits the data
Post navigation