STAT 513413: Lecture 4 A bit of linear algebra
Vectors and matrices
A tale of expert code II: You never invert a matrix
The theory for a linear model y X says that you obtain the least squares estimates via the formula b XTX1XTy
However, never ever every rule has an exception, but do
b solvetX X tX y
and writing
b solvecrossprodX crossprodX, y
doesnt really save it. Getting b via solving the system XTXb XTy
b solvecrossprodX, crossprodX, y
may be a bit better; but experts know that you should be doing in this case a socalled QR decomposition MATLAB backslash operator, which in R amounts to
b qr.solveX, y
This is correct but you dont need to know it unless in certain special situations; instead, you may just do
b coeflmy X1
1
To show the difference is, however, not that easy…
…because the numerics of R is very good
x runif10,0,10
X cbindrep1,lengthx,x
y 2 3x rnormlengthx
cbindX, y
xy 1, 1 5.088385 18.846518 2, 1 1.875540 8.434781 3, 1 4.509448 16.397015 4, 1 7.366187 24.432547 5, 1 4.914751 18.399520 6, 1 9.296908 29.273038 7, 1 8.083712 26.970036 8, 1 5.210684 16.587565 9, 1 5.028429 15.727741 10, 1 8.422086 28.772038
2
solvetX X tX y
,1
2.715395
x 2.954821
solvecrossprodX crossprodX, y
,1
2.715395
x 2.954821
solvecrossprodX, crossprodX, y
,1
2.715395
x 2.954821
qr.solveX, y
x
2.715395 2.954821
coeflmyX1
X Xx
2.715395 2.954821
See?
3
One has to go to extremes
x rep1,1000rnorm1000,0,.000001
X cbindrep1,lengthx,x
y 2 3x rnormlengthx
detcrossprodX
1 1.133003e06
solvetX X tX y
,1
20737.20
x 20732.22
solvecrossprodX, crossprodX, y
,1
20737.19
x 20732.21
qr.solveX, y
x
20733.83 20728.85
coeflmy X1
X Xx
20733.83 20728.85
4
crossprodA,B
crossprodA
rep
solveA, y
solveA
c
matrix
rbindA,B
cbindA,B
length
dim
is for componentwise multiplication components better match
vectormatrix multiplication ATB uses dedicated algorithm
in particular ATA
a repetition function, very flexible finds b such that Ab y
finds A1 if needed be concatenation of vectors, flexible too
setting up matrices
matrices are merged by rows must match matrices are merged by columns must match returns the length of a vector
returns the dimension of a matrix
Vector and matrix algebra
5
Type conversions
General format as.type qr.solveX, y
x
20733.83 20728.85
as.vectorqr.solveX, y
1 20733.83 20728.85
as.vectorcoeflmyX1
1 20733.83 20728.85
as.vectorsolvecrossprodX, crossprodX, y
1 20737.19 20732.21
as.vectorsolvetX X tX y
1 20737.20 20732.22
6
numeric4
1 0 0 0 0
rep0,4
1 0 0 0 0
repc0,1,4
1 0 1 0 1 0 1 0 1
repc0,1,c3,2
1 0 0 0 1 1
Xmatrix0,nrow2,ncol2
Xmatrix1:4,nrow2,ncol2 X
,1 ,2
1, 1 3
2, 2 4
as.vectorX
1 1 2 3 4
as.matrix1:4
,1
1, 1
2, 2
3, 3
4, 4
Potpourri
7