程序代写代做代考 ER Bayesian flex 统计学习导论+基于R应用.pdf

统计学习导论+基于R应用.pdf

An Introduction to Statistical Learning
with Applications in R

Gareth J ames )
Witten)
Hastie)

Robert Tibshirani)

2015.6

An 1ntroduction to Statistical Learning: with Applications in R

1SBN 978-7-111-49771-4

1V. C8

01-2013-7855

Translation from English language edition: An lntroduction to Statistical Learning by Gareth
]ames , Daniela Witten , Trevor Hastie and Robert Tibshirani

Copyright ( 2013 Springer_ Verlag New York , Inc
Springer is a part of Springer Science+ Business Media
All rights Reserved

Science+ Business Media

)

x 260mm 1/16
978-7-111-49771-4

(010) 88378991 88361066 (010) 88379604
(010) 68326294 88379649 68995259

When we wrote An Introduction to we had a single goal: to make key

concepts in statistical machine learning accessible to a very broad audience. We are thrilled that

Professor Xing Wang has taken the time to translate our book into Chinese , so that these concepts

will be made accessible to an even broader audience. We hope that the readers of this Chinese

translation will find our book to be a useful and informative introduction to a very exciting and im-

portant research area.

Sincerely!

Gareth James , Daniela Witten , Trevor Hastie and Robert Tibshirani

(The Elements of Statistical Learning ,

V

(The Elements of Statistical Learn-

ing , ESL) Tibshirani ,

(An to Statistical Learning ,

Pallavi

Basu , Alexandra Chouldechova , Patrick Danaher , Will Fithian , Luella Fu , Sam Gross , Max
Grazier G’Sell , Courtney Qiao , Elisa

Xin Lu Tano

Berra

Gareth James

Daniela Witten

Trevor Hastie

Robert Tibshirani

1. 1

1. 2 …………………… 4

1. 3 ……………………… 4

1. 4 …………… 6

1. 5 ………… 6

1. 6 ………………… 8

1. 7 ……… 9

1. 8

1. 9

2. 1

2.2

2. 3

2.4 37

3. 1

3.4

3.5

3.6

3. 7

4. 1

4.2

4.3

4.4

4.5

4.6

4. 7

5. 1

5.2

5.3

5.4

3.3 6. 1

6.2

6.3

6.4

6.5

6.6 ……… 173

6. 7

6. 8

7. 1

7.2

7.3

7.4

7.5

7.6

7. 7

7.8

7.9

8. 1

8.2

8.3

8.4

9. 1

9.2

9.3

9.4 ………………… 246

9.5

9.6

9. 7

10.1

10.2

10. 3

10.4

10. 5

10.6

10.7

1. 1

1. 1. 1

10

iA;,,
+

Education Level

CCN

2003 2006 2009
Year

• 2

2003

(

Yesterday Two Days Previous Three Days Previous

1. 1.2

s –
TT

i

T

u 0
Q)
eo

Q)

Down Up
Today’s Direction

Down Up
Today’s Direction

Down Up
Today’s Direction

3 •

component)

Up

Today’s Direction

1. 1

Down

N
pE –
o 0
H

…..
u

Q)

1. 1.3

A A

-r4

. . . . . . . . .
-40 -20 0 20

ZI

CN

C

NN R
o
T
o
?

. . .. .
. . . .
2 .

• .
-40 -20 0 20 40

ZI

.. _. ..,.. •• A ..
o j -_-.- “_fÞ .•

1. _ • •
“”,- .. NO I ..

o

60 40 60

4

1.2

statistical

model)

Fried-

Tibshirani

1.3

of Statistical

1. 3 • 5

(An Statistical

6

1.4

1.5

Wage

=3

age =
iable Narne

1. 5 • 7

i = 1 , 2 ,… , n; j = L
n) , p) 0

&P
ZZZ 12···n ZZZ Il-

zzz

X

I il \

I X ,,., I I i2 I
X.

, • ,
, • ,
,

11
.

11 /E\

•••

mu

zzz /’!lllIll—llt\

J
X

= 3

x = (x1 x 2 •• • xp )

transpose = (X i1 X i2 … X ip )
nnn
zz

z

ZZZ

T X

8

y

Yl) , Y2) , …, f

-iqF-hphn
/’!lllIll—llt\

a

0 A E X s 0

E ‘Xd , B E dxs

A=(; :),
1 1 2 \ 15 6 \ 11 X 5 + 2 X 7 1 X 6 + 2 X 8 \ 1 19 22 \

AB=I 11 1=1 1=1 1
501

1.6

1. 7 • 9

1.7

-1

Auto

Boston

Caravan

Carseats

College

Defalut

Khan

NCI60

OJ

Portfolio

Smarket

USArrests

Wage

Weekly

10

1.8

http://www – bcf. usc. edu/ gareth/ISL/

1.9

2.1

TV

N

o

m-c-

o 10 20 30 40 50
Radio

300

variable) , sales output

12

predictor

variable) re-

dependent variable)

(X) ,
Y = j( X) + B (2. 1)

term)

systematic

of

of

. o . .. . . .. o . . . . . o ugszH . • .
ogooa

. . o
o

•• o

lqL 10

0

8U la

c u

-MM

0

14s
lM

e

l2Y
·
-11 lo

10 12 14 16 18 20 22

Years of Educatio

of of educa-

of

ty

2.1 • 13

of

of

2. 1. 1

inference

Y = j(X) (2.2)
box)

14

E ( Y – y) 2 = E [j( X) + e – j( X) ] 2
= [j(X) – j(X) r + Var( B) (2.3)

2.1 • 15

2.1.2

training data)

= 1 , 2 ,… , = 1 ,
(xj’Yj) , (X U Y2)’ … , (Xn ,yJ = (x ij ‘X i2 ‘ … ,Xip)T O

( 1

j(X) (2.4)
model)

+

ordinary least squares)

parametric)

16

felexible

incorne X X seniori ty

of

2.1 • 17

18

2.1.3

additive model ,
GAM)

(

Subset Selection
ii Lasso

Least Squares

Generalized Additive Models
Trees

Bagging , Boosting
hFOJ Support Vector Machines

Low High

Flexibility

2.1 • 19

2.1.4

i = 1,… ,

regression)

unsupervised)

( cluster

20

23 4A
+ + +

f:,

!? 0_
u

Cè:>(j)

+

o 2 4 6 8 10 12
X 1

A

N –I

o 2 4 6
X 1

supervised

2.1.5

the value of a

the price of a

B,

2.2 • 21

2.2

2.2.1

squared error , MSE)

MSE – J(x;) ) 2 (2.5)

(

( history of

Yl) , Y2) , …,

, f( x 2 ) , … ,

22

xo) = Yo 0

Ave (f (xo) – YO)2 (2.6)
MSE)

o 0
ho u fcq p

E m qp

N
CCP >

o 20 40 60 80 100 2 5 10 20
X Flexibi1ity

2.2 • 23

of

(

2.2.2

E (Yo – J(XO))2 = Var(f(xo)) + [Bias (f(xo)) r + Var( B) (2.7)

m.N

o
N

o

N –
o –

• 24

N

20 10

Flexbility

5 2
nu nu

80 60

X

40 20

mHCH
o
N

o

o

o
o

20 10

Flexibility

5 2
hu

80 60 40

X

20

(xO)) test MSE)

test

(xO))

2.2 • 25

Var( B)

m.N

o

qdD p

2 5 10 20
Flexibility

F‘,J

cpp q

o

cqp p

2 5 10 20
Flexibility

o

2 5 10 20
Flexibility

26

trade-off)

2.2.3

Yl) , …,

error rate) ,

(2. 8)

indicator variable)

training

test

(2.9)

Pr (Y = j I x = xo) (2. 10)

2.2 • 27

Y = 1 I X = xo) > O.

Y = orange I

decision

X j

– maxjPr( Y = j I X =

1 – E ( max Pr (Y = j I X) ) (2. 11)

( Y = jX = xo) < 1 28 H 0, 1304 0 = KNN: K=10 X 1 29 • 2.2 KNN: K=100 KNN: K=1 KNN K 0 ] 0 l -; -............ \ I - Training Errors I Test Errors 0.01 0.02 0.05 1.00 0.50 0.20 0.10 lIK 30 2.3 http:// cran. r- proj ect. orgl 2.3.1 (inputl , input2) () ( 3 , c(1 , 3 , 2 , 5) [1] 1 3 2 5 c (1, 6 ,2) > x
[1] 1 6 2
> y = c (1, 4 ,3)

xvd fkrtnu hh1 +U+U nqunquvd ee+

18

> 15 ()
[1] “x” “y”
> rm(x , y)

character (0)

2.3 • 31

> rm (l ist=ls ())

rnatrix

>

> x=matrix(data=c(1 , 2 , 3 ,4) , nrow=2 , ncol=2)
> x

[ , 1] [, 2]
[1 ,] 1 3
[2 ,] 2 4

=

> x=matrix(c(1 , 2 , 3 ,4) , 2 , 2)

=

E U R Ti –w o r vu b , nJL , , qu , 4i
,
([ c -1

,
+b m”

> sqrt(x)
[, 1] [, 2]

[1 ,] 1. 00 1. 73
[2 ,] 1. 41 2.00
> x^2

[, 1] [, 2]
[1 ,] 1 9
[2 ,] 4 16

rnorrn

()

> x=rnorm(50)
>
> cor(x , y)
[1] 0.995

32

rnorrn

seed seed

> set.seed(1303)
> rnorm (50)
[1] -1.1440 1.3421 2.1854 0 . 5364 0.0632 0.5022 -0.0004

seed

> set.seed(3)
> y=rnorm (100)
> mean(y)
[1] 0.0110
> var(y)
[1] 0.7329
> sqrt(var(y))
[1] 0.8561
> sd(y)
[1] 0.8561

2.3.2

plot (x ,
plot

> x=rnorm(100)
> y=rnorm(100)
> plot(x , y)
> plot(x , y , xlab=”this is the x-axis” , ylab=”this is the y-axis” ,

main=”Plot of X vs Y”)

n e
“r dH PA—14 eo rc gvd(C 1

,
fi

(t·d dlel ppdl

u

off

seq (a ,
seq(O , 1 , =
(3:

2.3 • 33

> x=seq (1, 10)
> x

[1] 1 2 3 4 5 6 7 8 9 10
> x=1:10
> x

[1] 1 2 3 4 5 6 7 8 9 10
> x=seq(-pi , pi , length=50)

1. ,
2. ,

0

> y=x
> f=outer(x , y , function(x , y)cos(y)/(1+x-2))
> contour(x , y , f)
>
> fa=(f-t(f))/2
> contour(x , y , fa , nlevels=15)

irnage

persp

> image(x , y , fa)
> persp(x , y , fa)
>
> persp(x , y , fa , theta=30 , phi=20)
> persp(x , y , fa , theta=30 , phi=70)
> persp(x , y , fa , theta=30 , phi=40)

2.3.3

> A=matrix(1:16 , 4 ,4)
> A

[, 1] [, 2] [, 3] [, 4]
[1 , ] 1 5
[2 , ] 2 6
[3 , ] 3 7
[4 ,] 4 8

> A [2 , 3]
[1] 10

9 13
10 14
11 15
12 16

34

> A[c(1 , 3) , c(2 ,4)]
[, 1] [, 2]

[1 , ] 5 13
[2 , ] 7 15
> A[1:3 , 2:4]

[, 1] [, 2] [, 3]
[1 , ] 5 9 13
[2 , ] 6 10 14
[3 , ] 7 11 15
> A[1:2 ,]

[, 1] [, 2] [, 3] [, 4]
[1 , ] 1 5 9 13
[2 , ] 2 6 10 14
> A [ , 1: 2]

[, 1] [, 2]
[1 , ] 1 5
[2 , ] 2 6
[3 , ] 3 7
[4 , ] 4 8

> A [1 ,]
[1] 1 5 9 13

> A[-c(1 , 3) ,]
[, 1] [, 2] [, 3] [, 4]

[1 ,] 2 6 10 14
[2 ,] 4 8 12 16
> A[-c(1 , 3) , -c (1, 3 ,4)]
[1] 6 8

dirn

> dim(A)
[1] 4 4

2.3.4

read. table
te. table

Mac , U
table (

data frame) 0

fix

> Auto=read.table(“Auto.data”)
> fix(Auto)

2.3 • 35

table ( = T

header =

> strings=”?”)
> fix(Auto)

cav

> Auto=read.csv(“Auto.csv” , header=T , na.strings=”?”)
> fix(Auto)
> dim(Auto)
[1] 397 9
> Auto[1:4 ,]

dirn

orni t

> Auto=na.omit(Auto)
> dim(Auto)
[1] 392 9

> names(Auto)
[1] “mpg” “cylinders” “displacement” “horsepower”
[5] “weight”
[9] “name”

“acceleration” “year”

2.3.5

“origin”

> plot(cylinders , mpg)
Error in plot (cylinders , mpg) object’ not found

attach

> plot(Auto$cylinders , Auto$mpg)
> attach (Auto)
> plot(cylinders , mpg)

36

as. factor

> cylinders=as.factor(cylinders)

> plot(cylinders , mpg)
> plot(cylinders , mpg , col=”red”)
> plot(cylinders , mpg , col=”red” , varwidth=T)
> plot(cylinders , mpg , col=”red” , varwidth=T , horizontal=T)
> plot(cylinders , mpg , col=”red” , varwidth=T , xlab=”cylinders” ,

ylab=”MPG”)

hist

EU 4EA = s k a e r b
), =–1414 0O CC

PAP


p

mmm +U+U+U SSS ·l·-

-1

>>>

pairs

> pairs(Auto)
> mpg + displacement + horsepower + weight +

Auto)

()

> plot(horsepower , mpg)
> identify(horsepower , mpg , name)

summary

> summary(Auto)
mpg

Min. : 9.00
1 st Qu.: 17 . 00
Median :22.75
Mean : 23.45
3rd Qu.: 29 . 00
Max. : 46.60

horsepower
Min. : 46.0
1st Qu.: 75.0
Median : 93.5
Mean : 104.5
3rd Qu. :126.0
Max. : 230.0

t0004noo n-

…..

e851455 m605975 e11124-c··

………

.

a·· lunu PQaQ 5·ln· ·1ntdadz dlseera
M1MM3Mu nuhunun4nunu nununu7fnunu

s00040o r
…..

.

e344588 AU–

••••••••••

n -unu lQaQ vd··ln-cntdadx
-lseera M1MM3MU

weight acceleration
Min. :1613 Min. : 8.00
1st Qu.:2225 1st Qu . :13.78
Median :2804 Median :15.50
Mean : 2978 Mean : 15.54
3rd Qu.:3615 3rd Qu.:17.02
Max. :5140 Max. :24.80

37

year origin name
Min. :70.00 Min. :1 . 000 amc matador : 5
1st Qu.:73.00 1st Qu.: 1. 000 ford pinto : 5
Median :76.00 Median :1.000 toyota corolla : 5
Mean : 75.98 Mean : 1 . 577 amc gremlin 4
3rd Qu.: 79 . 00 3rd Qu . : 2 . 000 amc hornet 4
Max. : 82.00 Max. : 3.000 chevrolet chevette: 4

(Other) :365

> summary(mpg)
Min . 1 st Qu . Median Mean 3rd Qu . Max .
9 . 00 17 . 00 22 . 75 23 . 45 29 . 00 46 . 60

2.4

(

(

(

(

(

38

(

(

Obs. X j X2 X3
3

2 2

3 3

4 2

5 1

6 1 1

=X2 =X3
(

(

Y

Red
Red
Red
Green
Green
Red

of applications received)

of applicants accepted)

o of new students enrolled)
students from top 10% of high

school class)

students from top 25% of high
school class)

N umber of full- time undergraduates)

39

N umber of part – time undergraduates)
o tuition)

and board costs)

book costs)

personal spending)

of faculty with Ph. )
of )

ratio)

of alumni who )

Graduation rate)

( csv

> rownames (college) =college [, 1]
>

AE–, e e le ogb ce =14 e-i guo ec -irk –z cf >>

( c) (

A [, 1: 10

> Elite=rep(“No” , nrow(college))
> te [college$Top10perc >50] =” Yes”
> Elite=as.factor(Elite)
> college =data. frame (college , Eli te)

(

(rnfrow=c (2 ,

40

(

(

(

(

(

> library(MASS)

> Boston

> ?Boston

(

(

(

regression)

sales radio

(1

(3

42

3.1

linier

(3. 1)
Y on

X)

Sales X TV

Y

3. 1. 1

= 1,… , n ,

ei =Yi

residual

43 • 3.1

of

RSS = + + … +e:

RSS = +… + (3.3)

L

L (X i – x)
(3.4)

n i=i
squares coefficient

= 0.047 50

C’I

o
C’I

300 250 200 150

TV

nu
50

3.1.2

44

2 =

2

D

2 1

5 6 7

(3.5)

regression line)

least

squares line) (3.

y = 2 + 3X + B (3. 6)
=

2

45 •

/

3.1

m

o
– ‘

h
o

o

o 1 2 -2 -1 0
x x

=2

2 -2

systematically

= (3.7)

(3.8) =
L 2 l’U L

@

46

=

error) =
IRSS/(n

interval) 0 95

. ) (3.9)

[131 – ] (3. 10)

. (3. 11)

130 ,
[ O. 042 , O. 053
6

hypothesis) :

Ho:

alternative

=

(3. 12)

(3. 13)

3.1 • 47

– 0
(3.14 )

)

t 0

reject the null hypothesis)

TV

7.0325

0.047 5 0.0027

15.36

17.67

Intercept <0.000 1 <0.000 1 3.1.3 13) the extent to which the model fits the standard error , @ 48 = = - y;) 2 ''1 n - L ''1 n - L i=i (3. 15) 1. RSS = L (Yi - y;)2 (3.16) 3 260/14000 = of = 1 ,… , Yi Yi proportion) R2 = TSS - RSS RSS TSS TSS (3. 17) TSS = L (Yi _ Y ) sum of squares) TSS - (proportion of variability in Y that can be explained using X) 0 RSE 3.2 • 49 E: (x i - x) Cor(X , Y) = (3.18) JI J I 2 R2 = r2 3.2 Intercept 9.312 0.563 16.54 <0.000 1 0.203 0.020 9.92 <0.000 1 Intercept 12.351 0.621 19.88 <0.000 1 newspaper 0.055 0.017 3.30 <0.000 1 50 (3.5) (3.19) all other predictors X TV X radio + B (3.20) 3. 2. 1 Y (3.21) RSS = L (Yi _ y;)2 = L 2 (3.22) Y 3.2 • 51 Intercept 2.939 0.311 9 9.42 <0.000 1 TV 0.046 0.0014 32.81 <0.000 1 Radio 0.189 0.0086 21. 89 <0.000 1 newspaper -0.001 0.0059 -0.18 0.8599 newspaper TV , TV newspaper sales TV 0.0548 0.0567 0.7822 0.3541 0.5762 newspaper 1 0.2283 sales 1 52 3.2.2 ( 1 = 0 (TSS - RSS)/ F=P(3.23) RSS/(n - ) - y) El RSS/(n - f El (TSS - RSS)/pf E 1 (TSS - RSS) /p f @ 3.2 • 53 R2 1. 69 0.897 570 23 (RSSo - RSS)/q F (3.24) SS/( n - ) least one tors is = ) >

high –

@

54

which)

information criterion , Bayesian information cri-
terion ,

= 1 073 741

3.2 • 55

897

681
686

RSE = A RSS (3.25)
yn-p-l

interaction)

21

(1

56

Radio

squares plane)

true population regression plane)

f(X)

confidence

model

error) 0 in-

985 , 11 528

930 ,

@

3.3 •

3.3

3.3.1

quantitative )

qualitative) 0

t

age cards education incorne

lirnit

ethnici ty

g ; ; l | i i i j i l l I … I J I J l i
[ i I i ‘. :i: !

iU! j l | : ; : j U j i j i |
age , cards ,

come ,

57

58

factor)

r1
X. < LO (3.26) 8 + ++ 8 + Z + (3. 27) Intercept gender 509.80 19.73 33.13 46.05 15.389 0.429 <0.0001 0.6690 r1 X. < L - 1 + Bi Yi + Bi = { + Bi ethnicity 3.3 • 59 FEI-EL Z (3.28) r1 LO (3.29) 88 ++ 12z +++ .... 8 + + Z + (3.30) base- = 0 Intercept ethnicity 53 1. 00 -18.69 -12.50 46.32 65.02 56.68 11. 464 -0.287 -0.221 <0.000 1 0.7740 0.8260 contrast) 3.3.2 60 X J 0 + B interaction term) X2 + B (3. 31 ) 31 + B + B (3. 32) ers ts = 1. 2 + 3.4 X lines + O. 22 X workers + 1. 4 X (lines X workers) = 1. 2 + (3.4 + 1. 4 X workers) X lines + O. 22 X workers + 1. 4 X sales X TV X radio X (radio X TV) + B X radio) X TV X radio + B (3.33) 3.3 • 61 X -89.7)/(100 -89.7) radio) X 1 000 = 19 + 1. 1 X TV) X 1 000 = 29 + 1. 1 X Intercept TV radio TV X 6.7502 0.0191 0.0289 0.001 1 0.002 0.009 0.000 27.23 12.70 3.24 20.73 <0.000 1 <0.000 1 0.0014 <0.000 1 X incorne i + LO = þ) + { (3.34) 62 2 f .‘ D T P sE E E E 2 E FCCE E 4 2 J 50 100 150 50 100 150 Income Income + X X incorne i (3.35) e m o c × + , ... .. + e m o c × + ~~ e c a -i a b polynomial power rnpg X horsepower X horsepower 2 + B (3.36) = horsepower , X 2 = horsepower 2 3.3 • 63 o o 100 150 Horsepower horsepower ) 50 200 Intercept 56.900 1 -0.4662 0.0012 1. 800 4 0.031 1 0.0001 31. 6 -15.0 10.1 <0.000 1 <0.000 1 <0.000 1 horsepower horsepower 2 regression) 3.3.3 of 0 (3 non- constant variance of eITor outlier) 0 (5) residual = Yi - fitted)) • 64 Residua1 Plot for Quadratic Fit mlo--m-- Residua1 Plot for Linear Fit CNmHO- cmlo--m-l 155 0 35 20 25 30 Fitted Va1ues 15 30 10 15 20 25 Fitted Values 65 • 3.3 0.0 mN-OHlmul 80 60 0.5 40 20 100 80 jfiJ:ljT I ....0 J 0 , 0 0 - 0 V nO ....000 f"'l_ I .... 0 , , 60 p= 0.9 40 20 m.-m.OMU.Olm.Hl 100 80 60 Observation 40 20 66 VAR(e;) shape) heteroscedasticity Response Y Response log 3 :=: cF= a iEi T o :=: 6710 OCC E 10 IS 20 2S 30 2.4 2.6 2.8 3.0 3.2 3.4 Fitted Values Fitted Values = 67 • 3.3 o 0 0 '6 0 V o -2 0 2 4 6 Fitted Values 20 0 '" :-s! 0 0 0 08 0 00 I Io 0 0 0 -2 0 2 4 6 Fitted Values 200 þ.., N o -2 -1 2 o 1 X 0.000.050.100.150.200.25 Leverage 410 020 / …, ?48J 0 41 0 v O -2 -1 0 1 2 3 X o o 2 O X] -1 4 • 68 h. = _!_ + 2 = n L 2 (3.37 ) lirni / D o o 0 o 0 1"'\ 0 Q 0 0 e o 0 o 0 2000 4000 6000 8000 12000 2000 4000 6000 8000 Limit Limit 12000 3.3 • 69 N <'"l o 0.16 0.17 0.18 0.19 -0.1 0.0 0.1 0.2 Credi lirni lirnit 70 Credi Intercept 43.828 -3.957 <0.0001 Model1 age -2.292 0.672 -3.407 0.0007 0.173 0.005 34.496 <0.0001 Intercept -377. 537 45.254 -8.343 <0.0001 Mode12 2.202 0.952 2.312 0.0213 0.025 0.064 0.384 0.7012 multicollinearity) inflation factor , VIF) 0 VIF VIF(ß) = 01 , t worthiness 0 3.4 ( 1 ) = • 71 3. 1. (3 1. 0.049) , 172 , 0.206) , - 0.013 , 0.011) 0 1.145 , 21 Y=j(X) (7 72 3.5 parametric K- nearest neighbors regression) ( 0 K 0 f(xo) bias- variance trade- off) 3.5 • 73 H >

-1.0 -0.5 0.0 0.5 1.0
X

þ…, N

-1.0 -0.5 0.0 0.5 1.0
X

?

K=

0

=

=
p

74

(“‘l

-1.0 -0.5 0.0 0.5 1.0
X

_0/
E

CZl

ã 8l
(1) _:

::E ‘-‘

0.2 0.5 1.0
lI K

,

rV’3 2| / crn p
Fd

3S uE m 3

CvP CCC2 E 2

-1.0 -0.5 0.0 0.5 1.0 0.2 0.5 1.0
X l/K

HV3 o

GeeD

2
3

; | /
CFD d

E

cCO = E
-1.0 -0.5 0.0 0.5 1.0 0.2 0.5 1.0

X l/K

= 1 =9

3.6 • 75

p=l
G,·E ·4 q,·= ·4

2

=

2
p

p=2 p=3 p=4

16 /|iv
o

————–1 0 0 …..…
o
o

o
o

0.2 0.5 1.0 0.2 0.5 1.0 0.2 0.5 1.0 0.2 0.5 1.0 0.2 0.5 1.0 0.2 0.5 1.0
1/K

neigh-

curse of

3.6

library )

cunn quTL AAnb MUT4 (( V
M
Vu
rr aa rr bb -1·-1414

stall

instal l. packages

76

3.6.2

age

lstat

> fix(Boston)
> names(Boston)

[1] “crim” “indus” “chas” “nox” “rm” “age”
[8] “dis” “rad” “tax” “ptratio” “black” “lstat” “medv”

Boston o

(y x , data)

>
Error in eval(expr , envir , enclos) : Object “medv” not found

>
> attach
>

> lm.fit

Call:
lm(formula = medv lstat)

t5 aQd +U-Shu —
s nt5 ep& ie4 cc3 ·1r 4iaLV ft en oI

> summary(lm.fit)

Call:
lm(formula = medv lstat)

Residuals:
Min 1Q Median 3Q Max

-15.17 -3.99 -1. 32 2 . 03 24 . 50

Estimate Std . Error t value PrC>ltl)
(Intercept) 34.5538 0.5626 61.4 <2e-16 *** lstat -0 . 9500 0.0387 -24 . 5 <2e-16 *** Signif. codes: 0 *** 0.001 ** 0 . 01 * 0.05 . 0 . 1 3.6 • 77 Residual standard error: 6.22 on 504 degrees of freedom Multiple R-squared: 0.544 , Adjusted R-squared: 0.543 F-statistic: 602 on 1 and 504 DF , p-value: <2e-16 1m. > names(lm.fit)
[1] “coeff icients” “residuals ” “effects”
[4] “rank”
[7] “qr”

[10] “call”
> coef (lm.fit)

“s n14H FDel –ve Bed s-4o axm s eH ul la au vd
-l

ds” ees trm t·r Ife fdt

(Intercept) lstat
34.55 -0.95

> confint(lm.fit)
2.5 % 97.5 %

(Intercept) 33.45 35.659
lstat -1.03 -0.874

predict

> predict (lm. fit , data. frame (l stat=(c (5 , 10 , 15))) ,

fit lwr upr
1 29.80 29.01 30.60
2 25.05 24.47 25.63
3 20.30 19.73 20.87

> predict(lm.fit , data.frame(lstat=(c(5 , 10 , 15))) ,
)

fit lwr upr
1 29.80 17.566 42.04
2 25.05 12.828 37.28
3 20.30 8.078 32.53

47 , 25. 63)

>v et m-1 +U-am +U14 14e t·-014 pa

ab1ine (a , b)

> abline(lm.fit , lwd=3)
> abline(lm.fit , lwd=3 , col=”red”)
> plot(lstat , medv , col=”red”)

78

))
nJhH·· ==44 PAPAC ,,

PA

VV

,

ddo eenJh mm·-,,
4i

+U+u

,

aao ss–+U+U+U 000 141414 >>>

(

p10t

par (mfrow = c (2 ,
, nJ& ct =

-l

w44 0· rm ro a14 p

PA

> plot(predict(lm.fit) , residuals (lm.fit))
> plot(predict(lm.fit) , rstudent(lm.fit))

+b -1 44

)-1Jm +b14 84S
e

mu -414 sv et ua vz ta am h-tc lh P&WKU

7t

>>

3

which. max

3.6.3

(y xl + x2 +

(

>
> summary(lm.fit)

Call:
lm(formula = medv lstat + age , data =

Residuals
Max

-15.98 -3.98 -1. 28 1. 97 23.16

Coefficients:
Estimate Std. Error t value Pr(>ltl)

(Intercept) 33.2228 0.7308 45.46 <2e-16 *** lstat -1.0321 0.0482 -21.42 <2e-16 *** age 0.0345 0.0122 2.83 0.0049 ** Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1 Residual standard error: 6 . 17 on 503 degrees of freedom Multiple R-squared: 0 . 551 , Adjusted R-squared: 0.549 F-statistic: 309 on 2 and 503 DF , p-value: <2e-16 3.6 • 79 > , data=Boston)
> summary(lm . fit)

Ca11:
1m(formu1a = medv . , data = Boston)

Residua1s:
Min 1Q Median 3Q Max

-15.594 -2.730 -0.518 1 . 777 26.199

Coefficients:
Estimate Std . Error t va1ue Pr(>ltl)

(Intercept) 3.646e+01 5 . 103e+00 7.144 3 . 28e-12 ***
cr l. m -1. 080e-01 3.286e-02 -3.287 0.001087 **
zn 4.642e-02 1. 373e-02 3.382 0.000778 ***
indus 2.056e-02 6.150e-02 0.334 0.738288
chas 2.687e+00 8.616e-01 3.118 0.001925 * *
nox -1.777e+01 3.820e+00 -4.651 4.25e-06 ***
rm 3.810e+00 9.116 < 2e -16 *** age 6.922e-04 1. 321e-02 0.052 0.958229 dis -1.476e+00 1.995e-01 -7.398 6.01e-13 *** rad 3.060e-01 6.635e-02 4.613 5.07e-06 *** tax 3.761e-03 -3.280 0.001112 ** ptratio -9.527e-01 1.308e-01 -7.283 1 . 31e-12 *** b1ack 9.312e-03 2.686e-03 3.467 0.000573 *** lstat -10 . 347 < 2e -16 *** Signif. codes : 0 C ***' 0.1 C , 1 Residua1 standard error: 4.745 on 492 degrees of freedom Mu1tip1e R-Squared: 0.7406 , Adjusted R-squared : 0.7338 F-statistic: 108.1 on 13 and 492 DF , p-va1ue: < 2.2e-16 summary (1m. f i t ) $ r. summary(lm fit) vif > 1ibrary(car)
> vif (lm.fit)

cr l. m indus chas rm age
1. 79 2.30 3.99 1. 07 4 . 39 1. 93 3 . 10
dis rad tax ptratio b1ack lstat

3.96 7.48 9.01 1. 80 1. 35 2.94

>
> summary(lm.fit1)

80

> lm.fit1=update(lm.fit ,

3.6.4

x

+ age + lstat:

>

Call:
lm(formula = medv lstat * age , data = Boston)

Residuals:
Min 1Q Max

-15.81 -4.04 -1. 33 2.08 27.55

Coefficients:
Estimate Std. Error t value Pr(>ltl)

(Intercept) 36.088536 1. 469835 24.55 < 2e-16 *** lstat -1. 392117 0.167456 -8.31 8.8e-16 *** age -0.000721 0.019879 -0.04 0.971 lstat:age 0.004156 0.001852 2.24 0.025 * Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' , 1 Residual standard error: 6.15 degrees of freedom Multiple R-squared: 0.556 , Adjusted R-squared: 0.553 F-statistic: 209 on 3 and 502 DF , p-value: <2e-16 3.6.5 1m (X^2) > )
> summary(lm.fit2)

Call:
lm(formula = medv lstat + I(lstat-2))

Residuals
Min 1Q Max

-15.28 -3.83 -0.53 2.31 25.41

Coefficients:
Estimate Std. Error t value Pr(>ltl)

(Intercept) 42.86201 0.87208 49.1 <2e-16 *** lstat -2.33282 0.12380 -18.8 <2e-16 *** I (l stat-2) 0.04355 0.00375 1 1. 6 <2e-16 *** Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' , 1 Residual standard error: 5.52 on 503 degrees of freedom Multiple R-squared: 0.641 , Adjusted R-squared: 0.639 F-statistic: 449 and 503 DF , p-value: <2e-16 3.6 • 81 >
> anova(lrn.fit , lrn.fit2)
Analysis of Variance Table

Model 1: rnedv lstat
Model 2: rnedv lstat + I(lstat-2)

Res.Df RSS Df Sum of Sq F Pr(>F)
504 19472

2 503 15347 1 4125 135 <2e-16 *** Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' , 1 ) ct =-l 0· rm f1 ro a--PP ( > , 5))
> surnmary(lrn . fit5)

Call:
lm(formula = medv poly(lstat , 5))

Residuals:
Min 1Q Median

-13.543 -3.104 -0.705
3Q Max

2.084 27.115

Coefficients:

****** ****** ******
)666765 l1444444nunvn4 lleeeeeo >

222140
(-r

<<< 31o e042989 +U r255555 0311111 r· E055555 d +u qu t362555 a542042 m· .... t256221 ru-- 12345 ))))) +b+b+U+U+U+U PAaaaaa ettttt CSSSSE r---14141414 tvdvdvdvdvd n14141411 Ti00000 Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' , 1 Residual standard error: 5.21 degrees of freedom Multiple R-squared: 0.682 , Adjusted R-squared: 0.679 F-statistic: 214 and 500 DF , p-value: <2e-16 82 > )

3.6.6

> fix(Carseats)
> names(Carseats)

[1] “Sales” “CompPrice”” Income” “Advertising”
[5] “Population” “Price” “ShelveLoc” “Age”
[9] “Education” “Urban” “US”

veloc

>
> summary(lm.fit)

Call:
lm(formula = Sales . + + Price:Age , data =

Carseats)

Residuals:
Min 1Q Median 3Q Max

-2.921 -0.750 0.018 0.675 3.341

Coefficients:
Estimate Std. Error t value Pr(>ltl)

(Intercept) 6.575565 1.008747 6.52 2.2e-10 ***
CompPrice 0.092937 0.004118 22.57 < 2e-16 *** Income 0.010894 0.002604 4.18 3.6e-05 *** 0.070246 0.022609 3.11 0.00203 ** Population 0.000159 0.000368 0.43 0.66533 Price 0.007440 < *** ShelveLocGood 4.848676 0.152838 31.72 < 2e-16 *** ShelveLocMedium 1.953262 0.125768 15.53 < 2e-16 *** Age -0.057947 0.015951 -3.63 0.00032 *** Education -0.020852 0.019613 -1. 06 0.28836 UrbanYes 0.140160 0.112402 1. 25 0.21317 USYes -0.157557 0.148923 -1. 06 0.29073 Income:Advertising 0.000751 0.000278 2.70 0.00729 ** Price:Age 0.000107 0.000133 0.80 0.42381 Signif codes: o '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' , 1 Residual standard error: 1.01 degrees of freedom Multiple R-squared: 0.876 , Adjusted R-squared: 0.872 F-statistic: 210 and 386 DF , p-value: <2e-16 3.6 • 83 > attach(Carseats)
> contrasts (ShelveLoc)

Good Medium
nunu nU4Anu

m u

dod aoe

R

veLoc-

3.6.7

> LoadLibraries
Error: obj ect ‘LoadLi braries found
> LoadLibraries()
Error: find

> LoadLibraries=function(){
+ library (I SLR)
+ library(MASS)
+ print(“The libraries have been loaded.”)

> LoadLibraries

library (I SLR)
library(MASS)
print(“The libraries have been loaded.”)
}

> LoadLibraries()
[1J “The libraries have been loaded.”

84

3. 7

X) = GPA , X 2 = IQ , X3 = Gender
X4 = Xs

=20 ,

=
(

i

( /

=

( +

(

(

Yi =

(3.38)

Yi =
0

85

( horsepower

(

i

mpg

(

(

( mpg

(

i

iii. year

(

(

(

( O?

(

(f)
(

(

86

> set.seed(l)
> x=rnorm(100)
> y=2*x+rnorm(100)

(
(y x +

(c)

=
z

ny–M

1.

2

(

(

=

( =

seed
(

0 , 0.25)

(
y = – 1 + O. 5X + B (3.39)

87

(

(

(

( a)

( a)

(

> set.seed(1)
> x1=runif(100)
> x2=0.5*x1+rnorm(100)/10
> y=2+2*x1+0 . 3*x2+rnorm(100)

(b)

(

= 0
=

(

(

(f) (c)
(

> x1=c(x1 , 0.1)
> x2=c(x2 , 0.8)
> y=c(y , 6)

88

(

=Ü?

(

+ B

,

L

,

4.1

(3

y ,), Y2)’ …,

?

overdose seizure

,\, stroke
>'” i 2 , drug overdose

t3″ epileptic, seizure
“‘,

fE drug

seizure

+” default

0666-VOOOON

J

4.2

(1. epileptic seizure
r “, l2. stroke

drug overdose

r “, [0 , stroke
” lj , drug

overdose
1]

4.3

h

500 1000 1500 2000 2500 0 500 1000 1500 2000 2500
Balancc Balancc

[1]4.2 èefault

4.3

92

Pr(default = Yes! balance)
Pr(default=Yes I JlJ

p(balance)

p(balance)

4.3.1

. (4.1)
=

<0 , >1

10
gistic function) ,

p(X) (4.2)
e–

maximum

= e!’w (4.3)
1 – p(X)

p(X)/[ 1 – p(X)

0.2
1 -0.2

O. 9

(
logl 1=

4.3

(4.4)

)
3)

4.3.2

likelihood function)

p(x,) n (l-p(x,.)) (4.5)
i’:yó’ =0

=

O. 005

Intercept
balance

l

10.6513 0.3612 -29.5
0, 005 5 0.000 2 24 , 9

<0.0001 <0.000 1 93 94 ) 4.3.3 1 + e!.+ß,X O. 586 , 58. 32 -e…+iq~e s) YN tt na ee u ss ss ee uu aa ff ee dd b student [Ye$] Q. 404 9 -49.55 3.52 <0.000 1 0.0004 4.3.4 log( (4.6) \1-p(X)1 p(X) (4.7) 1 + 4.3 • 95 income 00 income , [Yes] Intercept balance student (YesJ t 10.8690 -22.08 O. 005 7 0.000 2 24.74 Q. 003 0 O. 008 2 O. 37 -0.6468 0.2362 -2.74 <0.000 1 <0.000 1 Q. 711 5 0.0062 EE 5 II I H 500 1000 1500 2000 No y" Credit Card Balance Status 96 p(X) =0.058 1 + _ 8 ><0 p(X) = .- = o. 1 + e (4.8) (4.9) 4.3.5 overdose seizUre Y = stroke I - Pr( Y = stroke IX) - Pr( Y = drug overdose 4.4 I X (1 (3 4.4.1 4.4 density h( Y = k I X = x) (4.10) I 4.4.2 = \ ç-f 0.5 (4.21)

Pr( I X =.) > 0.2 (4.22)

• 102

9432 138
235 195 430

9667 333

M
AV

dF M H
AF

F
aF

AF
AV

AF
AV

aF
AF

J ,
. ,

0.4 0.2 0.3 0.1 0.0

operating
under the ROC

crnve , Lil–

4.4

ROC Curve

q

B I I f

2

0.0 0.2 0.4 0.6 0.8 1.0

FaIse posìtive

N

P

w P’

FP/N
TP/P
TP/p.

1

104

4.4.4

analysis.

+

+ 1

N N

? N

T
-4 -2 2 4

X, X,
k! =

4.5 105

4.5

p::
x =x

llJ (4.13)

g(Pi(Z)l E iPAZ) J = Co + C]X (4.24) 1 – p, (x)

1og( (4.25)
– Pl’

P >

cross-

SCENAR103 SCENARI02 SCENARIOl

6

106

m
N

SCENARI06

EUK

004-10

SCENARI04

R
O

004-11

4.6 <$0 107 x; , X( 4.6 4.6. j > (I5LR)
> names
[11 “Year” “Lagl” “Lag2”
[6] “Lag5” “Volume” “Today”
> dim(Smarket)
(1] 1250 9
> summary(Smarket)

“Lag3” “Lag4”

Year
Min , :2001 Min. :-4.92200

1st Qu.:-0.63950 1s t Qu.:-0.S3950

Median : 2003
Mean :2003
3rd Qu.: 2004
Max. :2005

Median 0.03900 Median 0.03900
0.00383 Mean 0.00392

3rd Qu.: 0.59675 3rd Qu.: 0.59675
Max. Max. 6.73300

Lag3 Lag4 Lag5
Min , : -4.92200 Min. : -4 , 92200 Min. : -4.92200

Qu.:-O.64000 1st Qu.:-O.64000
0.03850 Median 0.03850
0.00172 Mean 0.00164 Mean 0.00561

3rd
Max. 5.73300

3rd Qu.: 0.59675 3rd Qu.: 0.59700
Max. 5 , 73300 Max. 6.73300

Volume
M1n. :0.356

:1. 257
: 1 .423

Mean :1.478
3rd Qu.:l.642

> (Smarket)

Today

: -0 , 63950
Q.03850

Mllan 0.00314
Qu.: 0.69675

Max. 5.13300

DOllo:602
U1> :648

cor ()

> cor (Smarket)
Error 1n cor{Sm j!.:rket) ‘x’ must be numeric
> cor (Smarltet [“,-9] )

Year Lag1 Lag2 Lag3 Lag4 Lag5
‘0.02970 0.03060 0.03319 0.03669 0.02979

Lag1 0 , 0297 ,.1.00000 -0.02629 -0.01080 -0.00299 -0.00667
Lag2 0.0306 ‘:”0.02629 1. 00000 -0.02590 -0.01085 -0.00366
Lag3 -0.02590 1. 00000 -0 , 02406 -0 , 01881
Lag4 0.0357′;”0.00299 -0.01085 -0.02708
Lag6 0.0298′:’:0 , 00567 -0.00356 -0.01881 -0 , 02708 1. 00000
Volume 0.5390 -0.04182 -0.04841 -0.02200
Today 0.0301 -0.02616 -0 , 01025 -0.03486

Volume Today
Year 0.5 3;90 0.03010
Lag1 0.0409 -0.02616
Lag2 -0.0434 -0.01025

-0 , 00245
Lag4 -0 , 0484 -0.00690

Lag6 -0 , 0220 -0.03486
Volume 1.0000 0.01459
Today 0 , 0146 1.00000

> attach(Smarket)
> plot (Volume)

4.6.2
l’J

linear model)

4.6 • 109

glm

>
• )

> summary(glm.fit)

Call:
glm(formula = ‘”” Lag1 + Lag2 Lag3 + Lag4 + Lag5

+ Volume. ‘7 Smarket)

Deviance Residuals:
Min 1Q Median 3Q Max

-1.20 1.07 1.15 1.33

Estimate Std. Error z value Pr(>!z!)
-0.12600 0.24074 -0.52 0.60

Lag1 -0.07307 0.05017 -1. 46 0.15
Lag2 -0.04230 0.05009 -0.84 0.40
Lag3 0.01109 0.04994 0.22 0.82
Lag4 0.00936 0.04997 0.19 0.85
Lag5 0.01031 0.04951 0.21 0.83

0.13544 0.15836 0.86 0.39

(Dispersion p’aram.eter for family taken to be 1)

Null deviance: 1731.2 on 1249 degrees of
RGsidual deviance: 1727.6 6n 1243 degrees of freedom
AIC: 1742

Number of Fisher Scoring 3

> coef
(Intercllpt) Lag1 Lag2 Lag3 Lag4

-0.12600 -0.07307 -0.04230 0.01109 0 , 00936
Lag5 Volume

0.01031 0.13644
> summary (glm $coef

Error z value Pr{>!z!)
(Intercept) -0.12600 0.2407 -0.623 0.601
Lag1 -0.07307 0.0502 -1.457 0.145
Lag2 -0.04230 0.0501 -0.845 0.398
Lag3 0.01109 0.0499 0.222 0.824
Lag4 0.187 0 , 851
Lag6 0.01031 0.0495 0.208 0.835
Volume 0.13544 0.1584 0.855 0.392
> summary(glm.fit)$coef[ ,4]
(Intercept) Lag1 Lag2 Lag3 Lag4

0.145 0.398 0.824 0.851
Lag5

0.835 0.392

predict

> (glm” “)
:> glm.probs [1:10J

1 :2 3 4 5 6 7 8 9 10
0.507 0.481 0.481 0.515 0.511 0.507 0.493 0.609 0.518 0.489

>
Up

Down 0
Up 1

> glm. (” Down” , 1250)
> glm. pred (glm. probs

table

> table(glm.pred , Direction)
Diraction

glm . pred Down Up
00 101’0 145 141
Up 457 607

> (507+145) /1250
[1] 0.5216
>
[1] 0.5216

145

:>
:>

4.6 R

> (Smarket .2005)
(1] 252 9
> Direction. 200S”‘Direction (! trainl

vector)

! train

>

> (glm . fi t , Smarket respo_nse”)

> .252)
> glm. pred [glm . probs
>

Direction .2005
glm.pred Down Up

Down 77 97
Up 34 44

>

(l J 0.48
> .2006)
(1) 0.52

l a i m 1″ b@ zg ya zp ms ae fx ‘” kp xy at ma ss ao $2 a ‘@ 2k L?m +S -b gt aAA LAZ N az is ec r>
i

nie < ar axp g=s aLbnr u usu gg >>

112

> glm. .252)
> glm.pred[glm.probs>.5)=”Up”
> table (gllll.pred ,

Direction.2005
glm.pred Down Up

Do W”n 35 35
Up 76 106

>
(1] 0.56
> 106/(106+76)
(1] 0.582

> ,
(1.1 , -0.8)) , type””’ response ,,)

2
0.4791 0.4961

4.6.3

>>aa il aa rr = tm e st be us sb G KB ae ak sz am ts a du 2a gt aa Ld t’ 1′ g2 ag m+ t-cg rL i
S( saz < zt rzie affr r-baaeo iddl<1111a ad >>>

cl

probabili_ties of groups:
Down Up

0.492 0.508

G:roup means
Lag1 Lag2

Down 0.0428 O. -0339
-0.0313

Coefficients of linea:r
LDl

Lagl -0.642
Lag2 -0.514
> plot (lda.

X 0.642 x

4.6 • 113

Lagl – 0.514 x
O. 642 x Lagl

0.514 x
predict

> lda. Smarket .2005)
> names(lda.pred)
(1) “class” “posterior n “x”

> lda.
>

Direction .2005
lda. pred Doy ;u Up

Down 35 35
Up 76 106

> .2005)

$

> sum{lda.pred$posterior[.1]>=.6)
<1J 70 > sum(lda.pred$posterior[.lJ<.5) [lJ 182 > lda.pred$posterior[1:20.11
> lda.class(1:20)

(lda.
(1] 0

> +Lag2 • data=Smarket • subset “‘train)
> qda.fit
Call:

“” Lagl + Lag2. data ‘” Smarket.

of
DOlln Up

0.492 0.508

114

Lag1 Lag2
Down 0.0428 0.0339
Up -0.0395 -0.0313

> qda. $claaa
> table (qda. clas3 , .2006)

.2006
qda.class Down Up

Down 30 20
Up 81 121

> møa:n (qda.
(1] 0.699

4.6.5

(1

Xo

(4)
column

> library(class)
> train.X=cbind{Lagl , Lag2) [train.l
> .J
>

,
> knn. pred=knn
>

Direction .2006
knn.pred Down Up

Down 43 58
Up 58 83

> /252
[1) 0.5

4.6 115

> knn.

Direction .2005
knn.pred Down Up

Down 48 54
Up 63 87

> .2005)
(lJ 0.536

4.6.6

> dim (Caravan)
(1) 5822 86
> attach(Caravan)
> $ummary(purchase)

No Yes
5474 348
> 348/5822
[1) 0.0598

> X=scale (Caravan [ • -86])
> var (Caravan ( , 1J)
[1) 165
> var(Caravan (, 2])
[1] 0.165
>
[1] 1
> X [ .2])
[1] 1

116

1000
> J

,J
> yCPllrchase

> set.seed(l)
> knn.pred-knn(train.X , test.X.train.Y.k=1)
> mean
[1] 0.118
>
(1) 0.059

>Y g e t -, daog ees ryy nr
to38

a8N76 ae8

>

kt97 1rny6 bp(o a/

a1

>
> (knn. pred ,

teat , Y
knn.pred No Yes

54
Yes 21 5

> 5/26
(1J 0.192
> (train. X,
> table(knn.pred , test.Y)

test.Y
knn.pred No Yes

No 930 55

117

> 4/15
tlJ 0.267

> (Purchaserv ,

message:
numerically 0 or 1 occurred

> glm. (glm- , fi t • Caravan ,J • type””’ response ,,)
> glm. , 1000)
> glm. pred
> table{glm.pred , test ,y)

test.Y
glm.pred No Yes

No 934 59
Yes 7 0

> , 1000)
> glm. pred [glm. probs >.25] “,” YGS”
> table(glm.pred , test.Y)

test .,y
glm.pred No YeB

48
YGS 22 11

> 11/(22+11)
[1] 0.333

4.7

(4.11)

of dimensionality)

( = 1

118

1] x [0 ,

=0.6 , X, =0.

= 1]

=

(

(

= -6 , ß, =0.05 ,

(

.. 119

(

(

(

(

(

(

(
frame

(

(

(

120

(

()

> Power2=function(x.a){

> Pouer2(3 ,8)

817 •

(result)

“,,”

(

> PlotPower(1:10 ,3)

10 , 23• …,

cross- validation)

el assessment) ,

5. 1

error error rate)

122 ..

1. 1

5.1.1

validation set

hold out

123 n

1

123 ‘.1

iiE . . . . . . . . \ \ i
2 4
> set.seed(l)
> traiu”‘samp1e (392 , 196)

>

> attach (Auto)
> (-trainJ-2)
(1) 26 , 14

> , data”‘Auto
> mean ((mpg-predict (1m.
(1] 19 , 82
> ,3) subset
> mean
(1] 19.78

> (2)
> (392 ,196)
> 1m.
> meau ((mpg-prediet (1m. fit_ , Auto) )
(lJ 23.30
> 1m.
> ,Auto)) -2)
[lJ 18.90
> , 3)
> 1-2)
[1] 19.26

5.3.2

= “binomial

5.3

1m

> (mpg……,horsepower
> coef

horsepower
39.936 -0.158

> .data”, Auto)
>
(Intercept)

39.936 -0.158

glm glm

> library(boot)
> .data “, Auto)
>
> cv.err$delta

1 1
24.23 24.23

cv. glm

> (0 , 5)
> fo1’
+
+
+ }
> cV.error
[1] 24.23 19.25 19.33

5.3.3
CV. glm

> set. seed(17)
> cv.error.l0″‘rep(O , 10)

(1 in ltl0){
+
+ $delta [1]
+ }
> cv , error.10
(1J 24.21 19.34 18.68 19.02 18.90 19.71 16.95 19.50

134 •

5.3.4

alpha. fn

>
+ X”‘data$X (index)
+ [indax]
+ return((var(Y)-cov(X , Y))/(var(X)+var(V)-2*cov(X ,Y)))
+ }

>
[1] 0.576

> (1)
> (100 .100 , replacG=T))
(1) 0.696

boot

> _, alpha. fn.

ORDINARY NONPARAMETRIC BOOTSTRAP

Call

5.3

‘” 1000)

bias error
-05 0.0886

> ,
> boot.fn(Àuto , 1:392)
(Intercept) horsepower

39.936

boot. fn

> set.
> (Auto , sample (392 , 392 ,
(

38.739
> )

40.038 -0.160

> boot. fn , 100Q}

ORDINARY BOQTSTRAP

Call :
‘” R “” 1000)

Statistics
bias std. arror
0.0297 0.8600

-0.158 -0.0003 0.0074

()

Error t value Pr(>!tj)
39.936 0.71750 55.7

horsepowar -0.158 7.03e-81

136 ..

>
+

subset””index) )
> set , S

ORDINARY , NONPARAMETRIC BOOTSTRAP

Call
boot statistic c ” , 1000)

std. error
6. 098
Esti m.ate Std. Pr(>1 t!)

(Intercept) 56.9001 1.80043 32 1.7 stO l.” e”‘rêp 10000)
in 1: 10000) {

“””’4) >0
>
> mean(store)

(
(b)

i

i
E

138 ..

(

fn ()

( fn

(
cv. glm

glm

Up” I Lagl , Lag2)

(
i

, (1)
> y”‘ :t’norm (100)
“> x”‘ :t’norm (100)
>

(

i.

139

+ e
tii. Y +13

+e
frame

(

(

(

(

(Boston $

(

(

Y = ß, (6.1)

1

interpretability) 0

(,infinite)

feature
variable

1)

141

)

6. 1

6.1.1
selectìon)

10

, p

1e.?9

“‘,

+ Jiil

+
p

142

. , , , , ,
, , t , ,

J5í.

Ml. …

6, l.
lìIT

=

6 , 1.2

.. 143

• p-l

“‘,

1 , 2 , … – k) = 1 +

1. 3

+

144 •
113

l
1 1 rating
2 I income
3 \ rating , income , student
4I cards , income student , limit

“‘p , p-l , “‘, 1

.

6.1.3

1.

0

6, 1

MSE

(Akaike information m

+

E
S
E
N

B
E

E
4 6 8 10 2 4 6 8 10 2 4 6 8 10

Number ofPredictors Number ofPredictors

11’1 6-2

e Cp

146 $

AIC

BIC• (RSS + (6.3)

RSS/(n-d-1) = 1 (6.4)

n -d-1
BSS

-1

147 6.2

:;11 •
00

148 •

6.2

6.2.1

mzzb

Xjl =Xa

[49 6.2

– Income
.”. Limit

Rating
Student

0,2 0.4 0.6
IIPflUIIPlb

l.0 0.8 le+02 le+OO

IIß:

X1J

‘6
(6.6)

150

~\/
\

-”
ih

le-Ol 0.0 0.2 0.4 0.6 0.8 1.0
IIßflh/llßlh

f

6.2.2 lasso

151 6.2

=

(sparse model)

(6.7)

, ,
4 , , , ,

,
w

Limit

0 0
0.6 1.0 0.8 0.2 2000 5000 50 100200 500 20

(6.8)

(6.9) ,

152

+

} , 10

lasso

153 • 6.2

I +

s
ii
iz

0.02 0 .1 0

.J ~

1.0 0 ,2 0.4 0.6 0.8
00 Training Data

10.00 50.00 0 .50 2.00
A

——–

0.02 0.10 0.50 2.00 10.00 50.00
A

S
M

J
0.4 0.5 0.6 0.7 0.8 0.9 1.0

R2 on Training Data

f!P n

PTLMR
+

PTLHR
(6.12)

ß; = r/ (1

(6. 13)

(6. 14)

6.2

Ir,

m
M

dv

-1.5 -0 .5 0.0 0.5 1.0 1.5
Y,

!…
-0.5 0.0 0.5 1.0 1.5

Y,
006-10

=

X) =f(YI

Y =ßo + … + Xpßp + e

156

d I I d
â

I d
/k ez

\
;;H / \ I ;J
d1 / “- I d

2
-3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3

6.2.3.

jiij , gJ
5e-03 5e-02 5• 01 5e-02 5e-Ol

(signal variable) 0

s
15

iE
0.0

)57

0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6
IIp,’ II ,l lI pll , IIp,’I!, II! IÎI!,

1.0 0.8

X,

Z2. •..
P ZmzZM

m =
M

Yi = ()o + L (J”,Zirn + = l ,”‘,n

+

{>f M P P M Þ

(6. 16)

(6.17)

6.3

158 •

M

(6. 18)

Zz. •.•

6.3.1
components

PCA

pop

R

ZE
BE

20 30 60
Population

Z , = 0.839 x (pop -pop) +0.544 x (ad – ad) (6.19)

+
=

X (pop -pop) (ad –

839 x (pop, – pop) + O. 544 x
zu’ …,

(6.20)

159 6.3

• • • 0 10
Population 1st

20 20

= =

=0‘ 839 x (pop , – + O. 544 x ( ad, – ad) < .' .... .; . : .... ••. .. 3 " .·kt . . .. ". : .., . " …3 > 2 -1 0 1 2 -2 -1 0 1 2
1st Princìpal Component 1st Principal Component

aE

2 2

s

160 •

Z, = 0.544 x (pop – pop) – O. – ad)

. .. . .
..

··..-··
.•

‘P·t·-.
…. –

2··· ..a
,,e

.
. .
..

. .
.

.
l

7
Il’ I
‘”

s

.

-0.5 0.0 0.5 1.0
2nd Principal Componcnt

1.0 0.0 0.5 1.0
2nd Principal Component

1.0

prirúiì Zu.

12

• 161

jjh\\
o 10 20 30 40 0 10 20 30 40

Number ofComponents Number of Components

PCR

;:
Ridge Regression and Lasso

/
/

o 10 20 30 40 0.0 0.2 0.4 0.6 0.8 1.0
Number of Compouents Shrinkage Factor

e of Statîstìca}

@ 162

\–,-, Rating Student

2 4 6 8
Number of Components

:të:

2 4 6 8 10
Number of

“‘,

6.3.2

163 6.4

mEE#Z

60 50 40
Population

30

thogonal-

6 , 4

6.4.1

164 •

6.4.2

= 1

flexible)

m

$A e

..’ @

@

@

./ .
-1.5 0.0 0.5 1.0

X

• m

?

-0.5 0,0 0.5 1.0
X

1

s

5 10 15 5 10 15 5 10 15
Nurnber ofVariables NumberofVariablcs

6. L

6.4.3

=20 ,

of dimensionali-

@ 166

p=2000

N

p “, SO
m

N

p”’20

n

N

1 70 111

p

=2 las.

1 28 51
Degrees ofFreedom

1 16 21

6, 4, 4

6.5 167

=

6.5

6.5.1

na

> library (ISLR)
> fix (Hittars)
> names(Hitte ;ra)

[1J “AtBat” “HmRun” “Runs” “RBI”
[$] “Walka” “Yaars” “CAtBat”

“CRuns” “Laague” “Division”
[16) “PutOuts” “Asaists” “Errors” “Salary”
> dim (Hi tters)
(1] 322 20
> sum(is
[1] 59

>
> dim
[1] 263 20
>
(1] 0

regsubsets

> library (1eaps)
>
>
Subset abject
Call: regsubsets .formula(Salary ‘” .,
19 Variables

1 SUbSêts ot each ta 8
Selectian Algarithm: exhaustive

AtBat Hits HmRun Runs RBI Walks Years

• 168

s t

*G

J 8 1 N e u

“”””””””g

se -g CUUUHHH”””L ze

r

US Ri-

>}>>>>>>)>>>>>>>>>>>>>>>111111111111111111111111 <{<<<<<<<<<<<(<<<<<<<<<<12848678 12845678 12345678 subsets .nvmax=19) > reg. full)

summary

> nam ,es (reg . summary)
(1) “which” “rsq”
(7)

“bic” “cp” “adjr2 ” “rss”

6.5

56 34 65 00 96 24 85 00 46 14 58 00 95 04 58 00 15 94 45 00 84 74 45 00 14 54 45 0o
q 683 r24 $45 y roo a m106 m244 usss s eooo g e]]] r109

-11
>

EE

>.
> (reg. summary$rss , xlab=”Number of Variahles”. •

> summary$adjr2 of Variables” ,
yhb””’

points fg points
max

>
[1] 11
> (11 , reg. summary$adjr? [1 1], col””’red” , pch”’20)

> summary$cp , xlab=” Number of Variables” , ylab=”Cp” ,
type””l’)

> lõhich. min Creg. summary$cp)
(1) 10
> points (1 0 , reg. summary$cp [10] ,
> )
[1] 6
> plot (reg. summary$bic , xlab””Number of Variablas” , ylab=” BIC” ,

)
> pCh””20)

regsubsets

>” 2)
>
r
>”

2dpz racb “””” ZUEm ll-I aaaa cccc sggg ”” 1111 1111 UUUHU ffff tttt 11ii ffff gggg eeee rrrr <<<>>>

t9 a6
>
88

6t ,A-
zm l u f t i f

>
2

gtz ep5 ze < liits 7.604 Walks 3.698 CRBI 0.643 DivisionW -122.952 0.264 170 6.5.2 "" " =" >
“)

> 8ummary(regfit.fwd)
> , data=Hi tters • nvmax=19 •

“)
> summary( :regfit.bwd)

> coaf
Hits

79.451 1.283
CHmRun

1.442 -129.987
> coef

109.787 -1.959
CWalks
-0.305 -127.122

> coef(regfit.bwd ,7)
(Intercept)

106.649 -1.976
CWalks DivisionW

0.716 -116.169

Walks
3.227

PutOuts
0.237

Hits
7.450

PutOuts
0.253

Hits
6.767

0.303

6.5.3

CAtBat
-0.375

Walks
4.913

Walks
6.056

CHits
1.496

CRBI
0.854

CRuns
1.129

> set.aeed(l)
> train=sample(c(TRUE , FALSE).
> test”‘(!train)

6.5 1> 171

> (Salary””,. (traln ,J.
nvmax=19)

test. (Sala:ry”-‘. • [test .J)

mode l. matrix

> val. (NA , 19)
in 1:19){

+ id=i)
+ names
+
>

> val. errors
(lJ 220968 169157 178518 163426 168418 171271 162377 157909
[9] 164056 148162 151156 151742 152214 157359 158541 158743

[17] 159973 159860 150106
> which.min(val.errors)
(1) 10
> coe :f , 10)

AtBat

CHits
1. 105

PutOuts
0.238

CHmRun
1.384

7.163
CWalks
-0.748

Walks
3.643

CAtBat

LeagueN

> predict. regsubsets •
+ ([2J J)
+ matm lll; odel.matrix(form , newdata)
+ coefi=coef(object

+ mat [, xvars)%* Y. coefi
+ }

172 •

> CSalary””. , data=Hitters •
> coef(regfit ,best ,10)
(Intercept) AtBat

162.535 -2.169
CRuus CRBI
1.408 0.774

0.283

Hits
6.918

CWalks
0.831

Walks
6.773

-112.380

CAtBat
-0.130

PutOuts
0.297

> k=10
> set , se8d(1)

> pasteC1:19)))

in l:k){
+ best. fit”‘regsubsets (Salary…….. [folds l”‘j , J ,

nvmax=19)
+ 1:19){
+
+ (j , i] “‘mean ( (Hi tters$Salary [folds ='” j] -pred) -:2)
+ )
+ )

x

> , 2 , mean)
> mean.cv , errors

(1) 160093 151159 146841 138303 144346 130208
(9J 129460 125335 125154 133461 133975 131826 131883

(17J 132751 133096 132805
> par
> CV.

6.6 .. 173

> (Salary”-‘. •
> coef Creg. best , 11)
(Intercept) Walks CA tBat

135.751 -2.128 6.924 5.620 -0.139
CRuns CRsr CWalks DivisionW
1.455 -0.823 43..112

Assists
0.289

6.6

> Hitters) [, -1)
>

mode l. matrix

6.6.1

> library (glmnet)
> length””lOQ)
>

x

> dim(coef(ridge.mod))
(1) 20 100

498

174 -to

> ridge. lIlod$lambda [50]
t1 J 11498
> coef(ridge .mod) [, 50]

407.356 0 , 037 0.138
RBI Walka Yeara

0.240 0 , 290 1. 108
CR.uns CRBr

0.088 0.023 0.024
Assists

-6.215 0 , 016 0.003
> sqrt (sum )
[1] 6.36

HmRun
0.526

0.003
CWalks
0.025

Runs
0 , 231
CHits
0.012

0.085
NEl wLeagueN

0.301

> ridge. mod$lambda [60]
(1) 705
> coef(ridge.mod) [.60]
(Intercept)

64.325 0.112 0.656
RBI Walks Years

0.847 1. 320 2.596
CRuns CRBr

0.338 0.094 0.098
Assists

-54.659 0.119 0.016
> sqrt(sum(coef(ridge.mod) [-1 , 60]A2))
(1) 57.1

HmRun
1.180

CAtBat
0.011

CWalks
0.072

Errors
-0.704

0.938

0.047
LeagueN

13.684
NEl wLeagueN

8.612

>
AtBat HmRun

48 ‘ 766 -0.368 1. 969 -1. 278
RBI Years

0.804 0.005
CHmRun CRuns CRBI CWalks

0.624 0.221 0.219 -0.150
OivisionW PutOuts Assists Errors

-118.201 0.250 0.122 -3.279

Runs
1.146
CHits
0.106

LeagueN
45.926

Ne 1ol’LeagueN
-9.497

> (1)
nrow(x)/2)

>
>

175

“” n

> , J , y [train] , alpha=’O , lambda”‘ßrid ,
-12)

> ridge. prad”‘pradict )

(1] 101037

(1] 193253

> ridge.pred=predict(ridge.mod , s=le10 , newx=x[test .])
>
<1] 193253 > . mod , $=0 , newx=x [test , J ;
> maan((ridge , pred-y.teat)-2)
[1]
>
> predict . mOd , 5″‘0 , exact “‘T , type””” coaffic ients”) [1: 20 ,]

glmnet

cv. glmnet

> sat.saad(l)
> , y alpha=O)
> plot (cv.
> min
> ..

212

predict
glmnet

176

> .mod .l)
> -2)
(lJ 96016

Runs
1.1132
CHits

0.0649
LeagueN
27.1823

Ne l.TLèagueN
7.2121

> out=glmnet
> pred i.ct (out , type””’ coøff ic ients” • [1:20 ,]
(Intercept) AtBat Hits HmItun

9.8849 0.0314 1. 0058 0.1393
RBI Walks Years CAtBat

0 , 8732 1. 8041 0.1307 0.0111
CHmRun CRuns CRSI CWalks
0.4516 0 , 1290 0.1374 0.0291

DivisionW Errors
91. 6341 0.1915 0.0425 -1. 8124

lasso

“”

6.6.2

> , alpha”‘l ,
> plot(lasso.mod)

> set.seed( 1}
> cv. ,J ,
> plot(cv.out)
> bestlam=cv. out$lambda. min
> , neyx”‘x [test ,))
>
[1) 100743

6.7 @

>
s”‘bestlam) (1 :20.J

> lasso. cO ,ef
(Intercept) AtBat Hits HmRun R.uns

18.539 0.000 1.874 0.000 0.000
RBI Walks Yêars CAtBat CHits

0.000 0.000 0.000 0.000
CHmRun

0.000

-103.485

CRuns
0.207

0.220

caBr CWalks LeagueN
0.413 0.000 3.267

Assists Errors
0.000 0.000

Walks caBr
2.218 0.207 0.413

PutOuts
0.220

> lasso.coef[lasso.coef!=O]
Hits

18.539 1.874
LeagueN DivisionW

6. 7

1

> library{pls)
,
> pcr. (Salary””” .scale=TRUE ,

pcr

scale

“”

pcr

> summary(pcr.fit}
Data: X 19

Y dimension: 263 1
Fit svdpc
Number of considered: 19

VALIDATION: RMSEP
Cross -validated using 10 random segments

(Intsrcept) 1 comps 2 comps 3 comps 4 comps
CV 452 348.9 352.2 353.5 352.8
adjCV 452348.7351.8352.9352.1

538 p64 c84 8 390 p29 c84 8 $32 p02 4 $47 P81 m
@C74 n ig a l ps68 xp15 6m ec64 22 a ez r$13 apse vm

080
%c34

1
0 N ZY MUZ za A-Ra TXS

178 ..

pcr mean

error)

= 124

va l. type =”

>

M=19 ,

summary of

=p =

, (1)
> pcr. fi t”‘pcr (Salary”‘. , data=Hi ttars • subset =train , scale=TRUE ,

validation “,” CV”)
> validationplot (pcr. fit. val. type=” MSEP “)

> ,x(test ,J , ncomp=7)
-2)

[1] 96556

> pcr (y””‘X. , ncomp=7)
>
Data: X dimension: 263 19

Y dimansion: 263 1
6vdpc

Number of components 7
TRAINING: % variance explained

x
y

x
y

2 comps 3 4 comps
38.31 60.16 70.84 79.03
40.63 4 1. 58 42.17 43.22

7 comps
92.26
46.69

5 comps
84.29
44.9 0.

6 c_omps
88.63
46.48

179 6.8

()

::> ßet. seed(l)
> pls. fi t”‘pl .s r (Salary^’,. scale”‘TRUE ,

;> summary(pls.fit)
19

Y dimension: 131 1
kernelpls

Number consider

TRAINING: Y. variance explained
1 cOmps :2 comps 3 comps

38.12 53.46 66.05
33.58 38.96 4 1. 57

>2 P c ” 3 s e t [ x ‘>aL
>

ss le pt >[

>
>
Data: X dimension: 263 19

Y dimension: 263 1
method: kernelpls

Number of COmponents considered: 2
TRAINING: % variance explained

2 comps
38.0$ 51.03
43.05 46.40

X
Salary

180 @

6.8

1 , 2 , …

(k+

(k + 1

(k +

(k +

(k +

(

(

llJ

?ilJ
(

(

(

p=2 , X21 X 12 +Xn =0 ,
(3,

(
=ß20

(

(

182 ..

(c)

(,)

(

Xl , ,..

fram

(

+ 8

(

183

( n=l

(

(

(

additive

7.1

1

7.1

7. 1

+ S;

+ Si (7. 1)

“‘, 1)

Polynomial

s R O “.,


h

20 30 40 50 60 70 80
Age Age

J(x,) = (3, +

E

, ,
s

“”

186 ..

> 250 I x;) (7·3)
1 + exp(ßo + …

7.2

C,(X) = I(Xg
h(x ,g) = (x – g)’. = 1

10

X2 , g,), g,

(7.10)

s
‘’ ::p

7.4.4

50%.

191 7.4

;:1(;:j
Natural Cubic Spline

\

tlili–

ijL~
2 4 6 8 10

Degrees ofFreedom ofNatural Spline
2 4 6 8 10

Degrees of Freedom of Cuhic Spline

192 ..

7.4.5

11

s
B

a
“e

20 30 40 50 60 70 80
Ago

7.5

7.5.1

3

‘”

7.5 @

g’

7.5.2

effective degree of

S,y (7.12)

193

(7.13)

X! ,

I l’ = >: (x,))’ =
\J’ CA

7.6

Spline

EE

w w w w
Ag,

v

bL

Regression

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

.
“‘ßo

(7.14 )

=0.

7.1

additive model ,

196 •

7. 7

Local

E

20 30 40 50 60 70 80
Ag,

7.7.1

YI = ßf}

+ 8 1
(7.15)

+ + …+ !p(x/p) + 8 ,

wage + e (7.16)

Coll
R

20 30 4{) 50 60 70 80 2009 2005 2001 2003

IE

198 • UT R -• A
education

7.7.2
= IT

g(4L) 1= ßo + … + ß,X, (7.17)
1 -p(X)1

Y = 1IX)/P( Y

+ (7.18)
I

7.7

X
1 – p(X)

(7.19)

p(X) =Pr (wage >250! year , age. education)

N-

NUO

2003
education

2005 2007 2009

-?

HS

NO

?

,
2005 2007 2009 20 30 40 50 60 70 80

year age education
2003

200

7.8

> library , (ISLR)
> (Wage)

7.8. i

> fit=lm(wage”‘poly(age ,4)
> )

Estimate Std. Error t value Pr(>ltl)
(Intercept) 111.704 0.729 153.28 <2e -16 poly(age , 4) 1 447.068 39.915 1 1. 20 <2e-16 poly{age , 39.915 -1 1. 98 poly{age , 4)3 125.522 39.915 3.14 0.0017 poly{age , 4)4 -77.911 39.915 -1. 95 0.0510 age^2 , () age^2 , >
> coef(summary(fit2))

Std. )
(Intercept) -1. 84e +02 6.009+01 -3.07 0.002180

4 , 2.12e+01 5.8ge+00 3.61 0.000312
poly(age. 4 , raw ‘” 1)2 -5.64e-01 2.06e-01 -2.740.006261
poly(age , 4 , 6.81a-03 3.070-03 2.22 0 ,026398
poly(ago , 4 , raw “” 0.051039

> • data=Waga)
>

I(age-4)
-1 ,84e+02 2 ,12e+01 -3.20a-05

>

çbind

7.8

>
> age. grid=seq (from “, agelims [lJ [2] )
>
>

>
> , wage • :x:lim=agelims • cex”‘.5. col=” darkgrGy”)
> title (“Degree -4 Polynomial” .-outer”‘T)
> <:(1 1 =" blue ,,) > (age. 56 . bands , 1wd”‘1 , co1=” blue” , 1 tY””3)

()

> preds2″‘predict
> i t))
[1]

anova

t . 1′” , data=Wage)
>
> fit. (age .3) ,data=Wage}
>
> fit
> .4 , fit .5)

of

Modal 1: wage ,…, age
Model 2: wage ,…, polyCage , 2)
Model 3: waga ‘”
Model 4: wage ‘” poly (age , 4)
Model 5: waga ,…, polyCage , 5)

Res . Df RSS Df SUlIl of Sq
2998 5022216

F Pr(>F)

670 -os e00 2 991 588 393 4 1 660 857 770 856 21 2 111 044 370 466 37k 977 777 444 765 999 999 222 234

5 2994 4770322 1 1283 0.80 0.3697

Signif. codes: 0 0.001 0.01 0.05 0 , 1 ‘ , 1

( < 202 .. > coaf(summary(fit.5))

poly (age. 5) 1
5)2

poly(age.6)3
poly (age. 5) 4
poly(age , 5)5

Std. Pr (> 1 t 1)
11 1. 70 0.7288 153 , 2780 O.OOOe+OO
447.07 39.9161 1 1. 2002 1. 4916-28

-478.32 39.9161 -1 1. 9830 2.3688-32
125.62 39.9161 3.1446 1. 67ge-03
-77.91 39.9161 -1. 9519

35.81 39.9161 -0.8972 3.697e-01

anova

:>
[1] 143.6

> 1-1m (wage”-‘education +age •
;. fit .2=lm(wage,,-,educatiou+poly(age ,2). , data=Wage)
> 1m (wage”-‘education +poly (age .3) • data=Wag El)
> .3)

000

> >250) “‘Po1y (age • data=Wage , family “, binomial)

predict

> (age=age. giid) , se””T)

=

P,(Y=IIX)
1 + exp(Xß)

7.8

> )
> se.bands.logit ‘”

preds$se .:f it}
> se.bands ‘” exp(se.bands.logit)/(l+Gxp(se.bands.logit))

> •
se”‘T)

> I (wage >250) • , tY )i) e=”n” , ylim=c (0 .‘ 2) )
> points(jitter(age} ,

col=” darkgrey “)
> Cage .grid col=”blue”)

(age . grid • S6 . bands , 1wd=1 , col=” blue” •

> table
(17.9 , 33.5] (33.5 ,49) (49 , 64.5] (64.5 , 80.1J

750 1399 779 72
> , 4) , data”‘Wage)
> coef(summary(fit))

cut (age , 4) (33.5.49]
cut(age , 4)(49 , 64.6)
cut

Estimate Error t value
94.16 1. 48 63.79 O.OOe+OO

13.15
23.66 2.07 11.44 1. 04e-29
7.64 4.99 1. 53

<33. 7.8.2 () > library(splines)
> age ,knots”‘c (25 ,40 , 60) ) •
>
> wage.
> linas (age . grid , pred$:f
> , 1
> , lty=” dashed “)

204 •

> dim(bs{age , knots=c(25.40.60)))
[1J 3000 6
>
[1] 3000 6
> att x: (bs(age.df=6)

25%
33.8 42.0 51. 0

> fit2=lm(wage “-‘Ils(age ,df=4) .dat ð. =Wage’)
>
> pred2$f i t , col “,’1

ns

>
Spline”)

> fi t”‘smooth. spline (age , wage • df =16)
> (age ,yaga , cv”‘TRUE)
> fit2$df
[1J 6.8
> lines(fit ,
> , lwd”‘2)
> legend (., topright ” 16

cOl”‘c (“red”.

spline

> plot Cage , Vage , •
> title (“Local
> fit”‘lollss (wage”lage data=Wage)
>
> lines(age.grid , predictCfit :f rame (age “‘agG . grid)) ,

> lines (age. grid , ,data. :f rame (age”‘aga. grid)) ,
, lwd””2)

> legend (” topright ” ,” •
cOl”‘c (“red” , “blue “) lwd”‘2 , 8)

=0. “”‘0.

7.8

7.8.3 GAM

(

>

> library (gam)
>

> (1 ,3) )
> plot{gam.m3. , col””’blua”)

> plot. gam (gaml , se=TRUE , col “,” xed”)

gam

>
> gam. m2=gam (wage”-‘year+s (age .5) +edu.cat ioo • data=Wage)
> F”)
Analysis of Table

vaga s(age , 6) + education
Model 2: wage ‘” year + s (age education

3: wage ‘” s(year , 4) + s(age , 5) + aducation
Resid. Df Resid. Dev Df F Pr(>F)

1 2990 3711730
2 2989 3693841 1 17889 14.5 0.00014 ***
3 2986 3689770 3 4071 1.1 0.34857

Signif. codes: 0 0.001 ‘**” 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ , 1

206

> summary(gam.m3)

Call: gam (formula ‘”‘lage ,…., 4) + s(age. 5} +
data ‘” Wage)

Deviance Residuals:
Min 30

14 , 17
Max

213.48

(Dispersion Parameter 1236)

5222086 on 2999 degrees of freedom
Deviancê: 3689770 on 2986 degrees of

AIC: 29888

Number of Local Scoring 2

DF for Terms and F-values for Nonparametric Effects

(Intercept)
a (year , 4)
s{age , 6)
education

Df Npar F
1

Pr(F)

1
4

3
4

1. 1 0.35
32.4

Signif. codes 0.001 0.01 0.05 ‘.’ 0.1 ‘ , 1

>

>
data=Waga)

> p1ot. gam (gam .10. se=TRUE. co1″”’ green “)

>

> 1ibrary(akima)
> plot(gam.lo.i)

207

nomial o

>

> par(mfrow=c(1.3))
>

:;. I (wage >250))

ed’ucation FALSE TRUE
1. < HS Grad 26. Grad 966 8 3. Some Col1age 643 7 4. College Grad 663 22 6. Advanced 381 45 :;. family'" data"'Wage , subset "'(education < HS Grad"}) :;. plot(gam.lr.s , se"'T , col="green") 7.9 ß3' =ß, +ß3X3 (x ( ßl. b,. C1 • djO aZ + bzx + C2 ;:(,2 + d2::lIl ( ( ( !', W ( = a1 + 2c1 ( 208 • m=l ( m=2 ( m=3 ( m=3 (X) + e = 1, 13, = 1 , ßl = - 4 =1(0 ,,; X<;2) b, (X) = + l( 4 X ;;, +ß,b,(X) +8 5 g, - - g(X,))' (x ( j obclass ( ( ( ( ( 11. ( 100 0 + B >
> (2J

Y-ß2X2 =ßo +e

>
> [2}

(

210

()

(

8. 1

8.1.1

<4. Years < x I Years < 4.5 1 • Rz ::::: I X ! Years'> “‘” IX
IYears> =4.5 , Hits>=117.5}o

000 x xe5. 999 :::::402 000 X e6. 740 :::::
845 346

212 •

Yea.r<4.5 5.11 il-- 6.74 < 6.00 117.5 terminaI < J 1 1 45 Yean 24 Years.. (1 Xz , 8 3 2 .. : ! ; ; u··E·E·-81· ·····2···288 8.1 • 213 R,o xeRz. Rp "', Rz. .., R, o L L (y,- (8.1) IcR, R,(j,s) = IXlx, = 214 .. I -") X, R‘ R, R, R, R, t. t, ., X, R, R ‘ 2 6 Rz. "', complexity ing) (Yl - YR)2 rl (8.4) i F 8.1 • 215 .", (b) . (cross Year<4.S 6.189 216 • 2 4 6 8 10 Tree Size 8.1.2 occurring (classifica- tion error EZI (8.5) (8.6) 8.1 .. 217 Thal Thal: 00 < 8.1.3 = ß. + r,Xßj (8.8) M • 1 (8.9) 218 .. 5 10 Tree Size y" 15 No 8.1.4 8.2 X, x, < ? -2 -1 0 2 -1 0 2 X, X, 8.2 '1 8. bootstrap aggregation) 220 • , RP , J"(x) vote) ES Test:Bagging SO 100 IS0 200 250 300 Number ofTrees 8.2 " 8.2.2 Fb, ExAng S library

> library (ISLR)
> )
> High=ifalae (Sales

frame

> (Carseats

> -5a1es , Carseats)

> summary(tree.carseats)

Classification trae
tree(formula = • – 5a1es , data ‘” Ca.rseats)
Variabl (-J s used in tree
(1) “Priçe” “Income” “CompPrice ”
(5)
Number of nodes; 27
Residual mean deviance: 170.7 I 373

rate: 0.09 ‘” 36 ( 400

-27

text

>
>

< 226 • :> tree.carseats
node) , aplit , n , deviance , yval , (yprob)

* node
root 400 54 1:5 No ( 0.5’90 0.410 )
2) ShelveLoc: Bad , Medium 315 390.6 No ( 0.689 0.311 )

4) Price < 92.5 46 56.53 Yes ( 0.696 ) 8) Income < 12.22 No ( 0.700 0.300 ) =" :> set.seed(2)
:> 200)
:>
:>

:> tree.
:> tree.
:>

Higb.
tree. pred No Yes

No 86 27
Yes 30 57

:> 1200
[1] 0.715

tree

tree

cv. tree

:> set.seed(3)
:> cv.
> names(cv ‘ carseats)
[1 J “k” “ll1ethod”
:> cv. carseats

[1] 19 17 14 13 9 7 3 2 1

$dev
[lJ 55 55 53 52 50 56 69 65 80

$k
[1) – Inf 0.0000000 0.6666667 1 ‘ 0000000 1.7500000

2.0000000 4.2500000
(8) 5.0000000 23.0000000

(1J

[1 J “tree. se CJ.uence”

227 • 8.3

> par
> , CV.
> plot (cv. • cv . carseats$dev

> prune.
> ‘plot (prune . carseats)

carseats ,

> tree. .
> table

High. test
tree.pred No Yes

No 94 24
Yes 22 60

> (94+60) /200
[1] 0.77

> prune. best=15)
> plot(pruna.carseats)
>
>
> ,pred

.pred No Yes
No 86 22
Yes 30 62

> (86+62) /200
[lJ 0.74

8.3.2

> library (MASS)
> set. seed

= nrow (Boston) /2}
> tree. (medv,””.
> summary

Regression tree
‘” medv .. subset ‘”

used
[1] “dis”

nodes: 8
deviance: 12.65″ 3099 / 245

Distribution of residuals
Min. 1

-2.0420 -0.0536 12.6000
3rd Qu

1.9600
Mean

0.0000

228 •

> plot(tree , boston)

<9 , tree > boston)
>

prune , tree

>
> plot
> , pretty”‘O)

> , newdata””Boston (-train ,])
>
> plot{yhat , boston.test)
> a’bline (0 ,1)
>
[1] 25.05

8.3.3

>
> set. seed (1)
> bag. (madv……. ,. dat a.””Boston.
> bag. boston

Call:
‘” ., data ‘” Boston.

importance ‘” TaUE. eub’sat .. train)
Type of random

Numbar 01 trees: 500
No. of variables tried at each split: 13

Mean of squared 10.77
86.96

.. 229

> yhat. bag ‘” predict (bag. bo’ston , (-train ,J)
> plot (yhat. bag.
> abline (0 ,1)
> mean ((
[1] 13.16

randomForest

>
ntree”’25)

> yhat. bag ‘” predict (bag.
>
[1} 13.31

randomForest

> set.sGGd(l)
> rf , (medv”-‘. , • sllbset “‘train ,

mtry”‘6 , importance “‘TRUE)
> predict J )

(1) 11. 31

> importance (rf.boston)
%IncMSE

zn 2.103 50.31
indus 8.390 1017.64
chas 2.294 66.32

12.791 1107.31
rm 30.754 5917.26
age 10.334 552.27
dis 14 , 641 1223.93
rad 3.583 84 , 30
tax 8.139 435.71
ptratio 11. 274 817.33
black 8.097 367.00

30.962 7713.63

>

230

8.3.4

=” gaussian”

tion =” trees “” 5

> libr ll.ry{gbm)
> set.seed(1)
> .data=Boston(train ,]

“gaussian” ,n. tre l’l s””5000 , interaction. depth=4)

>
var rel ‘ in’

45.96
2 rm 31.22
3 dis 6 ‘ 81
4 crim 4.07
5 nox 2.56
6 ptxatio 2.27
7 black 1.80
8 age 1.64
9 tax 1. 36
10 indus 1.27
11 chas 0.80
12 rad 0.20

zn 0.015

ence

> par (mfrow”‘c 2) )
> plot
> plot i””’ )

> yhat.
ll.txaes=5000)

>
[lJ 1 1. 8

(8.

> b005t.
,n. txees”‘5000 , shxinkaga =0.2.

vaxbose “‘F)
> yhat.

n. txaes =5000)
> mean-((yhat
(1) 11. 5

231 8.4

8.4

R2 •
t’}..

2.

1 –

0.1 , 0.15 , 0.2 , 0.2 , 0.55 , 0.6 , 0.6 , 0.65 , 75

X2<1 -1.06 0.63 -'1.80 232 • ( ( ( ( ( ( ( tree ( ( ( ( ( ( 233 , ( ( margin vector vector machine) 9. 1 9. 1. 1 +ß,X, = 0 (9.1) 1) (X" 9.1 235 , )(,) +ß2X:Z + ... = 0 (9.2) X" "', X,)' + ßzXz + ... < 0 (9.4) (9.3) UU ZZ(::) RP YI' ... E 1 -1 ,1} (xt = -1 + "11 …………JY -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 X, +2X! +2X1 +3Xz + 2X1 (9.5) 9. j.2 (9.6) < 0 , 0::: 1 , + ß,',,) > 0 (9.8)

=ßo +ß2XZ.

(9.7)

1

236 •

N . .

1
X,

2 3 -1 2 3

00 9.2

9.1.3

maximal margin hyperplane)
separating

margin

) +

9.1

vector)
p

9.1.4

Xn E Y2′ .,. Yn E 1 – 1,

. .

X,

maximizeM (9.9)
P..{J”….fJ,

I.ß! = 1 (9.10)
r,(ßo + ß1XIl +… + ßpxip) i = (9. 11)

Yj(ßq M , i = 1’…..n

+ … >

(9.9)

238

.
.
aw–

.
1 2 3

.

. . . .

. .

.

.

. .

.

.
mOOM

‘”

50ft

tor

9.1.5

9.2

9.2.1

. . m
-… . . . . . . .

F

3 2 3 2 1

9.2

vector

? 6$J

i 4 5
8

-0.5 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 2.5

9.2.2

maximize M

1

HO , ZMC

(9.12)

(9.13)

(9.14)

(9.15)

11
(9. 8″ variahle)

– (9.

(9.12) –

– (9.11)0

C

241 9.3 •

..:. ,
J‘.


3

-AV

-J-//
M;’

N

4tII .1 ;:..f 0
, . ” , , , , , , ,

,

2

.
• 0/.0

M; •

N

N

2

. . .

.
2

. .
• . ”

4

0 x;

x;
.

H

? N

2

– (9.

2

9.3

9.3.1

• 242


. .

.
.. ”al

..•
,8··· …

-14
.. .

,. . . .
.’\ e!

1i ‘:,
f
-2

‘*

N

?

0 4 2 4

x; ,
….

maximize M

1

(9 , 16)

9, 3, 2

9.3 .. 243

– inner product)

(9.17)

(9.18)

i = ,

= A + E

(9.20)

K(x”x ,.) = (1 (9.22)
kernel)

244 •

>

N

‘;>1 .
@

-4 -2 2 4 -4 -2 2 4
X, X,

kemel)

K(xnxi.) = (9.24)

T
)T

(x/

9.3

9.3.3

=

>

< u U U M U ,. U U U M U ,. False False positive rate 10-3 • 246 @ " q " • • SVM: 'FIO.l 0.2 0.4 0.6 0,& False positive 0 10 • ..LDA 02 0.4 0.6 0.8 False positive rate e 0.0 1.0 10 1.0 9.4 9.4.1 9.4.2 9.5 + ... + 9.5 =ß. + - (9.15) ilJ .1 15 I . (9.26) L(X , y)o (9.25 ] 1088) + ... === 00 248 @ • SVM Loss Regression Loss -6 -4 -2 0 2 + ". + +ßIXiJ + vector ß, 9.6 9.6.1 "linear". 9, 6 > 6Gt. aead
> x_””matrix{rnorm ‘(20*2). ncol”2)
> y=c(rep(-1.10) , rep(1 ,10)}
> + 1

> plot(x , col=(3-y))

>
> library(el071)
> svmfit=svm(y””. , data”‘dat. kernel=”linear”.

=

=TRUE o

> dat)

plot. svm

()

> svmfit$inde>:
(1) 1 2 5 7 14 16 17

>
Call
$vm(formula y = cost ‘” 10 ,

scal <'l '" FALSE) Parameters SVM-Kernel: linear cost: 10 gamma: 0.6 Nuwber of Support ( 43) Nurnber of Classes: 2 Levels -1 1 > svm.f i t “”svm (Y””‘, data=dat , kernel””’ linear”.

> dat)
>
[1] 1 2 3 4 5 7 9 10 12 13 14 15 16 17 18 20

jiii

> set .seed (1)
(8vm , Y””‘” • data”‘dat , kernel “,” linear” •

0.1.

> ßllmmary(tune.out)
tuning of

cross
– best

– best performancG: 0.1
performance result .s:

cost error
1 1e-03 0.70 0.422
2 0 , 70 0.422
3 0.10 0.211
4 0 ‘ 15 0.242
5 5e+00 0.15 0.242
6 16+01 0.15 0.242
7 0.15

=0.

> bestmod=tllne.out$best .model
> sllmmary

predict

, {rnorm
> 20 , rep=TRUE)
> + 1
> (ytest))

>
>

truth
predict -1 1

-1 11 1
1 0 8

9.6

= Q. 01

> data”‘dat. kernel=”linaar” , .01 ,

>

truth

1 11 2
1

>
> pch”’19)

>
> (y””‘” cost”‘l(5)
> summary(svmfit)
Call
svm .• ‘” “linear” ,

+05)
?arameters:

SVM-Type:
linea”r

coat: 1e+05
gamma: 0.6

Number of Support :3
( 12)

Number 2
Levels

-1 1
> plot

> (y””. ,
> summary(svmfit)
>

cost

252 •

9.6.2

nel “”,”

> set.seed(1)
> (rnorm (200*2) , ncol=2)
>
> ;o:: [101:150 , ]=x[101:150 ,)-2
> y=c(rep(1.150) .rep(2 ,50))
>

> plot(x ,

1 ,

> train””samplé(200 , 100)
> data”‘dat (train ,J. kernel=”radial”.

cost=1)
> plot(svmfit , )

> aummary {sllmf
Call
s V”m ‘” ., .. ,

gamma ‘” 1 ,
Paramete :r s

BVM-Type:
radial

cost : 1
gamma: 1

Numbar of SupPQrt Vectors: 37
( 17 ’20 )

Number of Classes: 2
Levels

1 2

> svmfit”‘svm(y””.. kernel””’radial”

> , dat [train .])

9.6

> set. selld (1)
> (svm , Y””. , data ,;, dat (train ,J.

1 , 10 ,100 , 1000).
gamma”c (0.5 ,1 , 2 , 3 , 4)))

> summary(tune.out)
Parameter tuning :
– sampling method: crO$$
– best

gamrna
1 2

performance: 0.12
røsults:

cost gamma error dispersion
1 0.5 0.27 0.1160
2 le+OC
3 h+Ol
4 16+02
5 1e+03
6 1e -Ql
7

0.5 0.13
0 , 15

0.5 0.17
0.5 0.21
1. 0 0.25
1. 0 0.13

0.0823
0.0707
0.0823
0.0994
0.1354
0.0823

>
[-train , 1))

9.6.3

> library (ROCR)
> (pred , truth , …) {
+ predob = (pred , truth)
+ (predob.
+ plot{perf ,.,.)}

+ß1Xl +ßzXz +…+

254

> (y,,-,_.

> ,J
. values


> , dat (train , “y”1 , main=”Training )

>
gamma”’50.

> (svmf i t . flex , dat [train .J ,
values=T)) values

> rocp’lot tted , dat

> {predict dat ,J ,
values

> rocplot , dat (-train , “y “) , main””’ Test Data”)
> (predict ,J , decision

9.6.4

> set , seed (1)
> l!:”‘rbind(x , , )
> rep(O , 50))
> x
> frame )
> par (mfro li’ ”’c )
> )

a m a g o s a d a x l a k t a a
Fvl lsp

9.6.5

9.6 .. 255

>; library
> names (Khan)

,
[lJ 63 2308
>
[1] 20 2308 ,
[1] 63

[1J 20

1 2: 3 4
8 23 12 20

> table
1 2: 3 4
3 6 S 5

> frame )
> out”‘svm(y”‘. ,
> summary (out)

$vm (formula ‘” Y ,.._. ., •
‘” 10)

Parameters
SVM-Type: C-elassification

SVM-Kernel: linear
coat: 10

gamma: 0.000433
Number of Support Vectors: 58
(2020117)

Number of

>y $ t a d z d e t i f s t u
4
< :e a31 1b @2a e1 L >

40000

2

30020

1

20300

2

18000

>>t s e Y S M h k

ae zt y
t

‘a td gs ea tt xa $d aw

>

ha$ vae m
>>

40005 80240 20600 18000 e1234 d e r p

• 256

9 , 7

( 1 +3X, -X, + 3X! – X2 > + 3Xl – X2 < +X1 +2Xz +X1 +X l +2X2 ( +X1)2 + (2 _XZ)2 =4" + (2 _XZ)2 +X,)' + (2 +X1 )2 + (2 _XZ)2 1)7 (2 , 2)1 (3 , 8)1 ( ( ( ( vh"-4244231 ( p >
> (500) -0.5
> > 0)

Xl xX2 ,

(

(
(h)

258 •

(

(c)

(

> plot

,

svm “,

(
cost “”

( ‘WJ
(

= 2 ‘”

X2 , ,..

X2 • ••••

clustering)

10.1

260 ‘’

10.2

component ana1ysis ,
X2 • •••

10.2.1

pnnciple

ZI (10.1)

10.2

(

,

Zil (10.2)

:::: -1 (10.3)

Zu

Zu. ••• Z”l

= •.•

,.,

::::0.544″

Z:n,

Za + …+

cþ” =

261

262 •

3

=40

Murde!

UlbanPop
Rape

PC1
0.5358995 1809
0.583 183
0.278 190 9 O. 872 806 2
0.543432

-0.5 0.0 0.5

JersJv

hd… “” …
MMMhgm~jo

ssault

?

1 0 1
Firnt Principal

!fI 10-1

(Rape

-3 -2 2 3

263 •

Murder

10.2

10.2.2

. . .
. . . • .’ . . ‘. … . . • . . .


-1.0

. . .
.‘ • • Æ”

• . .
. . . .

-0.5 0.0 0.5
First princìpal component

264 @

M
(10.5)

=
M

10.2.3

Murder ,
87.73 , 6 945. 16

Scaled Unscaleà
-0.5 0.0 05 -0.5 0.0 0.5 1.0

;1·J jEii
t

1 :
I <::> Ás;.J â æ I
!

8 <:'01 1 I
> states

> names(USArrests)
(1) “Murder” “Assault” “UrbanPop”” Rape”

278 ..

> 2 , mean}
Murder Assault UrbanPop R.ape

7.19 110.16 66.54′ 2 1. 23

apply

> apply(USArreStB , 2 , var)
Murder Assaul t UrbanPop

19.0 6945.2 209.5 87.1

> pr.out”‘prcomp{USArrests ,

prcomp =

> names (pr. out)
[lJ “scale” “x”

>
Murder Assault

7.79 170.76 65 , 64 2 1. 23
> pr ,

Murder A.s sault UrbanPop Rape
4 , 36 83.34 14.47 9.37

pr. out

>
PCl PC2 PC3 PC4

0.418
Assault -0.583 -0.268 -0 , 743
UrbanPop -0 , 278 -0 , 873 -0.378 0.134
Rape -0.543 -0.167 0 , 818 0.089

$

10.4 • 279

,
[1J 60 ‘4

, scale=O)
biplot =

> pr.out$rotation ,
> scale”‘O)

prcomp

>
[1J 1. 675 0. ,995 0.597 0.416

>
> pr.var
[1} 2.480 0.990 0.357 0.173

> pve=pr.varlsum(pr.var)
>’ pve
[lJ 0.6201 0.2474 0.0891 0.0434

> plot(pve , Component” , o :f
Variance Explained” , yliw”‘c )

> (pve) , xlab””’ Principal Component”.
CUlIlulative of ,

cumsum

> a-“‘c(1 , 2.S.-3)
> CU lIlSUlll (a)
[1) 1 3 11 8

• 280

10.5

kmeans

> set.sÐed(2)
> ncol=2)
>
> -4

> (x ,2. “’20)

out $

>
[1J 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 :2 2 2 2 2 1 1 1 1

(30J 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

10.5.1

> cOl=(km.out$clUster+l) , main=”K-Means
Resul ts xlab=””. ylab=”” , pch=20 , cax”‘2)

>
> km.out”‘kmeans(x , 3 , nstart=20)
> km.out
K-meana clustering with 3 clusters of sizes 10 , 23 , 17
Cluster means

[.lJ
1 2.3001545
2 -0.3820397
3 3.7789567

[.2J

-0.08740753

2 2 2 2 3 3 8 1 32 32 32 32 31 12 31 12 32 12 32 12 32 12 82
r t c32 @ v12 g32 a 112 r e82 t e-u1 1[ C

Wïthin sum of squares by cluster
[1] 19.56137 25.74089
(bet lJeen_SS I total < 55 '" 79.3 %) "withinss" Available components: [1J "cluster" "centers" "totss" "size" > plot(x , main=”K-Means
with K”‘3″ , y1ab=”” , pCh”’20 , cex=2)

281

“”

nstart

10.5

ßßed (3)
>
> km.out$tot.withinss
(1) 104.3319
:>
:>
(1) 97.9793

km tot.

out $ wi

seed

hclust

x

10.5.2

:> bC. completechçlust (dist(x) ,

:> avørage
> metbod””’single”)

> par(mfrow=c{1 ,3))
> Linkage ” , xlab”””’. Bub”””’ ,

cax=.9)
:> plot (hc. average. lllain””’ Averaga Linkage”. xlab”””’. sub=””.

cex=.9)
> plot (hc. single , main :,,” Single Linkage”.

2 2 2 2 2 1 Z 1 12 12 12 12 22 12 12 22 12 12
)12 2

12
, e12 1 p12 m c

22
c h12 [

282

21 21 21 11 11 11 1211 1211 1211 1211 1112 1221 1111 1211 1211 1211 1211
>212

>
11

2

‘1211 e

,

812@11 al r12g11 en V-2-11 a6 ·1111 cc <[>[

3 8 3 3 1 $ 1 1 13 18 13 13 23 13 13 14 18 13 13
>
13

4
as ,

eIS l g13 11s ee
-s

c h13

E

> (x)
> (xsc) , mathod=”completa

‘ll’ ith Scalad

dist

hclust

>
>
> (dd > cOlllpl.ete “), .main””’ Complete Linkage

with Gorrelation -Based- Distance”. )

10.6

> library (ISLR)
>
>

> dim(nci .data)
(1) 64 6830

10. 6 • 283

> nci .1abs
(1] “CNS” “CNS” “CNS”
> tabl e.
nci.labs

BREAST CNS COLON K562A -repro K562B
757 i i

LEUKEMIA MCF7A-repro MCF7D-r e.pro MELANOMA NSCLC
6 1 1 8 9

OVARIAN PROSTATE RENAL UNKNOWN
6 , 9

10.6.1

)0 (nci.

Cols=function(vec){
+
+
+ }

)0 par(mfrow=c(1 ,2))
)0 plot col”‘Cols(nci.labs) ,

xlab”””Zl” , ylab””’Z2”)
)0 plo:t (pr. out$x (, c col”‘Cols(nci.labs) ,

> (pr. out)
Importance of

PCl PC’ PC3 PC4 PC5
27.853 21.4814 19.8205 17.0326 15.9718

of Variance 0.114 0.0676 0.0575 0.0425 0.0374
CUlllulative Proportion 0.114 0.1812 0.3185

)0 plot(pr.out)

out $

284

,
aa

,
“,,! …VV J
.-“” ., – . , . . . .


N

. .
‘ . .

. . ..
..!”fto. ..

.. 8
v

……”” .

40 -20 0 20 40 60 -40 -20 0 20
ZI ZI

Q

;-

40

>
> par , 2))
> plot ‘. ylab=”PVE” , Component” ,

col”‘U
> PVE”. xlab=”

[2 r

cumsum (pr. Qut) [3 ,

10 20 30 40 50 60 0 10 20 30 40 50
Principa!

60

N

TO

10.6.2

10.6

> (nci. data)

> par
>
> plot .labs.

xlab=”” , sub”””’ , ylab=”-”}
> labels=nci. labs ,

main=-” Average Linkage” , sub=””. )
> plot(hclust(data.dist , labels=nci.labs.

main=”Single Linkage” I aub=”” .ylab=””)

> hc.out”‘hclust(dist(sd.data})
> hc.

> par(mfrow=c(l , l))
> labels=nci.labs)
> (b=139 , ,,)

> hc.

Call:
hclust (d ‘”

Cluster method compl_ete
Distance euclidean
Number of 64

286 ..

Complete Lìnkage j

A verage Linkage l EEago-

Single Lìnkage

, (2)
> 4 , =20)
>
> table(km.clustars , hc.clusters)

hc.clusters
km. 1 2 3 4

111009
2 0 0 8 0
3 9 0 0 0
420700

> hc. out”‘hclust (dist (pr.
> Clust. on First

Score Vectors “)
> table (hc . nci .labs)

10.7

( ;

[030407| 0.3 0.5 0.8
0.4 0.5 0.45
0.7 0.8 0.45

(

(

X, X,
1 1 4
2 1 3
3 4
4 5 1
5 6 2
6 4

288 ..

(

(
(

(e)

(a)

1 000 x

0

(

(

,289

(

(

(

(

(

290 •

.
edul –

( csv

(