Outline
Naive Bayes Laplacesmoothing
Event models kernel methods
Recap
Fairy
f
dy
g
n examples
D I notspam spam
nonspam Spann
Maximum Likelihood Estimates
Xj I word appears in email Generative Model
pexly ply play d pasty
Parameters Ply _1 PIX_Ily_O j1y o
01g PCxj Hy i loly L
Cly jlyo
IE I n
yen13 T
PCnty DPCy D
pcnly DPly D PCnly D.PT
Ly 0 o
I yaif
weird
0j1y o
Prediction
Ply L In
ly I
j
COULD
1273
PX1273 1 91 PXm3 1 90
O_
yL Of
yo Pln Ig
012731y 012731yo
PCaly D ply4 In
I
I
41273 ly D.pcy D pcnly D.my o
TO
PCaty D pcy DPCnly
O
0127314 0
Laplace smoothing
won
Arizona 0
Caltech O Okhohama
Wakeforest
OSU 0
O
PCX D
Is_ I Ist Os12
01I 0426
Laplace smoothing
More generally XCIK
Iq
CE
ElK L 400 feet
Estimate PCx D
jIyo EE Xjt y03 1
y
11
i examples
y soy j
12 words
Ni
400 800 XL23
size
71200
4
US bernoulli
800 1200
pcnlyj g.IT Pln Ig multinomial
fL 1
a L2 cfm soo
bank 1600 beneficiary
0,13
account bank
Xi C bank
New
Jordi
I 10,000
representation
I
X
di
E
length of
email
i
Bernoulli event model Multinomial event model
Multivariate
paly
plug Em It plagly
Parameters
log P y D
o P Nj Kl geo
is Kthword in dictionary of y 0
pen y
pcy
01k1g
chance that word j
4k1g 4 E4my o
P
IHI ya03
age Kly I
D
X
x Ky di
yakoy
Laplace smoothing
denom
n
numerator
1
t 1dictionary
10,000
rare words to UNK spam detection
mortgage mortgage
uNk
spoofed headers fetch URL
Methods
map
kernel
OT n n
x
x
to
Xt x
h n QIN
Oz N’t Oz n’t O n too
IR
IN
ng
hole Oo 01,02 9 aI
leg
OTcha
is linear in 0 0cm
hood
ya Cnc yes
Collat Iuh D yy
Kimy 73
oTdCniD5
3
LMS
on new
dataset
mom ta En Cy Gradient Descent
Loop
0 0
4 Eh ya IRP
features attributes
features
Otto WD lo na
g Terminology
IRD
attributes
x
Qln
CIRP
ERP O hp feature map
How do we di Xd
handle large values p
of
p
Xi
d
foe
Xgxi
x
QIN
X3 i
tiX X 3
d
u Es p
Xd high
dimensional
It d t d’t d per109
Ocd Runtime for GD depends on p 0Cup
D 103
key observation
If 0 is initialized at 0
then at any time
0 can be written as
TI Bi olCnn
for some
13 Pn CIRN
EIR
C IRP
of
this observation
Proof
By induction over
time
theration
00 Ot4cm
Base Case
O
Assume
Bi at some time
to
E
pilot
Next
iteration
Oi Otxqy Otoun 4cal
y Oto na loca
scalar
new pie update
IE pi 12
new
algo
p