Deep Learning Supervised learning
non linear models
non linear in a
before
w hoCn OT n
lenear 0 4
a ElRd yCK
kernel
method
ho n dataset Caiy
lostHoss for
eg
Inco y how’D costfn forentire dataset
ho te IRD squared loss
IR
2
10
JO
optimization objective
In Fi
mm
TCO gradient descent
0
0i o d17310 Stochastic Gradient Descent
SGD
sample j from El ng randomly
fori Ltoniter
0 O dDJ’tO
minibatch
SGD
idea Computing B gradients DJ co
DJ B co together is faster than sequential computation
for
i
niter
I to
Sample B examples Ji j from El n
II
a
o
g
DJ O
un
optimum
hole
t
local
remaining questions
How to define
how to
Logistic Regression
computegradient Neural Networks
computational power data available
algorithms
Logistic Regression
IIIs f 121288 1
1 12288
goat Find 16T I
presenceof cat
cats in images I
0 absenceof cat
d 64643 12,288
ff o T
i U
Cn
w
wb wut b
I
weights yblas
initials e w b
Find optimal
to predict
a g log tg
Use yn L
o
ylogya
w
d
71
2W
bb
221
parameters
2b c dt I
Note fixedtypo in the love notes
architecture
Fond cat lion iguana in images
neuron
model
lenear
12288 1 activation
parameters
Goal2
a.cl
f
yi
a3etbf’DEII
E
oCw
ai ocwi.name’D
ya age
y
ocw.ci ntbzc urchin layer
squarebrackets CD
lager
neuron
subscripts
ages e
Identify
1st layer
2nd neuron in 1st layer
3 del images c labels
param
Io
Edition
Egan
Goal 3.0 1 add constraint
animal in
ya I 3q ZED
Oyi
2,03
III
unique
Lmgage
zi’s
ET
ftp.t
eziyE eae e25 ezE’T
Notefixed.atpoin j
y Ib
yn
o
w
softmax
Malecclass network
parameters
Ig
added
Loss function
1419kdogIne 3
tyre logAyid
n
Binary
Cross Entropy Loss Function
Q
LCE Gklog yk
Predict age of cat instead of presence of cat
Several
Change
neurons to activation
predict different ages for
If
Rectified Linear Unit
fat
no if ReLU
Modified
Kya yl
Ilya y 1122
Loss fu
Neural Networks
ai
at N
3d13
773
12
i
23 2
1
32 inputlayer output layer
hidden layer House priceprediction
9 0
wealth
Propagation equations
3yd dxL 31 y 3yd dish
bedroom
t
Zepcode
familysize
0K 4
Size
price
quaky
zxn
31 IT w n t b’t 31 ACD or za
2 2 1 Z acts t be
29 we X BED Broadcasting
B b be n copies
I
IF
batch of examples
21A oZED 12 21
be
Zwa 297
11 11ao
x
fun
an
p example
capital
Optimize w
w WED be BED b
Define
loss
f