Deep Learning
Logistic Regression with a NN mindset Neural Networks
Backpropagation Improving your NN
I
ZED was a
a oza
was aCD BED AED o za
i
o bC
z
z
aCD o
zen
we’T a b
of
WED WED WED be be BED
Optimizing
ID
AED
r
a
a
a
y
ri
wa b wa b
WE b
Lossfor Jtg y f IL El
Lil Cy logga t c y log l I l
23
fll3
was was 221 2wee
BUT be a 23 Tba
2
F
2WED BackwardPropagation
3
2
ae
2 WED za
E
3 Een333as 1
I
ya log I t G y I y few log
3
logCi ya D
o w a t be a GIT
T
dw
Gy y
log
l
Ia
ly CDaCH
a
log n In O’Ca OCHA
oca
22
Ife
2L
fLj
z
a
2fogwE
2W
42 z2watbCD
tact ACN
2 WE a
ftw Ly
acid
1am
Ja
Gl
G yay
act
SE
2W
a
gil
w
a
gci
T
a
a
32
T
Cy
In
Hae 3 3
2L 2297 22733
of
35
2o w a tb a tach 2 a D
T
alot
tae
1 Gl 2,1 1,3
WENT a l acid ah ya acDT eetement 44 CH 1,3
wise
product
a
IT
3k
¦ÌI
In
Improving your Activation functions
ReLU IIqzs.org replaces
NN
a Cta’s amG aes
sigmoid OG z
RealG
Oz
qtzj0o
O2
tanh2 e
old toad ee z
tanhGa l tanh272 Initialization methods
input
Normalizing
n XT i
I
Eq
Cat
AIna oInE
5
X2X
X
Xy
NOOO
Vanishing
Exploding gradients
ii
WEactD wco
I
o
was wa D L
D
by initializing cuts close to 1
Assume initialization 6
0 Activation fn id 21 2
ACL 23 X
C
wa i WD
w
cos
avoid
Los o
Example wah
I hearon a
yo Idt
wi
a off
ZWXt wdXd
small we
Xz
large d we
hp random.randnefshape for sigmoid
Ld npsqrt
fg
for ReLU 2 Instead of 1
Her Initialization was
I
backward
prop
Descent
Stochastic Gradient Descent MiniBatch gradient Descent
Xavier
Initialization
wa n fT for tanh
F5 not na
Tforqroopd
Gradient
Ntv
Ismauer
Momentum
e
potato
larger
3kW
Cfp
ww
2W
421
v wwdV
current gradient
last gradent 2nd
BA p
pic p i