程序代写 CS 189 (CDSS offering)

Lecture 27: Neural networks (1) CS 189 (CDSS offering)
2022/04/04

Today’s lecture

Today marks our descent into deep learning: machine learning combined with deep neural network models
Your remaining homework assignments will also revolve around deep learning CS 182 is a good course for diving much deeper into deep learning
Much of modern machine learning revolves around deep learning, thus we will
spend much of our remaining time on this topic

Back to featurization
We are still thinking about the question from the end of lecture 1: machine learning is
(often) about predicting y from x, but what is x?
We have now seen techniques for lifting the dimensionality of x (via kernelization) as If I, the practitioner, pick an appropriate featurization, I can make my models work
But what if I don’t know how to pick a good featurization?
well as reducing it (PCA, t-SNE, …)
much better (e.g., an appropriate kernelization combined with linear models)
What if, instead, I could learn a good featurization? E.g., learn a transformation from x 3
to features such that a linear model works well on top of those features?

“Feature based” learning
Learning on top of the features can be simple Coming up with good features is very hard!
Before deep learning, dealing with complex inputs such as images required a
great deal of feature engineering (the techniques we’ve seen so far didn’t cut it)
Kind of a compromise solution — don’t hand program the model, but do hand
program the features

From feature based learning to deep learning
input fixed feature hand programmed learned label
learned feature extractor
features classifier

What is deep learning?
Machine learning with multiple layers of learned representations The function that represents the transformation from input to output
is a deep neural network
The parameters for every layer are usually (but not always) trained
with respect to the overall objective/loss (e.g., accuracy)
This is sometimes referred to as end-to-end learning

Recall: logistic regression The “linear neural network”
• Given x ∈ Rd, define fθ(x) = θ⊤x, where θ is a d × K matrix
• Then, for class c ∈ {0,…, K − 1}, we have pθ(y = c | x) = softmax( fθ(x))c
Remember: softmax( fθ(x))c =
exp fθ(x)c ∑K−1 exp fθ(x)i
• Loss function: l(θ; x, y) = − log pθ(y | x)

A diagram for logistic regression
“linear layer”
linear layer
Often, we will simplify this diagram:
Omit the θ box, the parameters Omit the layer box entirely!
cross-ent loss
are implicit in the diagram
we’re drawing “just the model”
Denote it with just the arrow
Omit the loss box at the end, if

Neural networks: attempt #1
Our drawing of logistic regression suggests that it is a “single layer model”
Are neural networks just more of these layers stacked on top of each other?
What’s the issue with this?
Composing linear transformations together is still linear!
x z x z(1) z(2)
linear layer softmax linear layer linear layer … softmax 9

Making neural networks nonlinear
Also called activation functions
Basically always element wise functions on the linear layer output
Examples: tanh(z), sigmoid(z) = 1 , ReLU(z) = max{0, z}
One of the main things that makes neural networks great is that they can How? The canonical answer: add nonlinearities after every linear layer
represent complex non linear functions
exp{−z} + 1 10

Neural networks: attempt #2
x z(1) a(1) z(2) a(2)
linear layer
nonlinear  layer
nonlinearity linear layer
… a(L) z
linear layer
nonlinearity
nonlinear  layer
… nonlinear  layer

What function is this?
θ represents all our parameters, e.g., [W(1), b(1), …, W(L), b(L), Wfinal, bfinal] If our neural network has parameters θ and L hidden layers, then it represents
the function fθ(x) = softmax(Afinal(σ(A(L)(…σ(A(1)(x))…))))
What can this function represent? Turns out, a lot
σ is the nonlinearity / activation function Ai(v) = Wiv + bi is the i-th linear layer

Visualizing neural network functions

Visualizing neural network functions
https://playground.tensorflow.org/

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts