Introduction to Machine Learning Deep Learning Redux
Intro to Unsupervised Learning
Prof. Neural Networks aka ConvNets or CNNs
• class of Neural Networks used primarily in vision
Copyright By PowCoder代写 加微信 powcoder
– image recognition and classification
– identifying faces, objects and traffic signs
CNNs exploit the inherent structure in images
• has the effect of reducing
#parameters
CNNs: Recap
• Convolution (CONV)
• Rectified Linear Unit (ReLU)
• Pooling (POOL)
• Fully Connected Layer (FC)
image source: http://cs231n.github.io/convolutional-networks/
CONV Layer
specify snifengm
y height width are
set as hyperparameter
t of theimage T part filter
Note: the connectivity is local in space, but full along the input depth. Figure from : Fundamentals of Deep Learning by ,
each filter produces a new feature map
6 filters: each 3x3x3
output of depth 6
heightI width
t size stridelength no pad
Figure from : Fundamentals of Deep Learning by ,
What do filters learn?
Edge detection:
difference between a pixel and its neighbors
“averages” each pixel with its neighbors
ReLU layer
f(z) = max(0,z)
activation map
images from: http://cs231n.github.io/convolutional-networks/; https://medium.com/data-science-group-iitr/building-a-convolutional-neural-network-in-python-with-tensorflow-d251c3ca8117
CNNs: Pooling layer
Idea: reduce dimensionality of feature map while retaining significant information. as a consequence, control overfitting
CONV; ReLU
In max pooling, the pooling operation is applied separately to each feature map so the depth dimension remains unchanged
example from: http://cs231n.github.io/convolutional-networks/#pool
Fully Connected Layer
CONV; ReLU; POOL
Softmax Layer (Example)
É’velabel isclass2
k class classification
“$ !! = #”!
Fee F j.fi
!# predictedlabel aftersoftmax !$
img src: https://www.andreaperlato.com/img/fullyconnected.png
All ! are non-negative and Σ%
Example CNN architecture
https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html
• Convolution (CONV)
• Rectified Linear Unit (ReLU)
• Pooling (POOL)
• Fully Connected Layer (FC)
Project 2: Karl’s Convolutional Kennel
https://www.bing.com/visualsearch/Microsoft/WhatDog
Project 2 Overview
• Data Preprocessing
– Given Architecture – Train
– Evaluate
• Visualizing what the CNN has learned
• Transfer Learning & Data Augmentation
• Challenge
For more see Project 2 QS
Lakkaraju, Kamar, Caruana, and Horvitz (2017). Identifying unknown unknowns in the open world: representations and policies for guided exploration
The importance of Interpretability
Looking ahead, we still see facial recognition technology as a powerful tool, for example, for people needing to verify their identity, or to prevent fraud and impersonation. … We will continue working on these technologies and engaging outside experts.
Guess the Genre
• Music Clip 1 • Music Clip 2
Lesson: Context Matters
Recurrent Neural Network (RNN)
“&’$ “&’# “&
#&’$ #&’# #&
parameters are shared “across time”
“& = % &)”&’# + &*#& + offset “&’# = %(&)”&’$ + &*#&’# + offset) “&’$ = %(&)”&'( + &*#&’$ + offset)
# #&'( recurrent neuron
Applications:
• NLP tasks including translation, captioning etc.
• financial applications including stock market prediction
Recurrent Neural Network (RNN)
recurrent neuron
can have multiple recurrent neurons in a layer
RNN: example
ReLU WaittWyy ”
initialization “+ O &#* = 3.5
Wy 2 and a sequence of inputs
offset = -6
#̅( = 2 −2
Determine “(()
·x+ 0 =1 ·x+ 0 = 1
·x+ 0 =1 ·x+ 0 = 1
Figure 2: The e↵ect of slack when examples are still linearly separable
The other advantage of the slack variables is that we can now solve problems that are no longer linearly separable. This is illustrated in Figure 2 with di↵erent values of
Story so far…
·x+ 0 =1 ·x+ 0 = 1 x+ 0=0
=0.1 (C=10)
=100 (C =0.01)
·x+ 0 = 1
Figure 3: The e↵ect of slack when examples are no longer linearly separable
Our current formulation of the support vector machines can be written in a familiar form. Indeed, the primal version is exactly the same as minimizing
21Xn (t) (t)
2k✓k +n Lossh y (✓·x +✓0) (8)
with respect to ✓ and ✓0. Here Lossh(z) = max{1 z, 0} is the hinge loss. To see that
the regularization parameter .
Where do labels come from
Domain experts
MTurk (crowdsourcing)
other sources… generate automatically e.g., snorkel.org
Caveat: labels can be noisy…
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
. Northcutt, , NeurIPS 2021
Machine Learning @
Unsupervised Learning
Sometimes labels are not available
Sn ={x ̄(i),y(i)}ni=1 x ̄2X
Welcome to unsupervised learning! Recommender systems
Generative Models
Clustering
Unlabeled DataSets
What can we do with unlabeled data?
Visualize it
What can we do with high-dim unlabeled data?
e.g., 3D to 2D embedding
Visualize it
find a low dimensional embedding (then maybe visualize)
Machine Learning @
Froogle unsupervised learning
Applications
”Now, when there are multiple stories related to your search, we’ll also organize the results by story so it’s easier to understand what’s most relevant and
Other Examples: informed decision on which
you can make a more specific articles to explore. ”
• finding similar homes for sale
• grouping patients by symptoms
• mining customer purchase patterns
• grouping search results according to topic
• group emails
• image processing à regions of image segmentation…
Unsupervised DataSets
Goal: Learn a representation of the data, uncover useful structure, identify groups/clusters of similar examples
unsupervised learning clustering
image courtesy Bishop 2006
Example: Image segmentation using k-means clustering
Pixels a, c
are in the same cluster Pixels a, b
are in different clusters
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com