Data Mining and Machine Learning
Types of Multi-Layer Perceptron Peter Jančovič
Slide 1
Data Mining and Machine Learning
Feed-forward Neural Networks Multi-Layer Perceptron – Feed-Forward Neural Network
Input Layer (Input Units)
Artificial neuron
Hidden Layers (Hidden Units)
Slide 2
Output Layer (Output Units) Data Mining and Machine Learning
What can you do with a (D)NN?
Approximate arbitrary non-linear mappings between the inputs and targets
Learn low-dimensional representations of data (Auto-encoder networks)
Learn to allocate data to classes (Classification networks)
Slide 3
Data Mining and Machine Learning
Auto-encoder (D)NNs During training, for each
i = i1,…,i5
input pattern i, t(i) = i
What’s the point?
By including one or more hidden layers with a small number of units (a “bottleneck”) the network learns a low-dimensional representation of the data
t(i) = i = i1,…,i5
Slide 4
Data Mining and Machine Learning
“Classification” Networks
Suppose each pattern belongs to one of N classes
For each input pattern i, let ci be the class of i
Let t(i) be the N dimensional vector with whose cith coordinate is 1 and all other coordinates are 0
i = i1,…,i5
Slide 5
010
i belongs to class 2 Data Mining and Machine Learning
Deep neural networks (DNNs) “Deep” refers to the number of hidden layers
In the past typically only NNs with few (1 or 2) hidden layers were considered:
– Computational considerations
– Difficulty of parameter estimation for multiple
hidden layers
Since ~2000
– Faster computers (in particular GPUs) – Larger training data sets
– Better parameter estimation algorithms
Slide 6
Data Mining and Machine Learning
A “deep” neural network (DNN)
Input Layer
Hidden Layers
Output Layer
o1
oM-1 oM
i1
iN-1 iN
Slide 7
Data Mining and Machine Learning
“Bottleneck” DNN
Input Layer
i1
iN-1 iN
Hidden Layers
“Bottleneck” Layer
Output Layer
o1
oM-1 oM
Slide 8
Data Mining and Machine Learning
THE END ..of lectures..
Slide 9
Data Mining and Machine Learning