COMP3308/3608 Artificial Intelligence
Week 9 Tutorial exercises Multilayer Neural Networks 2. Deep Learning.
Exercise 1. Backpropagation (Homework)
a) Show that the derivative of the hyperbolic tangent sigmoid is 1-a2.
Copyright By PowCoder代写 加微信 powcoder
b) Write the weight change rule for this function, using the result from a). See slide 26 from the lecture on backpropagation.
for an output neuron i: wji =
for a hidden neuron j: wkj =
Exercise 2.
a) Given is a 2-layer feed-forward neural network with 10 input units, 5 hidden units and 3 output units. How many weights does it contain? Don’t forget to include the bias weights.
b) Is the backpropagation algorithm guaranteed to achieve 100% correct classification for any linearly- separable set of training examples, given a sufficiently small learning rate? Explain briefly.
Exercise 3. Cybenko’s theorem (slide 39) states that any continuous function can be approximated by a backpropagation network with 1 hidden layer. Why do we use networks with more than 1 hidden layer?
Exercise 4. Using Weka
1. Load the iris data (iris.arff). Choose 10-fold cross validation. Run the Naïve Bayes and Multilayer percepton (trained with the backpropagation algorithm) classifiers with the default parameters and compare their performance. Note that multilayer perceptron in Weka is under “Functions”.
Which classifier was more accurate? Which one was faster to train?
2. Select “MultilayerPerceptron” as a classifier. In the pane “Choose” (where “MultilayerPerceptron” should appear click to open the options window. If you click “More” you can learn what each of them means. What does “a” in the number of hidden layers mean? Experiment with 1 hidden layer and different number of neurons in it. As you can see the classification performance of the backpropagation network is very sensitive to the architecture. Choose “GUI=true” from the window with the options to visualize the different networks.
COMP3308/3608 Artificial Intelligence, s1 2022
a=en −e−n en +e−n
COMP3308/3608 Artificial Intelligence, s1 2022
Exercises using Matlab and Matlab’s Neural Network demos
Exercise 5. Convergence with different learning rates
In Matlab, type nnd12sd2 to run the demo.
Try different starting positions and different learning rates. What is the influence of the learning rate? What
happens if the learning rate is too big or too small?
Exercise 6. Backpropagation with momentum
In Matlab, type nnd12mo to run the demo.
Experiment with different values for the momentum and learning rate. What is the advantage of using a
What is the advantage of using a momentum?
Exercise 7. Generalization
In Matlab, type nnd11gn or select “generalization” from Demos -> Toolboxes -> Neural Networks (upper left window)
Try a simple function (i.e. choose a small difficulty index) with too many hidden neurons. What is the reason for the poor generalization?
Exercise 8. Convolution in deep NNs
Follow the convolution example on this website to see how the convolved features are computed:
http://deeplearning.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/
Exercise 9. Backpropagation example
Follow the backpropagation walkthrough example prepared by Josh Stretton and available from Canvas; it is an extended version of the example from the first lecture on slides 33-37. The goal is to understand the forward and backward passes of the backpropagation algorithm and how the weights are updated.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com