Programming part 1
The report should include your explanation on your source code and screenshots of your results.
• Implement three ANNs with the following structure: (The ANNs take as input -dimensional row vectors, and they form a matrix )
• 2 neurons in input layer and 2 neurons in output layer (with softmax activation function).
• 2 neurons in input layer, 4 neurons in hidden layer (without activation function), and 2 neurons in output layer (with softmax activation function).
• 2 neurons in input layer, 4 neurons in hidden layer (with ReLU activation function), and 2 neurons in output layer (with softmax activation function).
• Implement a program to plot the 2-dimentional separating boundary of the ANNs you just implemented.
• Initialize your ANNs in any proper way if you want, show the separating boundaries and discuss why it is generally preferable to use multi-layer perceptron with non-linear activation functions in hidden layers, rather than ANNs with only one layer or those without activation functions in hidden layers.
Programming part 2
Note: to have a better understanding of BP, in this assignment, deep learning frameworks such as TensorFlow, Torch, Caffe, Theano etc. are not allowed to use. However, we suggest utilizing in the next assignment.
Download Iris flower data set , split the data set into training set, validation set and test set. Implement an ANN, BP algorithm (Bonus: mini-batch, Momentum or other variants of BP), and train it. Write a shot report, which includes your explanation on your source code, screenshots of your results and the problems you encountered.
Programming part 3
Note: in this assignment, if possible, we recommend using deep learning frameworks, such as TensorFlow, Torch, Caffe, Theano etc.
• SYSTEM: A Two-Nested-Spirals Problem
Two-Nest-Spirals problem is a well-known classification benchmark problem. It contains two nested spirals, ‘o’ and ‘+’, as shown in figure. The task is to separate the two nested spirals.
• Purpose: Separating the Two Classes Using Neural Network Classifier
• Write a program to generate the training data set.
• Design several neural network classifiers.
• Define a loss function.
• Train the neural networks.
• Discuss how you have tried to solve the local minimum problem and overfitting problem;
• Show the error curves for training data.
• After training the neural network, try to show in a figure (decision boundary) how well the trained neural network can separate the two spirals. (generalization ability)