ESE545-Project2
February 25, 2020
Due: March 22, 2020 at 11:59pm
• You are required to solve the problems using Python. You are allowed to use only python standard libraries, numpy, pandas, scipy and Pytorch.
• You are allowed to work in groups up to 3 members. You should submit your code with a brief report containing responses to each part.
• You should submit one code per group but each individual should submit their own report.
• Submit the Report and Code separately on Gradescope. For code, upload the zip file.
(5 points) Problem 1 For this assignment, we’ll be using SVM with Pega- sos and Adagrad optimizers and, Neural Networks to classify images from the Fashion MNIST dataset. Each example is a 28×28 grayscale image.
1. Downloadandloadtrain.npy,trainlabels.npyandtest.npy,testlabels.npy files from canvas. These files contain the training and test images along with their labels.
2. Discard all the images other than those associated with labels {2,5,7}. Problem 2 and Problem 3 are binary classification tasks involving labels {2, 5}. Problem 5 and Problem 6 are multi-class classification tasks involving labels {2, 5, 7}
3. Visualize a random sample of 10 images along with the labels.
(15 points) Problem 2. Using Pegasos, train a classifier on the training im- ages. Plot the training error and test error vs. the number of iterations and include it in your Report. Make sure to justify the selection of any parameters (with evidence) in your Report.
(20 points) Problem 3. Using AdaGrad, train a classifier on the training im- ages. Plot the training error and test error vs. the number of iterations in the same plot as above. Make sure to justify the selection of any parameters(with
1
evidence) in your Report.
Compare this model with the one from Problem 2 and choose the model which you think performs better. Briefly describe why and include your findings in the report.
(10 points) Problem 4. Briefly talk about the scenarios in which AdaGrad doesn’t perform well and propose an algorithm to rectify some of those issues. Write a pseudo code for the proposed algorithm and discuss when the proposed algorithm performs better than AdaGrad
(20 points) Problem 5. For this problem, you’ll work on the dataset cor- responding to three labels mentioned in Problem 1 and implement a multi class classifier. Implement your classifier using your best SVM model from the above problems and One vs One approach. Train the classifiers on the training images and evaluate your final model on the test dataset. Plot the training error and test error with the number of iterations and report the final accuracy on the test dataset.
(20 points) Problem 6. You will now train a convolutional neural net (CNN) for the multi class classification using the same dataset from Problem 5. Use the architecture given in the table below to design your CNN and implement it using Pytorch. Plot the training and test errors and report the final accuracy on the test dataset.
Layers
Hyper parameters
Convolution 1
Kernel size = (3, 3, 16), Padding = (1,1). Followed by ReLU.
Pooling 1
Max pool operation. Kernel size = (2, 2). Stride = 1. Padding = 0.
Convolution 2
Kernel size = (3, 3, 16), Padding = (1, 1). Followed by ReLU.
Pooling 2
Max pool operation. Kernel size = (2, 2). Stride = 1. Padding = 0.
Fully Connected 1
Output channels = 64. Followed by ReLU.
Fully Connected 2
Output channels = 3. Followed by Softmax.
(10 points) Problem 7. Compare the performance (in terms of training ac- curacy, test accuracy and convergence rate) of your two multi class classifiers from Problem 5 and Problem 6. Which classifier did best? Why do you think that is the case? Report your findings in the writeup.
Additional Resources:
1. An Intuitive Explanation of Convolutional Neural Networks
2. Everything you need to know about Neural Networks and Backpropaga- tion
2