Assignment #2: CIFAR-10 Image Classifica!on using Fully Connected Neural Network Solution

$30.00 $26.40

Description

In this assignment, you are going to implement a one hidden layer fully connected neural network using Python from the given skeleton code mlp_skeleton.py lu18330v2mj7p tmp beb38aaff93e1d5 on Canvas (find in the Files tab). This skeleton code forces you to write linear transformation, ReLU, sigmoid cross-entropy layers as separate classes. You can add to the skeleton code as long as you follow its class structure. Given N training examples in 2

categories lu18330v2mj7p tmp 75d52610b0eab74d lu18330v2mj7p tmp ae52bc78246035e1 lu18330v2mj7p tmp 8e2e3edba3a5477 lu18330v2mj7p tmp 4c5d98b8777ea38d lu18330v2mj7p tmp ff87b2c448782ab7 lu18330v2mj7p tmp fcf021808a039bd lu18330v2mj7p tmp ae52bc78246035e1 lu18330v2mj7p tmp 1cbd91036ddc0f0b lu18330v2mj7p tmp 27c4f3883aa385ad lu18330v2mj7p tmp 1b296c51c594efa9 lu18330v2mj7p tmp 54184411c98b2a07 , your code should implement backpropagation using the cross-entropy loss (see Assignment 1 for the formula) on top of a sigmoid layer: (e.g.

lu18330v2mj7p tmp 6a26019a945f8c7b lu18330v2mj7p tmp 6a26019a945f8c7b lu18330v2mj7p tmp 6a26019a945f8c7b lu18330v2mj7p tmp 6a26019a945f8c7b lu18330v2mj7p tmp 883c40bf0c02af84 lu18330v2mj7p tmp b41447bd2ea3f33f lu18330v2mj7p tmp 78c3ab5577f4847a lu18330v2mj7p tmp b41447bd2ea3f33f

), where you should train for an output

lu18330v2mj7p tmp f287a17d9caa3a6b lu18330v2mj7p tmp aa6f6bede27de1d9

lu18330v2mj7p tmp d049daf6eec7feec lu18330v2mj7p tmp 17f5d5e5ce2b483c lu18330v2mj7p tmp 663899e0381c5c6b lu18330v2mj7p tmp 454c553934724006 lu18330v2mj7p tmp 5df69ece289a92c3 lu18330v2mj7p tmp 54845ec43b929525 lu18330v2mj7p tmp e81ebb82e7c1e6b9 lu18330v2mj7p tmp dc8b0775a6ec1c0c lu18330v2mj7p tmp 6c6a3ecd2aa9ff58 lu18330v2mj7p tmp ff6a644206b718d1 lu18330v2mj7p tmp e59db4b48297f457 . lu18330v2mj7p tmp a11fbc896260545a lu18330v2mj7p tmp 118fb08af33ae029 lu18330v2mj7p tmp d499b1deb6e65a39 lu18330v2mj7p tmp 54f2305f6b740acf lu18330v2mj7p tmp 2d0796bfeca636b6 lu18330v2mj7p tmp 68d29278c568f211 lu18330v2mj7p tmp 118fb08af33ae029 lu18330v2mj7p tmp 4ef62b079e241bc7 is the ReLU activation function (note Assignment #1

lu18330v2mj7p tmp e7d64b9d89184974 lu18330v2mj7p tmp af32bd8968986553 lu18330v2mj7p tmp 6a26019a945f8c7b

used a sigmoid activation but here it’s ReLU), lu18330v2mj7p tmp 6acc2073c399cff6 is a matrix with the number of rows equal to the number of hidden units, and the number of columns equal to the input dimensionality.

Finish the above project and write a report (in pdf) with following questions:

Please put the report(in pdf) and the source code into a same zip file, “firstname_lastname_hw2.zip”. Submit this zip file on Canvas. You have to make sure your code could run and produce reasonable results!

  1. Write a function that evaluates the trained network (5 points), as well as computes all the subgradients of lu18330v2mj7p tmp 40987512ca0a0ea0 and lu18330v2mj7p tmp 38c86f88b2ef41c using backpropagation (5 points).

  1. Write a function that performs stochastic mini-batch gradient descent training (5 points). You may use the deterministic approach of permuting the sequence of the data. Use the momentum approach described in the course slides.

  1. Train the network on the attached 2-class dataset extracted from CIFAR-10: (data can be found in the cifar-2class-py2.zip file on Canvas.). The data has 10,000 training examples in 3072 dimensions and 2,000 testing examples. For this assignment, just treat each dimension as uncorrelated to each other. Train on all the training examples, tune your parameters (number of hidden units, learning rate, mini-batch size, momentum) until you reach a good performance on the testing set. What accuracy can you achieve? (20 points based on the report).

  1. Training Monitoring: For each epoch in training, your function should evaluate the training objective, testing objective, training misclassification error rate (error is 1 for each example if misclassifies, 0 if correct), testing misclassification error rate (5 points).

  1. Tuning Parameters: please create three figures with following requirements. Save them into jpg format: i) test accuracy with different number of batch size

ii)test accuracy with different learning rate

iii) test accuracy with different number of hidden units

  1. Discussion about the performance of your neural network.