Assignment 4
• In the form. of a competition or challenge
• In pairs or alone (pairs will receive the same mark)
• Modify/improve abaseline model to perform one of four tasks (see next slides)
• Submit (modified) python notebook and a two-page PDF report
Competition 1: Facial Recognition
• 28,709 images, 7 classes
• Size: 48 times 48
Competition 2: Flower Dataset
• 4317 images, 5 classes
• Size: Different, around 320*240
Competition 3: Road Segmentation
• 150 training images, 19 classes
• Around 800 times 256
Competition 4: Animal dataset
• 151 Classes, 4702 images
• Size: 224*224
Baseline
For each challenge we provide baseline code that includes the following features:
• Loading and Analysing the dataset using torchvision.
• Possibly providing some augmentations (on loading)
• Defining a convolutional neural network (baseline model).
• Defining/choosing a loss function for the model learning.
• Training the network on the training data.
• Testing the trained network on the testing data.
• Generating prediction for selected or random test image(s).
Improvements
You could consider the following improvements:
• Change of training hyper-parameters: Learning Rate, Optimizer, Batch-size, Number of Max Epochs, and Drop-out
• Use of a new/different loss function.
• Additional/better data augmentation
• Architectural Changes: Batch Normalization, More layers, Residual layers
• Pre-training on a large dataset (aform of “transfer learning”)
Assessment
Final mark (out of 50) = acc_mark + efficiency mark + report mark
Acc_mark 10:
• We will rank all the submission results based on their test accuracy. Zero improvement over the baseline yields 0 marks. Maximum improvement over the baseline will yield 10 marks. There will be a sliding scale applied in between.
Efficiency mark 10:
• Efficiency considers not only the accuracy, but the computational cost of running the model (flops:
https://en.wikipedia.org/wiki/FLOPS). Efficiency for our purposes is defined to be the ratio of accuracy (in %) to Gflops. Please report the computational cost for your final model and include the efficiency calculation in your report. Maximum improvement over the baseline will yield 10 marks. Zero improvement over the baseline yields zero marks, with a sliding scale in between.
Report mark 30:
• Your report should comprise:
1. An introduction showing your understanding of the task and of the baseline model: [10 marks]
2. A description of how you have modified aspects of the system to improve performance. [10 marks]
3. Explanation of the methods for reducing the computational cost and/or improve the trade-off between accuracy and cost: [5 marks]
4. Limitations/Conclusions: [5 marks]