For this assignment, you will carry out a binary classification task, and write a report on this. Please read this brief carefully and also the Marking Criteria and Requirements below.
The data come from photos, and your task is to come up with a machine learning method for classifying the photos according to whether or not they are 'memorable'. The data you are given for each photo consists of 4608 features. 4096 of these were extracted from a deep Convolutional Neural Network (CNN) [1], and the remaining 512 are gist features [2]. (You are given all these features as a 1-dimensional array, so you will not be performing any feature extraction on raw images.)
There are two files of training data. The first contains 600 samples with all the data present. The second contains 2800 samples, which have some missing data, as indicated by a NaN (not a number). The training data have class labels, 1 for memorable, and 0 for not memorable. In addition, there is also a confidence label for each sample. The class labels were assigned based on decisions from 3 people viewing the photos. When they all agreed, the class label could be considered certain, and a confidence of 1 was written down. If they didn't all agree, then the classification decided on by the majority was assigned, but with a confidence of only 0.66.
There is one file of test data, containing 2000 samples. You must obtain predictions for the class labels of these. (Note that, as with the second training set, the samples in the test data set contain missing features.)
Your job is to obtain the best predictions you can, and to justify your methods. You should reason for which classifier or combination of classifiers you use, how you do model selection (training-validation split or cross validation), and how you handle the specific issues with these data (large number of features, missing data, the presence of confidence labels for the classes of the training data). We value creative approaches!
You may make use of any classifier, such as: single-layer perceptron, multi-layer perceptron, SVM, random forest, logistic regression. You are not required to code classifiers from scratch, and you can use any machine learning toolbox you like, such as scikit-learn (
https://scikit-learn.org/stable/(Links to an external site.)
).