Have attached all the data set and task instruction
Microsoft Word - Assessment-4-T2-2022.docx : . Assessment 4 Submission Instructions a) Submit your solution codes into a notebook file with “.ipynb” extension. Write discussions and explanations including outputs and figures into a separate file and submit as a PDF file. b) Submission other than the above-mentioned file formats will not be assessed and given zero for the entire submission. c) Insert your Python code responses into the cell of your submitted “.ipynb” file followed by the question i.e., copy the question by adding a cell before the solution cell. If you need multiple cells for better presentation of the code, add question only before the first solution cell. d) Your submitted code should be executable. If your code does not generate the submitted solution, then you will get zero for that part of the marks. e) Answers must be relevant and precise. f) No hard coding is allowed. Avoid using specific value that can be calculated from the data provided. g) Use all the topics covered in the unit for answering this assignment. h) Submit your assignment after running each cell individually. ________________________________________________________________________________ Questions ________________________________________________________________________________ Background In this project you are given a dataset and an article that uses this dataset. The authors have developed eight ML models for cyber security intrusion detection and compared their performance. You must read the article to understand the problem, the dataset, and the methodology to complete the following tasks. Dataset NSL-KDD dataset has been developed to solve problems in KDD 99 challenge. It does not contain unnecessary and repetitive records according to the original KDD 99 data set. A detailed description of the dataset can be found in the Dataset section of the provided article. You can also use other sources for better understanding the dataset and answer questions. Please use the provided dataset “Intrusion_detection_NSL_KDD.csv” for answering the questions and DO NOT DOWNLOAD AND USE dataset from any other sources. Use the file “FieldNames.pdf” for pre-processing the independent and target variables BEFORE ANSWERING any questions. Tasks: 1. Read the article and reproduce the results (Accuracy, Precision, Recall, F-Measure) for NSL-KDD dataset : . using following classification methods: (10 marks) o SVM Linear o SVM Quadratic o SVM Cubic o KNN Fine o KNN Medium o KNN Cubic o TREE Fine o TREE Medium These results can be found in Table 4 of the manuscript and should be used for comparison purposes, if required. Write a report summarising the dataset, used ML methods, experiment protocol and results including variations, if any. During reproducing the results: i) you should use the same set of features used by the authors. ii) you should use the same classifier with exact parameter values. iii) you should use the same training/test splitting approach as used by the authors. iv) you should use the same pre/post processing, if any, used by the authors. [N.B. Definition of used algorithm can be found in this link: https://au.mathworks.com/help/stats/choose- aclassifier.html. However, your submission must be in python not in Matlab.] N.B. (i) If you find any issue in reproducing results due to incomplete description of model in the provided article then make your own assumption and explain the reason. If your justification is correct then your solution will be considered correct and assessed accordingly. (ii) If you find some subtle variations in results due to implementation differences of methods used in the study i.e., packages and modules in Python vs Matlab implementation, then appropriate explanation of them will be considered during evaluation of your submission. (iii) Similarly, variation in results due to randomness of data splitting will also be considered during evaluation based on your explanation. (iii) Obtained marks will be proportional to the number of ML methods that you will report in your submission with correctly reproduced results. (iv) Make sure your submitted Python code segment generates the reported results, otherwise you will receive zero marks for this task. Marking criteria: i) Unsatisfactory (x<4): tried="" to="" implement="" the="" methods="" but="" unable="" to="" follow="" the="" approach="" presented="" in="" the="" article.="" variation="" of="" marks="" in="" this="" group="" will="" depend="" on="" the="" quality="" of="" report.="" ii)="" fair=""><><5): appropriately="" implemented="" 50%="" of="" the="" methods="" presented="" in="" the="" article.="" variation="" of="" marks="" in="" this="" group="" will="" depend="" on="" the="" quality="" of="" report.="" iii)="" good=""><><7): appropriately="" implemented="" 70%="" of="" the="" methods="" presented="" in="" the="" article.="" variation="" of="" marks="" in="" this="" group="" will="" depend="" on="" the="" quality="" of="" report.="" iv)="" excellent(x="">=7): appropriately implemented >=90% of the methods presented in the article. Variation of marks in this group will depend on the quality of report. : . 2. Design and develop your own ML solution for this problem. The proposed solution should be different from all approaches mentioned in the provided article. This does not mean that you must have to choose a new ML algorithm. You can develop a novel solution by changing the feature selection approach or parameter optimisations process of used ML methods or using different ML methods or different combinations of them. This means, the proposed system should be substantially different from the methods presented in the article but not limited to only change of ML methods. Compare the result with reported methods in the article. Write a technical report summarising your solution design and outcomes. The report should include: (20 marks) i) Motivation behind the proposed solution. ii) How the proposed solution is different from existing ones. iii) Detail description of the model including all parameters so that any reader can implement your model. iv) Description of experimental protocol. v) Evaluation metrics. vi) Present results using tables and graphs. vii) Compare and discuss results with respect to existing literatures. viii) Appropriate references (IEEE numbered). N.B. This is a HD (High Distinction) level question. Those students who target HD grade should answer this question (including answering all the above questions). For others, this question is an option. This question aims to demonstrate your expertise in the subject area and the ability to do your own research in the related area. Marking criteria: Quality of solution Quality of report Overall score Unsatisfactory Unsatisfactory Unsatisfactory; Score<5 unsatisfactory="" fair="" unsatisfactory;=""><7 unsatisfactory="" good="" unsatisfactory;=""><10 fair="" unsatisfactory="" unsatisfactory;=""><10 fair="" fair="" fair;=""><12 fair="" good="" fair;=""><14 good="" unsatisfactory="" fair;="" score=""><14 good="" fair="" good;="" score=""><16 good="" good="" good;="" score="">=16 Quality of solution • Unsatisfactory: an appropriate solution presented whose performance is lower than the reported lowest performance in the article. • Fair: an appropriate solution presented whose performance is at least better than the lowest performance reported in the article. • Good: an appropriate solution presented whose performance is better than the best reported performances in the article. Quality of report : . • Unsatisfactory: either the report does not include all criteria mentioned above or the quality of description is poor. • Fair: the report has included all criteria mentioned above with an average quality of description. • Good: the report can be considered as a first draft for a publication. 3. Present your result in a 3 minutes video using PowerPoint slides/animation. (5 marks) Marking criteria: (i) Quality of audio presentation (ii) Quality of slides/animation. (iii) Completeness of the information. Microsoft Word - FieldNames.docx The data file contains the following independent attributes. Field Names duration: continuous. protocol_type: symbolic. service: symbolic. flag: symbolic. src_bytes: continuous. dst_bytes: continuous. land: symbolic. wrong_fragment: continuous. urgent: continuous. hot: continuous. num_failed_logins: continuous. logged_in: symbolic. num_compromised: continuous. root_shell: continuous. su_attempted: continuous. num_root: continuous. num_file_creations: continuous. num_shells: continuous. num_access_files: continuous. num_outbound_cmds: continuous. is_host_login: symbolic. is_guest_login: symbolic. count: continuous. srv_count: continuous. serror_rate: continuous. srv_serror_rate: continuous. rerror_rate: continuous. srv_rerror_rate: continuous. same_srv_rate: continuous. diff_srv_rate: continuous. srv_diff_host_rate: continuous. dst_host_count: continuous. dst_host_srv_count: continuous. dst_host_same_srv_rate: continuous. dst_host_diff_srv_rate: continuous. dst_host_same_src_port_rate: continuous. dst_host_srv_diff_host_rate: continuous. dst_host_serror_rate: continuous. dst_host_srv_serror_rate: continuous. dst_host_rerror_rate: continuous. dst_host_srv_rerror_rate: continuous. In addition, there is a target attribute named “attack_type”, which is a categorical variable with 22 different values. Before using this as a target variable, this one needs to be mapped into four types (dos, u2r, r2l and probe), as mentioned in Table 3 of the article, using following mapping