Professor-list.csv
Name,Title,University
Lynn Batten,Emeritus Professor,Deakin University
Andrzej Goscinski,Emeritus Professor,Deakin University
Jemal Abawajy,Professor,Deakin University
Maia Angelova,Professor,Deakin University
Gleb Beliakov,Professor,Deakin University
Terry Caelli,Professor,Deakin University
Jinho Choi,Professor,Deakin University
Chang-Tsun Li,Professor,Deakin University
Robin Doss,Professor,Deakin University
Peter Eklund,Professor,Deakin University
Seng Loke,Professor,Deakin University
Antonio Robles-Kelly,Professor,Deakin University
Jean-Guy Schneider,Professor,Deakin University
Yong Xiang,Professor,Deakin University
John Yearwood,Professor,Deakin University
Arkady Zaslavsky,Professor,Deakin University
Mohamed Abdelrazek,Associate Professor,Deakin University
Andrew Cain,Associate Professor,Deakin University
Richard Dazeley,Associate Professor,Deakin University
Guangyan Huang,Associate Professor,Deakin University
Gang Li,Associate Professor,Deakin University
Jianxin Li,Associate Professor,Deakin University
Xiao Liu,Associate Professor,Deakin University
Vicky Mak,Associate Professor,Deakin University
Tim Wilkin,Associate Professor,Deakin University
Chetan Arora,Dr,Deakin University
Zubair Baig,Dr,Deakin University
Shaun Bangay,Dr,Deakin University
Muneera Bano,Dr,Deakin University
Carlo Barca,Dr Jan,Deakin University
Morshed Chowdhury,Dr,Deakin University
Niroshinie Fernando,Dr,Deakin University
Longxiang Gao,Dr,Deakin University
Shang Gao,Dr,Deakin University
Marek Gagolewski,Dr,Deakin University
Thuong Hoang,Dr,Deakin University
Michael Hobbs,Dr,Deakin University
Jingyu Hou,Dr,Deakin University
Shamsul Huda,Dr,Deakin University
Amani Ibrahim,Dr,Deakin University
Simon James,Dr,Deakin University
Frank Jiang,Dr,Deakin University
Chandan Karmakar,Dr,Deakin University
Elicia Lanham,Dr,Deakin University
Jesse Laeuchli,Dr,Deakin University
Kevin Lee,Dr,Deakin University
Wei Luo,Dr,Deakin University
Musa Mammadov,Dr,Deakin University
Kerri Morgan,Dr,Deakin University
Asef Nazari,Dr,Deakin University
Thanh Nguyen,Dr Duc,Deakin University
Thi Nguyen,Dr Thanh,Deakin University
Bahador Ofoghi,Dr,Deakin University
Lei Pan,Dr,Deakin University
Guillermo Pineda-Villavicencio,Dr,Deakin University
Chathu Ranaweera,Dr,Deakin University
Sutharshan Rajasegarar,Dr,Deakin University
Imran Razzak,Dr,Deakin University
Sergiy Shelyag,Dr,Deakin University
David Tay,Dr,Deakin University
Dhananjay Thiruvady,Dr,Deakin University
Rolando Trujillo Rasua,Dr,Deakin University
Julien Ugon,Dr,Deakin University
Nayyar Zaidi,Dr,Deakin University
Zili Zhang,Dr,Deakin University
Sasan Adibi,Dr,Deakin University
Adnan Anwar,,Deakin University
Sunil Aryal,Dr,Deakin University
Alexander Baldwin,Dr,Deakin University
Alessio Bonti,Dr,Deakin University
Reda Bouadjenek,Dr Mohamed,Deakin University
Greg Bowtell,Dr,Deakin University
Feifei Chen,Dr,Deakin University
Michelle Cyganowski,,Deakin University
Robert Dew,Dr,Deakin University
Imali Dias,Dr,Deakin University
Atabak Elmi,Dr,Deakin University
Shahid Ferdous,Dr Hasan,Deakin University
Anuroop Gaddam,Dr,Deakin University
Alireza Hassani,,Deakin University
Damien Hutchinson,Dr,Deakin University
Jonathan Kua,Dr,Deakin University
Glory Lee,Dr,Deakin University
Xuequan Lu,Dr,Deakin University
Daniel Ma,Dr,Deakin University
Somaiyeh MahmoudZadeh,Dr,Deakin University
Sophie McKenzie,,Deakin University
Valeh Moghaddam,Dr,Deakin University
Bahar Nakisa,Dr,Deakin University
Ghari Neiat,Dr Azadeh,Deakin University
Jihong Park,Dr,Deakin University
Shiva Pokhrel,Dr,Deakin University
Sergey Polyakovskiy,Dr,Deakin University
Justin Rough,Dr,Deakin University
Atul Sajjanhar,Dr,Deakin University
Arash Shaghaghi,Dr,Deakin University
Keshav Sood,Dr,Deakin University
Rui Wang,Dr,Deakin University
Jason Wells,,Deakin University
Guy Wood-Bradley,Dr,Deakin University
Luxing Yang,Dr,Deakin University
(Leo) Zhang,Dr Yu,Deakin University
Ye Zhu,Dr,Deakin University
Mumtaz Ali,Dr,Deakin University
Francisco Cruz,Dr,Deakin University
Jie Ding,Dr,Deakin University
Rasul Esmaeilbeigi,Dr,Deakin University
Belinda Huang,Dr,Deakin University
Jay Jeong,Dr,Deakin University
Tariq Khan,Dr,Deakin University
Thai Doan,Dr Chuong,Deakin University
Sunny Lee,,Deakin University
Xufeng Lin,Dr,Deakin University
Ming Liu,Dr,Deakin University
Alexey Medvedev,Dr,Deakin University
Iynkaran Natgunanathan,Dr,Deakin University
Thanh-Binh Nguyen,Dr,Deakin University
Tianrui Zong,Dr,Deakin University
Yevhen Zolotavkin,Dr,Deakin University
Amin Abken,Dr,Deakin University
Borui Cai,Dr,Deakin University
Grishma Khadka,Mr,Deakin University
Budi Kurniawan,Mr,Deakin University
Diaz Millan,Dr Reinier,Deakin University
Mendivez Vasquez,Mr Bruno,Deakin University
Jack Murley,Mr,Deakin University
Ashish Nanda,Dr,Deakin University
Patrick Scolyer-Gray,Dr,Deakin University
Youyang Qu,Mr,Deakin University
Nan Sun,Dr,Deakin University
Radhagayathri Udhayakumar,Dr,Deakin University
Di Wu,Dr,Deakin University
Roozbeh Zarei,Dr,Deakin University
Brendan Hill,Mr,Deakin University
Abid Adam,Adjunct Professor,Deakin University
Catherine Buhler,Adjunct Professor,Deakin University
Eshan Dissanayake,Adjunct Professor,Deakin University
Ben Doyle,Adjunct Professor,Deakin University
John Ellis,Adjunct Professor,Deakin University
Manohar Esarapu,Honorary Fellow,Deakin University
Patrick Fair,Adjunct Professor,Deakin University
David Fairman,Adjunct Professor,Deakin University
Mark Fitzgerald,Honorary Professor,Deakin University
Praveen Gauravaram,Adjunct Professor,Deakin University
Nigel Hedges,Adjunct Professor,Deakin University
Darren Kane,Adjunct Professor,Deakin University
Kerry Kirk,Honorary Fellow,Deakin University
Len Kleinman,Adjunct Professor,Deakin University
James Kotsias,Adjunct Professor,Deakin University
Berin Lautenbach,Adjunct Professor,Deakin University
Min Livanidis,Adjunct Professor,Deakin University
Phillip Magness,Adjunct Professor,Deakin University
Joseph Mathew,Adjunct Associate Professor,Deakin University
William Moran,Honorary Professor,Deakin University
Ana Novak,Honorary Associate Professor,Deakin University
Zahir Quettawala,Adjunct Professor,Deakin University
Anthony Robertshaw,Adjunct Associate Professor,Deakin University
Jamie Rossato,Adjunct Professor,Deakin University
Chadi Saliby,Adjunct Professor,Deakin University
Malcolm Shore,Adjunct Professor,Deakin University
Anthony Travers,Honorary Fellow,Deakin University
EJ Wise,Adjunct Professor,Deakin University
Fiona Ebdon,,Deakin University
Belinda Evans-Hunt,,Deakin University
Sheryl Pascoe,,Deakin University
Sheena Saunders,,Deakin University
Emmica Schlobohm,,Deakin University
Laura Wilson,,Deakin University
Sia Alavian,,Deakin University
Abbas Kudrati,Professor,Deakin University
Jesse Mcmeikan,,Deakin University
Thuy Nguyen,,Deakin University
Nghia Dang,,Deakin University
Justin Li,,Deakin University
Professor-citation-information.csv
Name,Title,citation-all,citation-since2016,h-index-all,h-index-since2016,i10-index-all,i10-index-since2016
Lynn Batten,Emeritus Professor,2535,1179,23,16,45,21
Andrzej Goscinski,Emeritus Professor,,,,,,
Jemal Abawajy,Professor,,,,,,
Maia Angelova,Professor,682,399,15,11,21,14
Gleb Beliakov,Professor,8182,4640,42,32,135,87
Terry Caelli,Professor,8669,1477,52,20,177,36
Jinho Choi,Professor,7836,5045,37,29,168,96
Chang-Tsun Li,Professor,4347,2341,33,21,104,65
Robin Doss,Professor,,,,,,
Peter Eklund,Professor,3968,783,34,14,72,23
Seng Loke,Professor,7188,2913,38,21,126,50
Antonio Robles-Kelly,Professor,3659,1538,24,15,62,27
Jean-Guy Schneider,Professor,1814,647,24,15,48,21
Yong Xiang,Professor,6318,4981,40,35,117,94
John Yearwood,Professor,3801,2006,32,23,92,45
Arkady Zaslavsky,Professor,14966,8361,50,33,177,80
Mohamed Abdelrazek,Associate Professor,1866,1265,16,15,27,23
Andrew Cain,Associate Professor,291,194,10,9,11,8
Richard Dazeley,Associate Professor,1478,956,16,13,24,15
Guangyan Huang,Associate Professor,1131,812,20,18,30,24
Gang Li,Associate Professor,,,,,,
Jianxin Li,Associate Professor,1467,1088,21,18,39,32
Xiao Liu,Associate Professor,3231,1760,26,20,52,42
Vicky Mak,Associate Professor,387,229,11,8,14,6
Tim Wilkin,Associate Professor,389,312,10,10,11,10
Chetan Arora,Dr,431,388,10,9,10,9
Zubair Baig,Dr,1491,1236,24,22,45,31
Shaun Bangay,Dr,,,,,,
Muneera Bano,Dr,1069,876,15,15,23,17
Carlo Barca,Dr Jan,,,,,,
Morshed Chowdhury,Dr,1825,1156,22,16,46,28
Niroshinie Fernando,Dr,2227,1532,9,9,8,8
Longxiang Gao,Dr,1698,1614,18,18,27,26
Shang Gao,Dr,410,293,8,6,5,3
Marek Gagolewski,Dr,743,518,15,12,21,16
Thuong Hoang,Dr,486,427,13,12,19,15
Michael Hobbs,Dr,987,212,13,7,17,7
Jingyu Hou,Dr,,,,,,
Shamsul Huda,Dr,1038,911,19,17,29,22
Amani Ibrahim,Dr,1332,639,14,12,17,13
Simon James,Dr,,,,,,
Frank Jiang,Dr,,,,,,
Chandan Karmakar,Dr,2489,1830,22,20,57,44
Elicia Lanham,Dr,168,34,7,3,5,1
Jesse Laeuchli,Dr,,,,,,
Kevin Lee,Dr,1440,448,15,10,21,13
Wei Luo,Dr,,,,,,
Musa Mammadov,Dr,1751,731,25,16,48,21
Kerri Morgan,Dr,147,73,7,4,6,2
Asef Nazari,Dr,223,119,8,6,8,4
Thanh Nguyen,Dr Duc,,,,,,
Thi Nguyen,Dr Thanh,,,,,,
Bahador Ofoghi,Dr,,,,,,
Lei Pan,Dr,748,656,14,13,24,19
Guillermo Pineda-Villavicencio,Dr,245,100,9,5,6,3
Chathu Ranaweera,Dr,648,521,13,12,15,13
Sutharshan Rajasegarar,Dr,,,,,,
Imran Razzak,Dr,,,,,,
Sergiy Shelyag,Dr,2402,1057,26,20,37,30
David Tay,Dr,1997,523,23,13,51,16
Dhananjay Thiruvady,Dr,519,274,13,10,17,13
Rolando Trujillo Rasua,Dr,804,609,15,13,21,18
Julien Ugon,Dr,1064,638,18,12,31,16
Nayyar Zaidi,Dr,414,326,9,8,8,7
Zili Zhang,Dr,,,,,,
Sasan Adibi,Dr,1557,890,18,14,30,19
Adnan Anwar,,,,,,,
Sunil Aryal,Dr,196,184,7,7,6,6
Alexander Baldwin,Dr,126,116,5,5,4,4
Alessio Bonti,Dr,403,212,6,5,5,4
Reda Bouadjenek,Dr Mohamed,368,315,10,10,11,11
Greg Bowtell,Dr,,,,,,
Feifei Chen,Dr,854,778,15,15,18,18
Michelle Cyganowski,,,,,,,
Robert Dew,Dr,,,,,,
Imali Dias,Dr,442,360,12,11,13,13
Atabak Elmi,Dr,382,294,9,9,9,9
Shahid Ferdous,Dr Hasan,447,374,10,10,12,10
Anuroop Gaddam,Dr,652,281,15,9,17,9
Alireza Hassani,,147,147,8,8,7,7
Damien Hutchinson,Dr,,,,,,
Jonathan Kua,Dr,247,243,3,3,2,2
Glory Lee,Dr,,,,,,
Xuequan Lu,Dr,239,225,9,9,9,8
Daniel Ma,Dr,,,,,,
Somaiyeh MahmoudZadeh,Dr,489,457,13,13,18,17
Sophie McKenzie,,322,202,9,8,8,7
Valeh Moghaddam,Dr,177,174,5,4,3,3
Bahar Nakisa,Dr,,,,,,
Ghari Neiat,Dr Azadeh,470,354,10,9,12,9
Jihong Park,Dr,1576,1555,16,16,30,30
Shiva Pokhrel,Dr,437,419,10,10,13,12
Sergey Polyakovskiy,Dr,,,,,,
Justin Rough,Dr,,,,,,
Atul Sajjanhar,Dr,826,276,11,8,17,6
Arash Shaghaghi,Dr,107,107,5,5,2,2
Keshav Sood,Dr,397,388,9,9,9,9
Rui Wang,Dr,,,,,,
Jason Wells,,248,95,8,5,7,2
Guy Wood-Bradley,Dr,12,9,2,2,0,0
Luxing Yang,Dr,2005,1509,25,24,36,35
(Leo) Zhang,Dr Yu,1475,1350,20,20,34,34
Ye Zhu,Dr,251,250,8,8,8,8
Mumtaz Ali,Dr,2517,2302,30,28,62,55
Francisco Cruz,Dr,,,,,,
Jie Ding,Dr,,,,,,
Rasul Esmaeilbeigi,Dr,77,76,3,3,3,3
Belinda Huang,Dr,,,,,,
Jay Jeong,Dr,,,,,,
Tariq Khan,Dr,763,676,17,17,24,22
Thai Doan,Dr Chuong,,,,,,
Sunny Lee,,,,,,,
Xufeng Lin,Dr,256,244,6,6,6,6
Ming Liu,Dr,,,,,,
Alexey Medvedev,Dr,610,588,10,10,11,10
Iynkaran Natgunanathan,Dr,1003,865,14,13,15,15
Thanh-Binh Nguyen,Dr,,,,,,
Tianrui Zong,Dr,,,,,,
Yevhen Zolotavkin,Dr,,,,,,
Amin Abken,Dr,62,60,5,5,1,0
Borui Cai,Dr,69,69,3,3,2,2
Grishma Khadka,Mr,,,,,,
Budi Kurniawan,Mr,,,,,,
Diaz Millan,Dr Reinier,82,64,6,6,4,3
Mendivez Vasquez,Mr Bruno,,,,,,
Jack Murley,Mr,,,,,,
Ashish Nanda,Dr,81,80,5,5,2,2
Patrick Scolyer-Gray,Dr,,,,,,
Youyang Qu,Mr,443,441,11,11,13,13
Nan Sun,Dr,,,,,,
Radhagayathri Udhayakumar,Dr,119,116,6,6,5,5
Di Wu,Dr,,,,,,
Roozbeh Zarei,Dr,137,109,7,5,3,3
Brendan Hill,Mr,,,,,,
Abid Adam,Adjunct Professor,,,,,,
Catherine Buhler,Adjunct Professor,,,,,,
Eshan Dissanayake,Adjunct Professor,,,,,,
Ben Doyle,Adjunct Professor,,,,,,
John Ellis,Adjunct Professor,,,,,,
Manohar Esarapu,Honorary Fellow,,,,,,
Patrick Fair,Adjunct Professor,,,,,,
David Fairman,Adjunct Professor,,,,,,
Mark Fitzgerald,Honorary Professor,,,,,,
Praveen Gauravaram,Adjunct Professor,,,,,,
Nigel Hedges,Adjunct Professor,,,,,,
Darren Kane,Adjunct Professor,,,,,,
Kerry Kirk,Honorary Fellow,,,,,,
Len Kleinman,Adjunct Professor,,,,,,
James Kotsias,Adjunct Professor,,,,,,
Berin Lautenbach,Adjunct Professor,,,,,,
Min Livanidis,Adjunct Professor,,,,,,
Phillip Magness,Adjunct Professor,,,,,,
Joseph Mathew,Adjunct Associate Professor,,,,,,
William Moran,Honorary Professor,,,,,,
Ana Novak,Honorary Associate Professor,,,,,,
Zahir Quettawala,Adjunct Professor,,,,,,
Anthony Robertshaw,Adjunct Associate Professor,,,,,,
Jamie Rossato,Adjunct Professor,,,,,,
Chadi Saliby,Adjunct Professor,,,,,,
Malcolm Shore,Adjunct Professor,,,,,,
Anthony Travers,Honorary Fellow,,,,,,
EJ Wise,Adjunct Professor,,,,,,
Fiona Ebdon,,,,,,,
Belinda Evans-Hunt,,,,,,,
Sheryl Pascoe,,,,,,,
Sheena Saunders,,,,,,,
Emmica Schlobohm,,,,,,,
Laura Wilson,,,,,,,
Sia Alavian,,,,,,,
Abbas Kudrati,Professor,,,,,,
Jesse Mcmeikan,,,,,,,
Thuy Nguyen,,,,,,,
Nghia Dang,,,,,,,
Justin Li,,,,,,,
SIT742-DataDictionary.xlsx
Dataset Description
DATA SET NAME HTWebLog_p1
DATA SIZE 2,098,416,889 bytes
DATE OF RELEASE 2021
NO. OF FILES 120
NO. OF ATTRIBUTES 15
NO. OF DATA RECORDS 8438929
DATA SOURCE PROVIDER Hotel Tulip
DATA PRIVACY Publicly Available
NOTES
Prepared by: [enter name here]
Point of Contact: [enter name, email address]
Team Members: [list team members if applicable]
Data Type Name Convention Main Type Subtype
MD Metric Discrete BIN - Binary
MC Metric Continous DATE - Date/time
CURR - Currency
CO Categorical Ordinal
CN Categorical Nominal DI - Dichotomous
STR - Free String
ID - Identification
URL - links such as URLs
ADDR - Address
Attribute Dictionary
Attribute Name Data Type Data Subtype Description Examples Additional otes
date MC DATE The date on which the activity occurred. Default, Date
time MC DATE The time, in coordinated universal time (UTC), at which the activity occurred. Default, Time
s-sitename CN The Internet service name and instance number that was running on the client. Service name and Instance number
s-ip CN URL The IP address of the server on which the log file entry was generated. Default, Server IP Address
cs-method CN The requested action, for example, a GET method. Default, Method
cs-uri-stem CN URL The target of the action, for example, Default.htm. Default, URI Stem
cs-uri-query CN STR The query, if any that the client was trying to perform. A Universal Resource Identifier (URI) query is necessary only for dynamic pages. Default, URI Query
s-port CN The server port number that is configured for the service. Default, Server Port
cs-username CN The name of the authenticated user who accessed your server. Anonymous users are indicated by a hyphen. Default, User name
c-ip CN The IP address of the client that made the request. Default, Client IP Address
cs(User-Agent) CN STR The browser type that the client used. Default, User Agent
cs(Referrer) CN STR The site that the user last visited. This site provided a link to the current site. Referrer
sc-status CN The HTTP status code. Default, HTTP Status
sc-substatus CN The sub status error code. Default, Protocol Substatus
sc-win32-status CN The Windows status code. Win32 Status
SIT742Task1.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "3D514Ei_kSUS"
},
"source": [
"# SIT742: Modern Data Science \n",
"**(2021 Assessment Task 01 - Data Manipulation)**\n",
"\n",
"---\n",
"- Please refer to the CloudDeakin for the detailed assessment requirements.\n",
"- Please submit to Clouddeakin before the due date.\n",
"- Students with difficulty in meeting the deadline because of illness, etc. must apply for an assignment extension (up to 3 days) no later than *12:00pm on 16/04/2021 (Friday)*.\n",
"\n",
"\n",
"Prepared by **SIT742 Teaching Team**\n",
"\n",
"\n",
"---\n",
"\n",
"
\n",
"Enter your information in the following cell. Please make sure you fill your Student ID and Name correctly. \n",
"
\n",
"\n",
"**Student Information:** Please fill your information below\n",
"\n",
"- Name: \n",
"- Student ID: \n",
"- Email: \n",
"- Lab Session and Tutor (for on campus students): \n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1X7L24t43CoU"
},
"source": [
"# Part I - Web Log Data\n",
"\n",
"## Overview\n",
"\n",
"Hotel TULIP a five-star hotel located at Deakin University, and its CIO Dr Bear Guts has asked the Team-SIT742 team to analyse the weblogs files. As an employee for Hotel Tulip, working in the Information Technology Division, it is required to prepare a set of documentation for Team-SIT742 to allow them to understand the data being dealt with. Throughout this report, some source codes are to explore the weblog, which afterwards the information is presented to Dr Bear Guts in the format of a report.\n",
"\n",
"In this part, you are required to use give dataset to finish some required analysis, with the exploreation data analytics skills as well as visuilization skill.\n",
"\n",
"\n",
"You will be required to complete the coding to answer the questions with a neat and clean format, and you must keep outputs for code. Your commenting and adherence to code standards will also be considered when marking. Further information on Python standards, please visit https://www.python.org/dev/peps/pep-0008/ \n",
"\n",
"\n",
"**Note**: You are not necessary to stick to the partial code provided, and you can write your own code to implement the required tasks. But you should provide sufficient comments.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "r85LNibZkSUZ"
},
"source": [
"\n",
"\n",
"## 1. Data ETL"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HtL_9QR27_Iu"
},
"source": [
"### 1.1 Data Loading"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "19EhpLspkSUZ"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import pandas as pd\n",
"import time\n",
"import datetime\n",
"from scipy import stats\n",
"import matplotlib"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jY2Br4cBN956"
},
"source": [
"
\n",
"\n",
"**Code**: \n",
" You are required to load Hotel TULIP data into dataframe df_ht and check how many files are loaded.\n",
"
\n",
" *Keep the outputs for code in your notebook.\n",
"
\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"id": "IIB_xH-hOqEj",
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"120 files were loaded\n"
]
}
],
"source": [
"from zipfile import ZipFile\n",
"\n",
"field_names = [\n",
" \"date\",\n",
" \"time\",\n",
" \"s-sitename\",\n",
" \"s-ip\",\n",
" \"cs-method\",\n",
" \"cs-uri-stem\",\n",
" \"cs-uri-query\",\n",
" \"s-port\",\n",
" \"cs-username\",\n",
" \"c-ip\",\n",
" \"cs(User-Agent)\",\n",
" \"cs(Referer)\",\n",
" \"sc-status\",\n",
" \"sc-substatus\",\n",
" \"sc-win32-status\"\n",
"]\n",
"\n",
"data = []\n",
"with ZipFile(\"HTWebLog_p1.zip\") as zipfiles:\n",
" print(f'{len(zipfiles.namelist())} files were loaded')\n",
" file_list = zipfiles.namelist()\n",
" data = [pd.read_csv(file_name, \n",
" sep=' ',\n",
" comment='#',\n",
" engine='python',\n",
" na_values='-',\n",
" names=field_names)\n",
" for file_name in file_list]\n",
"df_ht = pd.concat(data)"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"date 121\n",
"time 86372\n",
"s-sitename 1\n",
"s-ip 1\n",
"cs-method 6\n",
"cs-uri-stem 4761\n",
"cs-uri-query 1988\n",
"s-port 1\n",
"cs-username 0\n",
"c-ip 74548\n",
"cs(User-Agent) 4959\n",
"cs(Referer) 39703\n",
"sc-status 12\n",
"sc-substatus 4\n",
"sc-win32-status 14\n",
"dtype: int64"
]
},
"execution_count": 59,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df_ht.nunique()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Dyt9lwm_QKFV"
},
"source": [
"
\n",
"\n",
"**Code**: \n",
" Check the data statistics and general information by printing its top 5 rows.\n",
"
\n",
" *Keep the outputs for code in your notebook.\n",
"
\n",
"\n",
"
\n",
" \n",
"**Report**: \n",
" **Part I 1.1** Please include your result for this coding task into your SIT742-DataDictionary.xlsx, then add to your report - Part I with proper section title '1.1.1' and '1.1.2'.\n",
"
\n",
"
" 0 1 \\\n",
"date 2006-11-01 00:00:00 2006-11-01 00:00:00 \n",
"time 2021-04-20 00:00:08 2021-04-20 00:00:08 \n",
"s-sitename W3SVC1 W3SVC1 \n",
"s-ip 127.0.0.1 127.0.0.1 \n",
"cs-method GET GET \n",
"cs-uri-stem /Default.aspx /Tulip/home/en-us/home_index.aspx \n",
"s-port 80 80 \n",
"c-ip 70.80.84.76 70.80.84.76 \n",
"sc-status 200 200 \n",
"sc-substatus 0 0 \n",
"sc-win32-status 0 0 \n",