Python 101 for Data Analysis. Homework is Jupyter notebook using Pandas. The Jupyternotebook and 3 files are attached. Total 5 questions. 1. Find the movies that have more than one genre2. Create a...

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### `Project - MovieLens Data Analysis`\n",
"\n",
"The GroupLens Research Project is a research group in the Department of Computer Science and Engineering at the University of Minnesota. The data is widely used for collaborative filtering and other filtering solutions. However, we will be using this data to act as a means to demonstrate our skill in using Python to “play” with data.\n",
"\n",
"### `Objective:`\n",
"- To implement the techniques learnt as a part of the course.\n",
"\n",
"### `Datasets Information:`\n",
"\n",
"*rating.csv:* It contains information on ratings given by the users to a particular movie.\n",
"- user id: id assigned to every user\n",
"- movie id: id assigned to every movie\n",
"- rating: rating given by the user\n",
"- timestamp: Time recorded when the user gave a rating\n",
"\n",
"*movie.csv:* File contains information related to the movies and their genre.\n",
"- movie id: id assigned to every movie\n",
"- movie title: Title of the movie\n",
"- release date: Date of release of the movie\n",
"- Action: Genre containing binary values (1 - for action 0 - not action)\n",
"- Adventure: Genre containing binary values (1 - for adventure 0 - not adventure)\n",
"- Animation: Genre containing binary values (1 - for animation 0 - not animation)\n",
"- Children’s: Genre containing binary values (1 - for children's 0 - not children's)\n",
"- Comedy: Genre containing binary values (1 - for comedy 0 - not comedy)\n",
"- Crime: Genre containing binary values (1 - for crime 0 - not crime)\n",
"- Documentary: Genre containing binary values (1 - for documentary 0 - not documentary)\n",
"- Drama: Genre containing binary values (1 - for drama 0 - not drama)\n",
"- Fantasy: Genre containing binary values (1 - for fantasy 0 - not fantasy)\n",
"- Film-Noir: Genre containing binary values (1 - for film-noir 0 - not film-noir)\n",
"- Horror: Genre containing binary values (1 - for horror 0 - not horror)\n",
"- Musical: Genre containing binary values (1 - for musical 0 - not musical)\n",
"- Mystery: Genre containing binary values (1 - for mystery 0 - not mystery)\n",
"- Romance: Genre containing binary values (1 - for romance 0 - not romance)\n",
"- Sci-Fi: Genre containing binary values (1 - for sci-fi 0 - not sci-fi)\n",
"- Thriller: Genre containing binary values (1 - for thriller 0 - not thriller)\n",
"- War: Genre containing binary values (1 - for war 0 - not war)\n",
"- Western: Genre containing binary values (1 - for western - not western)\n",
"\n",
"\n",
"*user.csv:* It contains information of the users who have rated the movies.\n",
"- user id: id assigned to every user\n",
"- age: Age of the user\n",
"- gender: Gender of the user\n",
"- occupation: Occupation of the user\n",
"- zip code: Zip code of the use\n",
"\n",
"**`Please provide your insights wherever necessary.`**\n",
"\n",
"### `Learning Outcomes:`\n",
"- Exploratory Data Analysis\n",
"\n",
"- Visualization using Python\n",
"\n",
"- Pandas – groupby, merging \n",
"\n",
"\n",
"### `Domain` \n",
"- Internet and Entertainment\n",
"\n",
"**Note that the project will need you to apply the concepts of groupby and merging extensively.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Observations:**\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data--> user id movie id rating timestamp\n",
"0 196 242 3 881250949\n",
"1 186 302 3 891717742\n",
"2 22 377 1 878887116\n",
"3 244 51 2 880606923\n",
"4 166 346 1 886397596\n",
"item--> movie id movie title release date unknown Action Adventure Animation \\\n",
"0 1 Toy Story 01-Jan-1995 0 0 0 1 \n",
"1 2 GoldenEye 01-Jan-1995 0 1 1 0 \n",
"2 3 Four Rooms 01-Jan-1995 0 0 0 0 \n",
"3 4 Get Shorty 01-Jan-1995 0 1 0 0 \n",
"4 5 Copycat 01-Jan-1995 0 0 0 0 \n",
"\n",
" Childrens Comedy Crime ... Fantasy Film-Noir Horror Musical \\\n",
"0 1 1 0 ... 0 0 0 0 \n",
"1 0 0 0 ... 0 0 0 0 \n",
"2 0 0 0 ... 0 0 0 0 \n",
"3 0 1 0 ... 0 0 0 0 \n",
"4 0 0 1 ... 0 0 0 0 \n",
"\n",
" Mystery Romance Sci-Fi Thriller War Western \n",
"0 0 0 0 0 0 0 \n",
"1 0 0 0 1 0 0 \n",
"2 0 0 0 1 0 0 \n",
"3 0 0 0 0 0 0 \n",
"4 0 0 0 1 0 0 \n",
"\n",
"[5 rows x 22 columns]\n",
"head--> user id age gender occupation zip code\n",
"0 1 24 M technician 85711\n",
"1 2 53 F other 94043\n",
"2 3 23 M writer 32067\n",
"3 4 24 M technician 43537\n",
"4 5 33 F other 15213\n"
]
}
],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import os\n",
"\n",
"data=pd.read_csv(r\"C:\\Users\\HP\\Desktop\\data.csv\")\n",
"item=pd.read_csv(r\"C:\\Users\\HP\\Desktop\\item.csv\")\n",
"user=pd.read_csv(r\"C:\\Users\\HP\\Desktop\\user.csv\")\n",
"\n",
"print(\"data-->\",data.head())\n",
"print(\"item-->\",item.head())\n",
"print(\"head-->\", user.head())\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Insights:**\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 6. Find the movies that have more than one genre - 5 marks\n",
"\n",
"hint: use sum on the axis = 1\n",
"\n",
"Display movie name, number of genres for the movie in dataframe\n",
"\n",
"and also print(total number of movies which have more than one genres)"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"required dataframe= movie title genre\n",
"0 Toy Story 3\n",
"1 GoldenEye 3\n",
"2 Four Rooms 1\n",
"3 Get Shorty 3\n",
"4 Copycat 3\n",
"5 Shanghai Triad (Yao a yao yao dao waipo qiao) 1\n",
"6 Twelve Monkeys 2\n",
"7 Babe 3\n",
"8 Dead Man Walking 1\n",
"9 Richard III 2\n",
"10 Seven (Se7en) 2\n",
"11 Usual Suspects, The 2\n",
"12 Mighty Aphrodite 1\n",
"13 Postino, Il 2\n",
"14 Mr. Holland's Opus 1\n",
"15 French Twist (Gazon maudit) 2\n",
"16 From Dusk Till Dawn 5\n",
"17 White Balloon, The 1\n",
"18 Antonia's Line 1\n",
"19 Angels and Insects 2\n",
"20 Muppet Treasure Island 5\n",
"21 Braveheart 3\n",
"22 Taxi Driver 2\n",
"23 Rumble in the Bronx 3\n",
"24 Birdcage, The 1\n",
"25 Brothers McMullen, The 1\n",
"26 Bad Boys 1\n",
"27 Apollo 13 3\n",
"28 Batman Forever 4\n",
"29 Belle de jour 1\n",
"... ... ...\n",
"1651 Entertaining Angels: The Dorothy Day Story 1\n",
"1652 Chairman of the Board 1\n",
"1653 Favor, The 2\n",
"1654 Little City 2\n",
"1655 Target 2\n",
"1656 Substance of Fire, The 1\n",
"1657 Getting Away With Murder 1\n",
"1658 Small Faces 1\n",
"1659 New Age, The 1\n",
"1660 Rough Magic 2\n",
"1661 Nothing Personal 2\n",
"1662 8 Heads in a Duffel Bag 1\n",
"1663 Brother's Kiss, A 1\n",
"1664 Ripe 1\n",
"1665 Next Step, The 1\n",
"1666 Wedding Bell Blues 1\n",
"1667 MURDER and murder 3\n",
"1668 Tainted 2\n",
"1669 Further Gesture, A 1\n",
"1670 Kika 1\n",
"1671 Mirage 2\n",
"1672 Mamma Roma 1\n",
"1673 Sunchaser, The 1\n",
"1674 War at Home, The 1\n",
"1675 Sweet Nothing 1\n",
"1676 Mat' i syn 1\n",
"1677 B. Monkey 2\n",
"1678 Sliding Doors 2\n",
"1679 You So Crazy 1\n",
"1680 Scream of Stone (Schrei aus Stein) 1\n",
"\n",
"[1681 rows x 2 columns]\n"
]
},
{
"data": {
"text/html": [
""
],
"text/plain": [
"col_0 count\n",
"genre \n",
"False 836\n",
"True 845"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df1 = item.iloc[:,3:21]\n",
"df1=df1.sum(axis = 1) \n",
"item['genre'] = df1\n",
"\n",
"df2 = pd.DataFrame(item,columns=['movie title','genre'])\n",
"print(\"required dataframe=\",df2)\n",
"\n",
"df3=df2[\"genre\"]>1\n",
"my_tab = pd.crosstab(index=df3, columns=\"count\")\n",
"my_tab"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Insights:**\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7. Univariate plots of columns: 'rating', 'Age', 'release year', 'Gender' and 'Occupation' - 10 marks\n",
"\n",
"*HINT: Use distplot for age and countplot for release year, ratings, *\n",
"\n",
"*HINT: Please refer to the below snippet to understand how to get to release year from release date. You can use str.split() as depicted below or you could convert it to pandas datetime format and extract year (.dt.year)*"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"brown\n",
"brown\n",
"brown\n"
]
}
],
"source": [
"a = 'My*cat*is*brown'\n",
"print(a.split('*')[3])\n",
"\n",
"#similarly, the release year needs to be taken out from release date\n",
"\n",
"#also you can simply slice existing string to get the desired data, if we want to take out the colour of the cat\n",
"\n",
"print(a[10:])\n",
"print(a[-5:])"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 83,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png":...