Homework Assignment- Web data scraping

Homework 10 Web Data Scraping
Homework 10 Web Data Scraping
Due by 11:59pm, Friday, August 5, 2022
S&DS 230e
# Required libraries
library(car)
## Loading required package: carData
library(leaps)
library(lubridate)
##
## Attaching package: ’lubridate’
## The following objects are masked from ’package:base’:
##
## date, intersect, setdiff, union
library(rvest)
1) Scraping Info on S&P 500 Companies (30 pts - 5 pts each section, except part 1.5 which is 10 pts)
The first site you’ll scrape data from is HERE. The site contains some information on companies that make
up the Standard and Poor 500 index.
1.1) Make an object called url that has the website location. Get R to open this webpage. Read the html
information into an object called webpage.
# creating a object called "url"
url <- 'https://www.slickcharts.com/sp500'
# Reading url information
webpage <- read_html(url)
1.2) Using the web scraping techniques we discussed in class, create an object called companyRank that has
the company ranking. Make sure the object is stored as an integer. Get the head and length of this object.
Interestingly, the length is more than 500.
# Creating a object called "companyRank"
companyRank <- html_nodes(webpage, '.table-sm td')
companyRank = html_text(companyRank[seq(from = 1, to = length(companyRank),by = 7)])
companyRank = as.integer(companyRank)
# top six company ranking
head(companyRank)
1
https://www.slickcharts.com/sp500
## [1] 1 2 3 4 5 6
# Number of company ranks
length(companyRank)
## [1] 507
1.3) Similarly, create an object called companyName that contains the names of the companies. Again, get
the head and length of this object.
# Creating a object called "companyName"
companyName <- html_nodes(webpage, '.table-sm...