04 — Web Scraping & APIs

Collecting data from the web using rvest and APIs.

Whole lotta’ boxes.

This module covers web scraping with rvest, working with APIs, and the Billion Prices Project as a research application. Includes a practical exercise scraping IKEA Billy bookcase prices across countries.

Lecture slides

View fullscreen | Source

Code

Check the course repository for the R scripts.

Key packages: rvest, httr2, jsonlite

Further resources

• The Billy Index: The Billy Instead Of The Big Macs (Der Spiegel)
• Grant McDermott on web scraping with CSS and APIs
• The Billion Prices Project
• SelectorGadget — Chrome extension for identifying CSS selectors
• rvest documentation