Lesson 40 of 55
PythonWeb Scraping with requests and BeautifulSoup
20 minutes
Course progress73%
What you'll learn
Keep these ideas in mind as you work through the lesson.
Lesson 40 of 55
Python20 minutes
Keep these ideas in mind as you work through the lesson.
Scraping lets you extract data from web pages. requests fetches the HTML, while BeautifulSoup parses it into a searchable structure.
import requests
from bs4 import BeautifulSoup
url = 'https://quotes.toscrape.com/'
response = requests.get(url)
response.raise_for_status()
soup = BeautifulSoup(response.text, 'html.parser')
quotes = soup.select('.quote .text')
for q in quotes[:5]:
print(q.get_text())
Always respect a site’s robots.txt and terms of service when you scrape.