2min chapter

Talk Python To Me cover image

#283: Web scraping, the 2020 edition

Talk Python To Me

CHAPTER

How to Parse With Scrappy?

With scrappy, it's really easy to maintain the code. For each thing that you find, you can yield that back as part of the thing,. And then you can follow the next one. In tools like BeautifulSoup, you would need to sort of create your own logic or like, you know, how you want to do that. But in scrappy, there are high-level functions and methods that let you paginate with scrappy. Yeah, yeah, very cool. I think that's probably the biggest differentiator from what I can tell is,  you're directing it to go do more discovery, maybe on linked pages and things like that.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode