2min chapter

Software Engineering Radio - the podcast for professional software developers cover image

Episode 503: Diarmuid McDonnell on Web Scraping

Software Engineering Radio - the podcast for professional software developers

CHAPTER

How Much Data Do You Need?

Web scraping can be collected at a scale that's very difficult to do through traditional means like surveys or focus groups, interviews, experiments and so on. T's when i've ben looking at the impact of the pandemic on non profit sectors. For example, i'm collecting, you know, tens of thousands, if not millions, of records for each jurisdiction. And than i'm aggregating all that information into a time series of the number of charities or non profits that are disappearing every month.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode