Software Engineering Radio - the podcast for professional software developers cover image

Episode 503: Diarmuid McDonnell on Web Scraping

Software Engineering Radio - the podcast for professional software developers

00:00

Web Scraping

It would be lovely if there was a very clear distinction between which web sites were legal and which were not. In the u k, for example, there isn't a specific piece of legislation that forbids web scraping. It's those restrictions on what you can do with the data having collected it through your web scraper. So i may be able to technically and legally scrape it, but i mightn't be able to do any analysis, or repackage it, or share it in some findings. But often web sits aren't explicit in banning web scraping, but they will have lots of conditions about the use of the data you find on the web pages.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app