How To Think About Scraping

This week on my podcast, I read a recent Medium column. “How To Think About Scraping: In privacy and labor fights, copyright is a clumsy tool at best,” about the real risks (and benefits) of web-scraping, and how to formulate policy responses that preserve those benefits while targeting the harms head-on”

Scraping when the scrapee suffers as a result of your scraping is good, actually.

Mario Zechner is an Austrian technologist who used the APIs of large grocery chains to prove that they were colluding to rig prices. Zechner was able to create a corpus of historical price and product data to show how grocers used a raft of deceptive practices to trick people into thinking they were getting a good deal, from shrinkflation to cyclic price changes that were deceptively billed as “discounts.”

At first, Zechner worked alone and in fear of reprisals from the giant corporations whose fraudulent practices — which affected every person in the country — he had revealed.

But eventually, he was able to get the Austrian bureaucrat in charge of enforcing competition rules to publish a report lauding his work. Zechner open-sourced his project and attracted volunteers who started pulling in data from Germany and Slovenia.

MP3

(Image: syvwlch, CC BY 2.0, modified)