Support us on patreon and get an ad-free RSS feed with early episodes sometimes
Late Night Linux Late Night Linux – Episode 348
8 snips
Aug 26, 2025 The AI crawler bot arms race is heating up, as crawlers tackle security challenges with surprising speed. Google seems to be feigning concern for community input, raising eyebrows among tech enthusiasts. Excitingly, Linux desktop apps may soon make their way to Android devices, which could change the mobile landscape. Thunderbird is unveiling new paid services while trying to balance free offerings, and finally, a new domain for PuTTY sparks humor and security discussions among the hosts.
AI Snips
Chapters
Transcript
Episode notes
AI Crawler Arms Race
- AI crawlers quickly adapted to Anubis proof-of-work, defeating its protections in short order.
- The hosts conclude this is an arms race requiring new approaches beyond simple token challenges.
Use Blocks And Distribution To Limit Scraping
- Consider using distribution and blocklists to reduce abusive scraping load on servers.
- Deploy server-side mitigations like blocking IP ranges and removing easily-scraped assets to lower CPU and bandwidth costs.
Community Feedback Didn’t Change Google
- Google asked the community about removing XSLT then moved forward regardless of responses.
- The hosts see the engagement as performative and frustrating to standards stakeholders.
