Talk Python To Me #299: Personal search engine with datasette and dogsheep
Jan 17, 2021
In this engaging talk, Simon Willison, co-creator of the Django web framework, dives into his journey from programming on a Commodore 64 to shaping open-source tools for data journalism. He introduces Dataset, a web-native SQLite explorer that simplifies working with data, and discusses Dogsheep, his innovative personal data warehouse idea. Highlighting the importance of SQL as a read-only API and showcasing practical examples like tracking his dog's weight, Simon shares insights into unifying personal data and the future of data management.
AI Snips
Chapters
Books
Transcript
Episode notes
SQLite As A Web-Native API
- Dataset turns any SQLite file into a web-native, bookmarkable JSON/CSV API for exploration and publishing.
- It treats SQL as the API language and exposes queries via URLs for sharing and integration.
Extend With Plugins, Not Core Changes
- Use Dataset's plugin system to add formats, visualizations, and auth without changing core code.
- Publish new features as independent plugins so others can extend the ecosystem without heavy maintainer involvement.
Publish Read-Only DBs With Serverless
- Publish read-only SQLite files with serverless hosts by bundling the .db file as a static asset.
- Use this pattern to avoid managed databases when your data doesn't need writes and keep hosting cheap and simple.
