Glauber Costa, co-founder and CEO of Turso, shares insights on reimagining databases by rewriting SQLite in Rust with a project called Limbo. He discusses the challenges of SQLite's public domain status and the need for contributions. Glauber highlights the innovative use of Deterministic Simulation Testing for ensuring reliability and the future aspirations of the Terso platform, including open-source engagement. The conversation also touches on the creation of LibSQL and the team's vision for a billion databases, emphasizing the importance of flexibility and community involvement.
Turso is pioneering the Limbo architecture to rewrite SQLite in Rust, addressing the database's limitations for better community contributions.
The introduction of Deterministic Simulation Testing (DST) enables rigorous testing scenarios that bolster developer confidence in system stability.
Turso aims to balance open-source contributions with business interests, ensuring a comprehensive transition strategy for existing SQLite users and developers.
Deep dives
Introduction to Terso and Limbo
Terso, founded by Glabra Costa, is an open-source distributed database that aims to replace SQLite. The project was initiated to address the limitations of SQLite, particularly its lack of contribution opportunities despite being public domain. By utilizing the LibSQL codebase, the team began to rewrite the database with a new architecture, codenamed Limbo, enabling additional flexibility and community contributions. This ambitious endeavor aims to maintain SQLite’s user-friendly features while incorporating innovations that allow for distributed systems.
The Journey from LibSQL to Limbo
Initially, LibSQL served as a fork of SQLite to provide a different feature set without compromising the original codebase. However, through practical experience, the Terso team realized that to truly innovate and expand capabilities, a complete rewrite was necessary. The Limbo architecture allows for unprecedented contributions from the open-source community, showcasing a clear commitment to developing an advanced database solution. As a result, Limbo has gained significant traction within a short period, with many developers eager to contribute and explore new possibilities.
Implementation of Deterministic Simulation Testing
A key aspect of the Limbo project is the introduction of Deterministic Simulation Testing (DST), a method that allows for extensive testing of the database under various scenarios. This process involves simulating countless input situations to identify potential growth areas before full-scale implementation. With support from companies like Antithesis, the team can conduct integrated tests that rigorously validate system stability and functionality. This level of testing instills confidence in developers, enabling faster iterations and promoting collaborative innovation.
Setting New Goals and Compatibility Measures
The Terso team envisions a realistic timeline to bring Limbo to a production-ready state within nine months to a year, with the objective of achieving significant milestones such as reaching a billion databases. They aim to provide extensive compatibility with current SQLite features while introducing advanced functionalities, making it a compelling choice for developers. To ease the transition for existing SQLite users, meticulous documentation will outline compatibility measures and ensure a smoother switch to Limbo. The project seeks to establish itself not only as an alternative to SQLite but as a necessary evolution in database technology.
Future Directions and Open Source Integration
Terso's future strategy emphasizes merging their platform with the open-source community while maintaining a balance with business interests. The separation of server code and client libraries will maintain a clear focus on user needs, allowing Limbo to act as a fully open-source database option. While LibSQL will continue to provide a reference for users wanting an open-source server solution, Terso intends to keep its new server proprietary for scalability improvements. This approach aims to optimize deployment across different environments while welcoming contributions from developers to ensure the long-term success of the Terso project.
Glauber Costa, co-founder and CEO of Turso, joins us to discuss libSQL, Limbo, and how they’re rewriting SQLite in Rust. We discuss their efforts with libSQL, the challenge of SQLite being in the public domain but not being open for contribution, their choice to rewrite everything with Limbo, how this all plays into the future of the Turso platform, how they test Limbo with Deterministic Simulation Testing (DST), and their plan to replace SQLite.
Changelog++ members save 6 minutes on this episode because they made the ads disappear. Join today!
Sponsors:
Retool – The low-code platform for developers to build internal tools — Some of the best teams out there trust Retool…Brex, Coinbase, Plaid, Doordash, LegalGenius, Amazon, Allbirds, Peloton, and so many more – the developers at these teams trust Retool as the platform to build their internal tools. Try it free at retool.com/changelog
Temporal – Build invincible applications. Manage failures, network outages, flaky endpoints, long-running processes and more, ensuring your workflows never fail. Register for Replay in London, March 3-5 to break free from the status quo.
Notion – Notion is a place where any team can write, plan, organize, and rediscover the joy of play. It’s a workspace designed not just for making progress, but getting inspired. Notion is for everyone — whether you’re a Fortune 500 company or freelance designer, starting a new startup or a student juggling classes and clubs.
Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes.
