Turso is rewriting SQLite in Rust (Changelog Interviews #626)
Jan 30, 2025
auto_awesome
Glauber Costa, co-founder and CEO of Turso, discusses the exciting initiative of rewriting SQLite in Rust with innovative solutions like Limbo and libSQL. He explains the challenges posed by SQLite’s public domain status and how they aim to foster a community-driven approach to database technology. Glauber delves into the project's unique testing methods, including Deterministic Simulation Testing, and shares anecdotes that add a personal touch to the technical conversation. The vision for a more robust, scalable database system shines through, promising a new era for open-source databases.
Turso is creating LibSQL as a community-driven alternative to SQLite, addressing limitations in public contributions and innovation.
The ambitious project Limbo aims to rewrite SQLite in Rust, introducing new features while fostering a collaborative development environment.
Terso's innovative use of Deterministic Simulation Testing (DST) enhances quality assurance by allowing systematic testing of various database scenarios.
Deep dives
Introduction to Terso and LibSQL
Terso, co-founded by Glabra Costa, focuses on delivering an open source distributed database powered by LibSQL, an initiative created as a community-driven version of SQLite. The primary motivation for developing LibSQL arose from the recognition that, while SQLite is public domain software, it does not support public contributions, limiting community involvement and innovation. The founders sought to create a more adaptable and collaborative platform by forking SQLite, which would allow for enhancements and the addition of new features. This pursuit underscores their vision for creating a robust open source community around database technology.
Transition to Limbo Rewrite
The Terso team is now pursuing an ambitious project called Limbo, which involves rewriting SQLite from the ground up using Rust, with plans for new features and functionalities. This decision stems from their desire to remove the limitations associated with the existing fork of SQLite, allowing them to fully leverage the capabilities of a modern programming environment. They believe that creating Limbo, with a complete codebase and open community contributions, will unlock greater creativity and innovation in database functionality. By adopting a fresh approach, they aim for Limbo to become a worthy competitor to SQLite, addressing specific challenges that developers encounter.
Challenges with SQLite's Structure
A critical issue raised during discussions about SQLite centers on its inability to accept community contributions, which can hinder evolution and adaptability. The maintainers of SQLite have established their own criteria for changes and improvements, which restrict external collaboration despite its label as open source. This ultimately results in missed opportunities for a broader range of innovative features that could be integrated into the codebase. The Terso team believes this bottleneck has created an opening for Limbo, where the community can directly influence development and propel the software in new directions.
Testing Methodologies and Quality Assurance
Terso is incorporating a novel testing technique known as Deterministic Simulation Testing (DST) in the development of Limbo, which offers a significant upgrade in quality assurance. DST allows the team to systematically generate and test thousands of scenarios, verifying the integrity and reliability of the database. By leveraging DST, the developers can isolate and reproduce issues with ease, leading to faster and more efficient debugging processes. This rigorous testing framework is designed to ensure that Limbo's code meets high-quality standards before it reaches users, setting a new benchmark for database testing.
Future Goals and Community Building
Terso is committed to making Limbo a truly community-driven project, fostering an environment where contributors can actively participate in the development process. In parallel with the growth of Limbo, the team intends to maintain the LibSQL platform as an open-source alternative while scaling towards a more ambitious vision. Their target is to develop a user base comparable to SQLite, engaging developers and creating applications that leverage the advancements offered by Limbo. By focusing on transparent communication, encouraging external contributions, and treating the community as collaborative partners, Terso hopes to create a thriving ecosystem around their products.
Glauber Costa, co-founder and CEO of Turso, joins us to discuss libSQL, Limbo, and how they’re rewriting SQLite in Rust. We discuss their efforts with libSQL, the challenge of SQLite being in the public domain but not being open for contribution, their choice to rewrite everything with Limbo, how this all plays into the future of the Turso platform, how they test Limbo with Deterministic Simulation Testing (DST), and their plan to replace SQLite.
Changelog++ members save 6 minutes on this episode because they made the ads disappear. Join today!
Sponsors:
Retool – The low-code platform for developers to build internal tools — Some of the best teams out there trust Retool…Brex, Coinbase, Plaid, Doordash, LegalGenius, Amazon, Allbirds, Peloton, and so many more – the developers at these teams trust Retool as the platform to build their internal tools. Try it free at retool.com/changelog
Temporal – Build invincible applications. Manage failures, network outages, flaky endpoints, long-running processes and more, ensuring your workflows never fail. Register for Replay in London, March 3-5 to break free from the status quo.
Notion – Notion is a place where any team can write, plan, organize, and rediscover the joy of play. It’s a workspace designed not just for making progress, but getting inspired. Notion is for everyone — whether you’re a Fortune 500 company or freelance designer, starting a new startup or a student juggling classes and clubs.
Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes.