Oxide and Friends cover image

Oxide and Friends

Crucible: The Oxide Storage Service

Feb 14, 2024
01:38:40
Snipd AI
Members of the Oxide storage team, including Josh, Alan, James, and Matt, discuss Crucible, a block storage service. They talk about the challenges of storing data, building a crash-safe disk representation, starting a new project, and collaboration. They also cover testing system reliability, resolving back pressure issues, and the development and open-sourcing of Crucible.
Read more

Podcast summary created with Snipd AI

Quick takeaways

  • The team utilizes Rust and extensive testing tools to ensure reliability and optimize different parts of the storage system, resulting in improved performance and efficiency.
  • Authenticated encryption adds complexity to the system, making it challenging to fit encrypted blocks into existing structures, highlighting the importance of security considerations.

Deep dives

The Crucible team begins development of the storage system

Alan and James join the team and start working on implementing the storage system. Alan works on the lowest layers, interacting directly with the bits on disk, while James focuses on simulating different failure modes and testing the system. They utilize Rust and create extensive testing tools to ensure reliability and allow for future refactoring. James demonstrates the ability to boot on Crucible, marking a significant milestone. The team develops a suite of tests, including unit tests and integration tests, to cover different scenarios and failure conditions. Matt joins the team and works on improving performance, leveraging the extensive test suite and Rust's ability to make confident refactorings. They optimize different parts of the system, such as changing the on-disk format and replacing SQLite with raw files for storing metadata. The team continues to iterate and make improvements to make the storage system faster and more efficient.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode