Recently I was in need of a simple job queue for a Rust project. I already had Postgres in place and wondered if I could reuse it for this purpose. I found Tembo, a simple job queue written in Rust that uses Postgres as a backend. It fit the bill perfectly.

In today's episode, I talk to Adam Hendel, the founding engineer of Tembo, about their project, PGMQ, and how it came to be. We discuss the design decisions behind job queues, interfacing from Rust to Postgres, and the engineering decisions that went into building the extension.

It was delightful to hear that you could build all of this yourself, but that you would probably just waste your time doing so and would come up with the same design decisions as Adam and the team.

About Tembo

Tembo builds developer tools that help teams build and ship software faster. Their first product, PGMQ, was created to solve the problem of job queues in a simple and efficient way, leveraging the power of Postgres. They since made a pivot to focus on AI-driven code assistance, but PGMQ can be used independently and is available as an open-source project.

About Adam Hendel

Adam Hendel is the founding engineer at Tembo, where he has been instrumental in developing PGMQ and other tools like pg_vectorize. He has since moved on to work on his own startup, but remains involved with the PGMQ project.

Links From The Episode

PostgreSQL - Super flexible ~40 year old relational database that just works
R - Statistical Programming Language
pgrx - Extend Postgres with Rust
Postgres Docs: PL/pgSQL - Scripting with Procedural Language in PostgreSQL
Postgres Docs: SPI - The Postgres Server Programming Interface
pgmq - A lightweight message queue extension, initially written in Rust
Tembo Blog: Introducing PGMQ - a blog post about the project
sqlx - All of the great things of an ORM, without all of the bad things of an ORM
tokio - The de facto standard async runtime for Rust
AWS SQS - Amazon Web Services Simple Queue Service
Postgres Docs: LISTEN - The native Postgres sub part of of pubsub
Postgres Docs: NOTIFY - The native Postgres pub part of of pubsub
tokio-stream - Tokio utility for asynchronous series of values
Postgres Docs: Full Text Search - Postgres included FTS capabilities
pgvector - The standard extension for vector/AI workloads in Postgres
pg_vectorize - Automatically create embeddings for use with pgvector
Python Standard Library: None - A type, but not an enum
Rust in Production: Astral with Charlie Marsh - Massively improving Python day 1 experience
Hugging Face candle - Use ML models in Rust

Official Links

#149 "Las herramientas"