Radar processes billions of location events daily, powering geofencing and location APIs for companies like Uber, Lyft, and thousands of other apps. When their existing infrastructure started hitting performance and cost limits, they built HorizonDB, a specialized database which replaced both Elasticsearch and MongoDB with a custom single binary written in Rust and backed by RocksDB.

In this episode, we dive deep into the technical journey from prototype to production. We talk about RocksDB internals, finite-state transducers, the intricacies of geospatial indexing with Hilbert curves, and why Rust's type system and performance characteristics made it the perfect choice for rewriting critical infrastructure that processes location data at massive scale.

About Radar

Radar is the leading geofencing and location platform, trusted by companies like Uber, Lyft, and thousands of apps to power location-based experiences. Processing billions of location events daily, Radar provides geofencing APIs, geocoding, and location tracking that enables developers to build powerful location-aware applications. Their infrastructure handles massive scale with a focus on accuracy, performance, and reliability.

About Jeff Kao

Jeff Kao is a Staff Engineer at Radar, where he led the development of HorizonDB, Radar's geospatial database written in Rust. His work replaced Elasticsearch and MongoDB with a custom Rust stack built on RocksDB, achieving dramatic improvements in performance and cost efficiency. Jeff has deep experience with geospatial systems and previously open-sourced Node.js TypeScript bindings for Google's S2 library. He holds a degree from the University of Waterloo.

Links From The Episode

Radar Blog: High-Performance Geocoding in Rust - The blog post, which describes Radar's migration from Elasticsearch and MongoDB to Rust and RocksDB
FourSquare - The compay Jeff worked at before
Ruby - The basis for Rails
PagerDuty) - Another company Jeff worked at. Hes' been around!
CoffeeScript - The first big JavaScript alternative before TypeScript
Scala - A functional JVM based language
Wikipedia: MapReduce - Distributed application of functional programming
Wikipedia: Algebraic Data Types - The concept behind Rust's Enums, also present in e.g. Scala
Kotlin - Easier than Scala, better than Java
Apache Lucene - The core of ElasticSearch
Discord Blog: Why Discord is switching from Go to Rust - Always the #1 result in searches for Rust migrations
Radar Blog: Introducing HorizonDB - A really nice write up of Horizon's architecture
RocksDB - The primary storage layer used in HorizonDB
MyRocks - A MySQL Storage Engine using RocksDB, written by Facebook
MongoRocks - A MongoDB Storage Layer using RocksDB
CockroachDB - PostgreSQL compatible, distributed, SQL Database
InfluxDB - A timeseries database that used RocksDB at one point, and our very first guest in this Podcast!
sled - An embedded database written in Rust
rocksdb - Rust bindings for RocksDB
H3 - Uber's Geo Hashing using hexagons
S2 - Google's Geo Hashing library
Wikipedia: Hilbert Curve - A way to map 2 dimensions onto 1 while retaining proximity
Wikipedia: Finite-State Transducer - A state machine used for efficiently looking up if a word exists in the data set
Rust in Production: Astral - Our episode with Charlie Marsh about tooling for the Python ecosystem
burntsushi - A very prolific Rust developer, now working at Astral
fst - FST crate from burntsushi
Wikipedia: Trie - A tree-structure using common prefixes
Wikipedia: Levenshtein Distance - The number of letters you have to change, add, or remove to turn word a into word b
tantivy - A full-text search engine, written in Rust, inspired by Lucene
LightGBM - A gradient boosted tree, similar to a decision tree
fastText - A text classification library from Meta
Wikipedia: Inverted Index - An index used for e.g. full text search
Wikipedia: Okapi BM25 - The ranking algorithm used in tantivy
Wikipedia: tf-idf - A classic and simple ranking algorithm
Roaring Bitmaps - A very fast bitset library used in many places
corrode.dev: Be Simple - A sentiment right down Matthias' alley
loco-rs - Rust on Rails

Official Links

Live community event: Rachel Maddow Presents: Burn Order

Rachel Maddow Presents: Burn Order

Panel Closing Reflections

The AI-powered Podcast Player