Arpad Borsos from Sentry shares insights on using Rust for speed improvements, developing relay and symbolicator projects, and discussing actor-based vs. task-based concurrency. The podcast covers challenges in Rust SDK development, understanding clones and lifetimes, handling high traffic, and exploring mini dumps for native crashes.
Rust's stack trace integration still poses challenges for Sentry
Deep dives
Handling High Traffic with Rust
Rust has enabled Sentry to handle high volumes of traffic, ensuring the processing pipeline for over 2000 to 3000 events per second. By rewriting critical parts of the infrastructure in Rust, Sentry observed significant improvements of 10 to 20x in throughput and cloud costs. Rust's performance optimizations have allowed Sentry to scale efficiently and process over 100K Kafka messages per second in some services.
Python and Rust Integration in Sentry
Sentry integrates Rust with Python by developing extension modules in Rust called from Python and utilizing Rust for tasks like validating native debug files and ProGuard processing. The seamless integration allows Sentry to handle untrusted user inputs confidently and enhances error handling mechanisms within the stack.
Improving Error Handling and Stack Traces in Rust
Though Rust's detailed error handling is useful, improving stack trace integration remains a challenge, especially in capturing stack traces at the right origin points of errors. Expectations for Rust 2024 include advancements like Polonius for teaching the borrow checker new functionalities and support for first-class generator syntax.
Future of Rust in Sentry and the Ecosystem
Arpad recommends trying Rust for isolated projects, commending tools like PyO3 for seamless Rust integration with other languages. Additionally, the upcoming Rust 2024 edition holds promise for features like improved error handling, generator syntax, and self-referential data types. Sentry's robust SDK offers capabilities for capturing panics, tracing, and distributed tracing, making it a valuable observability tool for developers.
Conclusion
In summary, Rust in Production with Sentry showcases the benefits of using Rust for handling high traffic volumes efficiently, integrating Rust with Python seamlessly, and the ongoing improvements and challenges in error handling and stack trace integration within Rust. With the future outlook on Rust 2024 and continuous enhancements in Rust's ecosystem, Sentry remains at the forefront of utilizing Rust for robust and scalable infrastructure.
In this episode, we talk to Arpad Borsos, Systems Software Engineer at Sentry, about how they use Rust to build a modern error monitoring platform for developers.
We discuss the challenges of building a high-performance, low-latency platform for processing and analyzing large amounts of data (like stack traces and source maps) in real-time.
Arpad maintains the `symbolic` crate for stack trace symbolication, which is used on the Sentry platform.
About Sentry: Sentry provides application performance monitoring and error tracking software for JavaScript, Python, Ruby, Go, and more. Their platform also supports session replay, profiling, cron monitoring, code coverage, and more.
About Arpad Borsos: Arpad Borsos works on high-performance, low-latency systems and maintains open source projects like the popular `rust-cache` GitHub Action. He is an expert in asynchronous programming and spoke about async functions at EuroRust 2023.