Rust in Production

InfinyOn with Deb Chowdhury

Oct 31, 2024

Deb Roy Chowdhury, VP of Product Management at InfinyOn, shares his insights on evolving data infrastructures. He explains how legacy systems create integration chaos and how InfinyOn's Fluvio simplifies data streaming across various platforms. Chowdhury delves into Rust’s role in enhancing asynchronous tasks and the efficient handling of diverse data formats. He also discusses the challenges of no-code solutions and the importance of observability, emphasizing how a unified approach can empower data teams and streamline workflows.

56:34

Creator website

Episode guests

Deb Roy Chowdhury

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The podcast emphasizes the shift from traditional healthcare roles to data-centric positions, highlighting the growing importance of data analysis in improving patient care.

It discusses the challenges posed by legacy systems in data analytics, advocating for modern big data technologies to enhance performance and efficiency.

The use of Rust in building the Fluvio data streaming platform showcases a trend towards adopting efficient solutions for complex data workflows and integration challenges.

Deep dives

Career Transition and Data's Role in Healthcare

The discussion highlights the transition from traditional healthcare roles to roles centered around data analysis. As healthcare systems became digitized, there was a growing need to analyze population health and improve quality of care, leading to more data-centric roles in the field. Early experiences involved descriptive statistics and the handling of previously paper-based systems, showcasing a paradigm shift towards data accumulation and analysis. This transition underscores the importance of data management in modern healthcare, ultimately shaping approaches to patient care and operational efficiency.

Intro

3min

Navigating the Data Landscape

21min

Asynchronous Evolution in Rust

8min

Transitioning Front-End Technologies for Enhanced Efficiency

2min

Enhancing Data Handling with Fluvio

5min

Integrating Rust in Data Ecosystems

13min

Challenges of No-Code Platforms and the Importance of Observability

4min

Picture this: Your organization's data infrastructure resembles a busy kitchen with too many cooks. You're juggling Kafka for messaging, Flink for processing, Spark for analytics, Airflow for orchestration, and various Lambda functions scattered about. Each tool excellent at its job, but together they've created a complex feast of integration challenges. Your data teams are spending more time managing tools than extracting value from data.
InfinyOn reimagines this chaos with a radically simple approach: a unified system for data streaming that runs everywhere. Unlike traditional solutions that struggle at the edge, InfinyOn gracefully handles data streams from IoT devices to cloud servers. And instead of cobbling together different tools, developers can build complete data pipelines using their preferred languages - be it Rust, Python, or SQL - with built-in state management.

At the heart of InfinyOn is Fluvio, a Rust-based data streaming platform that's fast, reliable, and easy to use.

About InfinyOn

Data pipelines are often slow, unreliable, and complex. InfinyOn, the creators of Fluvio, aims to fix this. Built in Rust, Fluvio offers fast, reliable data streaming. It lets you build event-driven pipelines quickly, running as a single 37 MB binary. With features like SmartModules, it handles various data types efficiently. Designed for developers, it offers a clean API and intuitive CLI. Streamline your data infrastructure at infinyon.com/rustinprod.

About Deb Roy Chowdhury

For fifteen years, Deb has been a behavioral detective, piecing together human decision-making through conversations, data, and research. His passion lies in product innovation—finding that sweet spot where desirability, viability, and feasibility converge. From 7-person startups to tech giants of 165,000, he helped build products that people love. Deb is currently the VP of Product Management at InfinyOn, where he leads the product strategy and roadmap for Fluvio, a Rust-based data streaming platform.

Proudly Supported by CodeCrafters

CodeCrafters helps you become proficient in Rust by building real-world, production-grade projects. Learn hands-on by creating your own shell, HTTP server, Redis, Kafka, Git, SQLite, or DNS service from scratch.

Start for free today and enjoy 40% off any paid plan by using this link.

Links From The Show

Polars - Fast DataFrame library implemented in Rust
Apache Arrow - Cross-language development platform for in-memory data
Arroyo - SQL-based data streaming platform in Rust
Arroyo Podcast Episode with Micah Wylde
NATS - High-performance messaging system written in Go
Memphis - Go-based streaming stack built with NATS as its core
Four Horsemen Of Bad Rust Code (FOSDEM 2024) - Matthias' talk on writing bad Rust code

Official Links

About corrode

"Rust in Production" is a podcast by corrode, a company that helps teams adopt Rust. We offer training, consulting, and development services to help you succeed with Rust. If you want to learn more about how we can help you, please get in touch.

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Rust in Production

InfinyOn with Deb Chowdhury

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

Career Transition and Data's Role in Healthcare

Challenges of Legacy Systems

The Emergence of Rust in Data Engineering

Fluvio and Modular Data Infrastructure

Community and Future of Rust in Data

Remember Everything You Learn from Podcasts