InfinyOn with Deb Chowdhury

Oct 31, 2024

Deb Roy Chowdhury, VP of Product Management at InfinyOn, shares his insights on evolving data infrastructures. He explains how legacy systems create integration chaos and how InfinyOn's Fluvio simplifies data streaming across various platforms. Chowdhury delves into Rust’s role in enhancing asynchronous tasks and the efficient handling of diverse data formats. He also discusses the challenges of no-code solutions and the importance of observability, emphasizing how a unified approach can empower data teams and streamline workflows.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

SQL Server Struggles

Deb Roy Chowdhury worked with a complex SQL Server database with 4,200 tables.
Generating reports took days, and troubleshooting SQL queries spanning multiple pages was a nightmare.

INSIGHT

Composable Systems and JVM Disillusionment

Composable systems with separate storage, processing, and infrastructure layers theoretically enable better team collaboration.
However, the "write once, run anywhere" promise of the JVM often turned into "write once, debug everywhere."

INSIGHT

Shift to Stream Processing

Modern data processing now involves stream processing, smaller functions, and a Unix-like philosophy.
This contrasts with the older paradigm of batch processing and large machines.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Picture this: Your organization's data infrastructure resembles a busy kitchen with too many cooks. You're juggling Kafka for messaging, Flink for processing, Spark for analytics, Airflow for orchestration, and various Lambda functions scattered about. Each tool excellent at its job, but together they've created a complex feast of integration challenges. Your data teams are spending more time managing tools than extracting value from data.
InfinyOn reimagines this chaos with a radically simple approach: a unified system for data streaming that runs everywhere. Unlike traditional solutions that struggle at the edge, InfinyOn gracefully handles data streams from IoT devices to cloud servers. And instead of cobbling together different tools, developers can build complete data pipelines using their preferred languages - be it Rust, Python, or SQL - with built-in state management.

At the heart of InfinyOn is Fluvio, a Rust-based data streaming platform that's fast, reliable, and easy to use.

About InfinyOn

Data pipelines are often slow, unreliable, and complex. InfinyOn, the creators of Fluvio, aims to fix this. Built in Rust, Fluvio offers fast, reliable data streaming. It lets you build event-driven pipelines quickly, running as a single 37 MB binary. With features like SmartModules, it handles various data types efficiently. Designed for developers, it offers a clean API and intuitive CLI. Streamline your data infrastructure at infinyon.com/rustinprod.

About Deb Roy Chowdhury

For fifteen years, Deb has been a behavioral detective, piecing together human decision-making through conversations, data, and research. His passion lies in product innovation—finding that sweet spot where desirability, viability, and feasibility converge. From 7-person startups to tech giants of 165,000, he helped build products that people love. Deb is currently the VP of Product Management at InfinyOn, where he leads the product strategy and roadmap for Fluvio, a Rust-based data streaming platform.

Proudly Supported by CodeCrafters

CodeCrafters helps you become proficient in Rust by building real-world, production-grade projects. Learn hands-on by creating your own shell, HTTP server, Redis, Kafka, Git, SQLite, or DNS service from scratch.

Start for free today and enjoy 40% off any paid plan by using this link.

Links From The Show

Polars - Fast DataFrame library implemented in Rust
Apache Arrow - Cross-language development platform for in-memory data
Arroyo - SQL-based data streaming platform in Rust
Arroyo Podcast Episode with Micah Wylde
NATS - High-performance messaging system written in Go
Memphis - Go-based streaming stack built with NATS as its core
Four Horsemen Of Bad Rust Code (FOSDEM 2024) - Matthias' talk on writing bad Rust code

Official Links

About corrode

"Rust in Production" is a podcast by corrode, a company that helps teams adopt Rust. We offer training, consulting, and development services to help you succeed with Rust. If you want to learn more about how we can help you, please get in touch.