180: Data Observability and AI for Data Operations Featuring Kunal Agarwal of Unravel Data
Mar 6, 2024
auto_awesome
Kunal Agarwal, CEO of Unravel Data, discusses data operations evolution, Unravel's role, challenges at scale, ROI on data products, cost management, observability in AI, and AI adoption challenges. He shares his journey from fashion to enterprise data management, emphasizing efficiency in cost management and measuring productivity and reliability. The podcast explores diverse technologies in data operations and final takeaways on driving better outcomes with data.
Unravel Data simplifies data operations to enhance productivity for data teams
AI adoption presents challenges that require observability for successful implementation
Deep dives
Introduction to Data Council Austin and Data Council's Offer
Data Council Austin, an event gathering top speakers and startups in data science, engineering, and AI, aims to provide a vendor-neutral technical conference setting. Attendees include smart founders, engineers, CTOs, and others driving innovations in data and AI. Listeners of The DataStack Show are offered a 20% discount using the promo code datastack20.
Unraveled Data's Mission to Simplify Data Operations
Unraveled Data, led by founders Kunal and Shivnath, focuses on automating and simplifying data operations to enhance productivity for data engineers and teams. Recognizing the time-consuming nature of troubleshooting, Unraveled Data aims to streamline tasks and make data work more efficient and exciting for teams.
Evolution of Data Operations Over the Last Decade
Over the past decade, data operations have shifted from on-prem batch workloads to real-time and AI-driven productions. Data teams have transitioned from traditional BI to advanced analytics and machine learning, requiring continuous upskilling and adaptation to evolving technologies. Unraveled Data's platform supports this evolution by simplifying data operations, ensuring reliability, scalability, and efficiency in today's dynamic data landscape.
Challenges and Solutions in Data Engineering Productivity and Efficiency
Data engineers face challenges in productivity due to troubleshooting, debugging, and inefficient problem-solving processes. Unraveled Data alleviates these issues by proactively identifying, diagnosing, and automating solutions for performance and reliability problems. By streamlining actions and enhancing collaboration, Unraveled Data improves team efficiency, reduces downtime, and fosters a more unified approach to problem-solving.
Unravel's role in simplifying data operations (2:17)
Kunal’s journey from fashion to enterprise data management (5:23)\
The Unravel platform and its components (10:08)
Challenges in data operations at scale (16:34)
Users of Unravel within an organization (22:32)
Calculating ROI on data products (25:55)
Understanding the cost of data operations (27:01)
Measuring productivity and reliability (30:59)
Diversity of technologies in data operations (34:52)
Efficiency in cost management (44:15)
Implementing observability in AI (47:55)
Challenges of AI Adoption (50:17)
Final thoughts and takeaways (51:36)
The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.
RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode