Shreya Shankar

Join Willem Pienaar, CTO of Cleric and creator of Feast, along with PhD student Shreya Shankar, as they tackle the toughest challenges in building AI agents. They discuss the ambiguity of 'ground truth' in evaluations, revealing three key gulfs of human-AI interaction that hinder success. The duo emphasizes the importance of moving humans out of the feedback loop, using implicit signals for faster learning. Practical techniques like heat maps for task failures and the complexities of simulated environments are also explored, shedding light on the inevitable performance ceiling of AI.

Shreya Shankar, a PhD student at UC Berkeley specializing in intelligent data processing, shares her insights on the innovative DocETL system. They discuss how this technology optimizes LLM-powered data pipelines, enhancing analysis of complex documents. Shreya highlights the challenges of data extraction from PDFs, the importance of human feedback in AI systems, and the need for tailored benchmarks in data processing. Real-world applications and the future of agentic systems are also examined, showcasing a visionary path in data management.

Shreya Shankar, a PhD student in EECS at UC Berkeley, dives into how Large Language Models (LLMs) are changing the game for unstructured enterprise data. She explains her innovative framework, DocETL, which streamlines semantic extraction and thematic analysis of text and PDFs. The conversation touches on the practical challenges of data extraction and the evolution towards multimodal processing with tools like DocWrangler. Shreya also highlights the importance of aligning user intent with model capabilities for better user experiences.

About This EpisodeShreya Shankar is a computer scientist, PhD student in databases at UC Berkeley, and co-author of "Operationalizing Machine Learning: An Interview Study", an ethnographic interview study with 18 machine learning engineers across a variety of industries on their experience deploying and maintaining ML pipelines in production.Shreya explains the high-level findings of "Operationalizing Machine Learning"; variables that indicate a successful deployment (velocity, validation, and versioning), common pain points, and a grouping of the MLOps tool stack into four layers. Shreya and Lukas also discuss examples of data challenges in production, Jupyter Notebooks, and reproducibility.Show notes (transcript and links): http://wandb.me/gd-shreya---💬 *Host:* Lukas Biewald---*Subscribe and listen to Gradient Dissent today!*👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

Experts from Amazon, Hex, Modal, Parlance Labs, and UC Berkeley share lessons learned from working with Large Language Models. They discuss the importance of evaluation and monitoring in LLM applications, data literacy in AI, the fine-tuning dilemma, real-world insights, and the evolving roles of data scientists and AI engineers.

Shreya Shankar, a PhD candidate at UC Berkeley with experience at Google Brain and Facebook, dives into the world of AI agents and document processing. She sheds light on how LLMs can efficiently handle vast amounts of data, maintaining accuracy without breaking the bank. Topics include the importance of human error review, the intricacies of transforming LLM workflows into reliable pipelines, and the balance of using cheap vs. expensive models. Shreya also discusses how guardrails and structured approaches can enhance LLM outputs in real-world applications.

Join Shreya Shankar, a UC Berkeley researcher specializing in human-centered data management systems, as she navigates the exciting world of large language models (LLMs). Discover her insights on the shift from traditional machine learning to LLMs and the importance of data quality over algorithm issues. Shreya shares her innovative SPaDE framework for improving AI evaluations and emphasizes the need for human oversight in AI development. Plus, explore the future of low-code tools and the fascinating concept of 'Habsburg AI' in recursive processes.

In this episode, Shreya Shankar, a computer scientist pursuing her PhD in databases at UC Berkeley, discusses the challenges faced by machine learning engineers, including ensuring consistent performance and accuracy in production ML models. She also explores the importance of preserving information for anomaly detection, quantitative statistics for data cleaning, and the evolution from MLOps to LLMOps and FMOps. The speakers discuss the difficulties of implementing ML models and ML monitoring, as well as varying perspectives on AI as a product and its impact on customer expectations.

In this engaging conversation, Shreya Shankar, a talented ML Engineer with a rich background at Facebook and Google Brain, shares her path to becoming a PhD candidate at UC Berkeley. She dives into the complexities of MLOps, highlighting crucial aspects like velocity, validation, and risks such as data leakage. Shreya emphasizes the importance of supporting underrepresented minorities in tech and discusses innovative AI tools that enhance creativity. Her insights into the evolving landscape of AI and data management spark excitement about the future of technology.

Explore insights from Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, and Shreya Shankar on building end-to-end systems with LLMs, the experimentation mindset for AI products, strategies for building trust in AI, the importance of data examination, and evaluation strategies for professionals. These lessons apply broadly to data science, machine learning, and product development.