Eric Ma

Eric Ma, a research data science leader at Moderna specializing in AI for biotech, discusses the surprise finding that Moderna's systems are built on reliable workflows rather than autonomous agents. He emphasizes the importance of mapping permissions in regulated environments and the risks of data leaks from LLM execution traces. Offloading “janitorial” tasks to AI can improve efficiency, but Eric advises starting with simpler tools to reduce risks. He astutely highlights the need for evaluation rigor that matches the stakes involved in biotech applications.

Eric Ma, who leads data science research at Moderna, dives into the challenges of aligning retrieval with user intent in LLM-powered systems. He argues that most features fail not at the model level but with context. Eric reveals how a simple YAML-based approach can outperform complex pipelines and discusses the pitfalls of vague user queries. He also emphasizes the importance of evolving retrieval workflows to meet user needs and when it's sufficient to rely on intuition versus formal evaluation in refining these systems.

Join Eric Ma, who heads research data science at Moderna, as he discusses the wild world of AI systems over breakfast at SciPy. He reveals why 'perfect' testing can lead you astray and introduces three key personas in AI development, each with unique blind spots. Discover how curiosity can elevate builders from good to great, and learn about maintaining observability in both development and production. Eric also shares insights on fostering experimentation in large organizations, embracing the chaos that comes with creating thriving AI products.