3227: Dataiku on Managing LLMs Without the Chaos

Apr 1, 2025

In this engaging discussion, Emma Irwin, Director of Sales Engineering at Dataiku, shares her expertise on enterprise AI and ethical development. She dives into the challenges organizations face when implementing generative AI, emphasizing the need for effective governance and diverse teams. Emma outlines three pillars for sustainable success: flexible access controls, robust LLM usage documentation, and quantifying AI value. The conversation highlights the importance of tackling algorithm bias and the necessity for businesses to create secure environments for effective AI deployment.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

ADVICE

Flexible Access to LLMs

Implement an abstraction layer to decouple business solutions from specific technologies.
This allows flexibility and swapping between providers, and locking down access appropriately.

ADVICE

Discover and Document LLM Usage

Establish a mechanism to discover and document LLM-related objects.
If this technology becomes embedded across the business, tracking its impact is crucial.

ADVICE

Quantify GenAI Value

Establish clear metrics and benchmarks to measure ROI for GenAI projects.
Quantify effectiveness and address potential roadblocks early on.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

What does it really take to implement generative AI in a way that balances innovation, governance, and long-term value? In today’s episode, , I’m joined by Emma Irwin, Director of Sales Engineering at Dataiku. With a deep background in enterprise AI and experience supporting major organizations like AVIVA and the NHS, Emma brings a grounded, real-world perspective to one of the most hyped areas in tech today.

While most businesses are ramping up GenAI investments, few have the processes, controls, or workforce skill sets needed to scale safely and effectively. Emma and I dive straight into the challenges that IT leaders are facing right now—from managing LLM usage and controlling cost, to building secure frameworks that actually reduce risk rather than amplify it.

Emma unpacks the three key pillars every organization needs for sustainable GenAI success: access controls that keep your stack flexible, robust discovery mechanisms to document LLM usage across the enterprise, and value quantification to show the real return on AI initiatives. But what really stood out is the need for diverse teams and strong governance models to address bias in AI development. From sentiment analysis to internal chatbots and large-scale summarization use cases, Emma brings a mix of strategy and execution to the conversation.

We also explore the importance of secure sandbox environments, the value of audit-readiness through documentation automation, and why it’s time for every business to move beyond experimentation into a more structured, responsible phase of AI maturity. Emma is also a vocal advocate for women in tech and AI, and shares how mentorship, representation, and inclusive leadership can help shape a more equitable future for the industry.

So what guardrails do you have in place for GenAI? Are you really ready to move from pilot projects to enterprise-scale implementations? This episode is packed with insights for anyone building, managing, or scaling AI across the enterprise.