AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics

Adam Wenchel, Arthur founder and CEO, discusses evaluating AI model performance

Jan 15, 2024

Adam Wenchel, CEO of Arthur, discusses evaluating AI model performance. He shares insights on Capital One's early AI strategy and the value of monitoring AI model performance. The podcast also explores the debate between closed foundation models and OpenSURS models, managing accuracy in enterprise deployment of LLMs, and the importance of explainability in AI models. The speakers discuss the intense process of creating enduring startups.

30:25

Episode guests

Adam Wenchel

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The importance of evaluating AI model performance and monitoring its performance in production.

The challenges faced by enterprises in operationalizing and scaling generative AI models.

Deep dives

Importance of AI Ethics and Interdisciplinary Approach

The podcast episode highlights the importance of AI ethics and proposes an interdisciplinary approach to address AI's biggest questions. The suggested approach involves forming an AI ethics team consisting of a technologist, data ethicist, philosopher, and neuroscientist. This team would help make decisions about algorithm inclusion, ethical outputs, and human involvement. Transparency in AI usage and involving the public in key decisions are also emphasized to build trust.

Introduction

3min

Operationalizing AI Models with Arthur

15min

Closed Foundation Models vs OpenSURS Models

4min

Managing Accuracy and Responsibility in Enterprise Deployment of LLMs

2min

Explaining AI Models and the Importance of Explainability

4min

The Intense Process of Creating and the Road to Success

2min

Adam Wenchel is the CEO of Arthur, a company with a platform that gives you an immediate comprehensive AI performance solution across LLMs, Computer Vision, Tabular Data, and NLP.

Adam and the team have been making AI observable for almost 5 years with Arthur. Arthur has raised over $60 million from a legendary group of investors including Index Ventures, Greycroft Partners, Work-Bench, and others. An equal list of impressive customers they’ve served includes Humana and Plaid, among others.

Listen and learn

How Adam started his career in AI
How he helped map out Capital One’s early AI strategy
The value of evaluating AI model performance
How Arthur launched its first LLM-specific product and what the team learned
How to monitor the performance of an LLM model in production and which questions to ask when evaluating it
The lessons from growing a startup that nobody talks about

References in this episode…