Analyzing the Costs of False Positives and False Negatives in Decision-Making

Ever wondered how to truly bridge the gap between technical expertise and practical business implementation? How did the terminology shift from "data mining" to "predictive analytics" and revolutionize the business world? 

Eric Siegel, Ph.D., is a leading consultant and former Columbia University professor who helps companies deploy machine learning. He is the founder of the long-running Machine Learning Week conference series and the author of several books. His latest work is titled, The AI Playbook: Mastering the Rare Art of Machine Learning Deployment.

Eric and Greg discuss what motivated Eric to leave academia to see real-world applications of his machine learning models. Eric explains the pressing challenges organizations face when deploying machine learning projects, and provides an insightful look at the cultural and incentive-driven barriers that often lead to failed projects and unmet expectations. By focusing on collaboration from the outset, Eric reveals how businesses can align machine learning initiatives with their core needs to foster successful integration and operational change.

*unSILOed Podcast is produced by University FM.*

Episode Quotes:

Ramping up on a semi technical understanding of data

03:46: Prediction is the most actionable thing you get from data, and the way you get it is with machine learning. Learn some data to predict. That's basically what it is. So, will the world wake up to this? Are they going to forever see it as arcane? What does that mean? So, be careful what you wish for, because flash forward to now, and everyone's all over this stuff in a way that's overzealous. We fetishize the core technology as the most awesome thing. We're more excited about the rocket science than the actual launch of the rocket. That is, getting it deployed, getting into action, making a difference in terms of actual business operations. And we're stuck there. Most new machine learning projects fail to reach deployment. So, still, there's a skill gap. Still, there's a kind of data literacy that's greatly needed across the non-data science community. But it's not foreboding once you actually dip your toe in. As a business stakeholder, you got to get your hands dirty, or your feet will get cold, and you won't get to the point. But that dirty hand stuff, it's only semi-technical. It's totally accessible.

Demos don't equal human intelligence

36:21: Generative AI is the most amazing thing I've ever seen in my life. But that's the problem. A great demo doesn't necessarily mean valuable, right? I think it's probably about five percent as valuable as the world seems to think, right? So, I mean, I spent six years in the Natural Language Processing Research Group at Columbia, where I was subsequently a professor during graduate school. I never thought I'd see what I can see today, but we need to recognize there's a big difference between something that's seemingly human-like and human.

On recognizing change

34:53: Do change management because the basic idea is so often overlooked. Again, we're fetishizing the core technology. More excited about the Rock Advanced Launch, but the launch is changed, right? You need to manage that change. The project needs to be reframed. It's not just a technology project. It's not a machine-learning project. It's an operations improvement project that uses machine learning as a core component but ultimately involves improvement, that is to say, change.

How do you drive a successful machine learning project?

56:38: We need to get everybody on the same page. We need to get those to speak in business terms, and for the business people to be interested in some of those concrete details. Business people might say, "Hey, look, I don't need to get involved in details. I don't need to pop the hood of my car to drive it, right? I don't need to know how the engine works." And that's true. Like, I personally have no idea, right? I know the general principles of internal combustion, but I don't know where the spark plugs are. But I'm totally an expert at driving. I know momentum, friction, the rules of the road, how the car operates, and the mutual expectations of drivers. The analogy holds: to drive a machine learning project successfully through to deployment, you need, analogously, those kinds of semi-technical understanding of what it means to run the project so that it will succeed.

Show Links:
Recommended Resources:

• Moneyball
• Artificial general intelligence
• Rexer Analytics
• Change Management

Guest Profile:

• LinkedIn Profile
• Machine Learning Week
• Gooder.ai

His Work:

• www.bizML.com
• MachineLearning.courses
• Amazon Author Page
• The AI Playbook: Mastering the Rare Art of Machine Learning Deployment
• HBR Guide to AI Basics for Managers
• Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die
• Strategic Analytics: The Insights You Need from Harvard Business Review
• Data Science and Business Intelligence: Advice from important Data Scientists around the World
• Forbes Articles