#187 Mohamed Elgendy: Systematic Testing for Generative AI Models with Kolena

May 18, 2024

Mohamed Elgendy from Kolena talks about systematic testing for AI models, emphasizing the importance of structured frameworks, human evaluation, and collaboration with industry bodies to shape AI standards. They discuss challenges in testing AI models, defining metrics, fine-tuning generative AI models, and risk mitigation strategies, highlighting the importance of ongoing monitoring and standardization.

Ask episode

Chapters

Transcript

Episode notes

Introduction

00:00 • 5min

Challenges and Solutions in Testing AI Models

05:05 • 5min

Importance of Defining and Testing Metrics for AI Models

10:24 • 3min

Testing and Fine-Tuning Generative AI Models

12:57 • 29min

Risk Mitigation Strategies for Generative AI Models

41:43 • 14min

Closing Remarks, AI Impact, and Sponsors' Promotions

55:24 • 2min

This episode is sponsored by Oracle. AI is revolutionizing industries, but needs power without breaking the bank. Enter Oracle Cloud Infrastructure (OCI): the one-stop platform for all your AI needs, with 4-8x the bandwidth of other clouds. Train AI models faster and at half the cost. Be ahead like Uber and Cohere.

If you want to do more and spend less like Uber, 8x8, and Databricks Mosaic - take a free test drive of OCI at https://oracle.com/eyeonai

In this episode of the Eye on AI podcast, join us as we dive into the realm of AI quality assurance with Mohamed Elgendy, CEO and co-founder of Kolena.

Discover the intricacies of ensuring robust AI performance as Mohamed shares his journey from a software engineer to leading an AI quality platform. Kolena is setting new standards in AI testing, addressing the non-deterministic nature of machine learning models with a systematic framework for validation.

The conversation navigates the challenges of AI testing, the importance of structured quality assurance, and the collaboration with industry bodies to shape the future of AI standards. Mohamed elaborates on Kolena's innovative approach to both human and automated evaluations, providing comprehensive testing solutions tailored to diverse customer needs.

We also explore real-world use cases and the platform's future plans to make AI testing more accessible and reliable. Mohamed's insights illuminate the path toward achieving high-quality AI systems, ensuring they perform as intended across various applications.

Tune in to understand the advancements propelling AI quality forward and how Kolena is pioneering the way in this crucial aspect of AI development.

Don't forget to like, subscribe, and hit the notification bell for more insights into the technologies driving the AI revolution.

This episode is sponsored by Shopify. Shopify is a commerce platform that allows anyone to set up an online store and sell their products. Whether you’re selling online, on social media, or in person, Shopify has you covered on every base. With Shopify you can sell physical and digital products. You can sell services, memberships, ticketed events, rentals and even classes and lessons.

Stay Updated:

Craig Smith Twitter: https://twitter.com/craigss

Eye on A.I. Twitter: https://twitter.com/EyeOn_AI