Rogue Startups

RS335: Evaluating AI Model Performance with Stuart Grey

Jan 22, 2025
In this engaging discussion, Dr. Stuart Grey, an AI expert and university educator, shares his insights on the transformative power of AI in everyday life and business. He talks about the balance of teaching technical skills with critical thinking in engineering education. The episode dives into practical AI tools for content generation and the importance of ethical considerations. Grey also reveals his AI Rules of Thumb, and emphasizes the need for human oversight in an increasingly automated world.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
ANECDOTE

Teaching AI Through Chatbots

  • Stuart Grey teaches AI to engineering students focusing on critical thinking over prompt engineering.
  • He has students build chatbots to explore AI capabilities and limitations, especially in non-technical topics like ethics.
ADVICE

Experiment Beyond Platforms

  • Experiment with prompts outside specific platforms using plain text editors for flexibility.
  • Try multiple models to understand their unique strengths before committing to one tool.
ADVICE

Human Judgment in AI Evaluation

  • Evaluate AI outputs subjectively by eyeballing multiple runs for each model.
  • Check consistency by running the same prompt in new chats to avoid bias from previous context.
Get the Snipd Podcast app to discover more snips from this episode
Get the app