Full Roadmap: Become an AI PM in 2025 | Ankit Shukla, HelloPM Founder
Product Growth Podcast
Mastering AI Evaluations for Product Managers
This chapter underscores the significance of systematic AI evaluations for product managers to produce reliable and consistent AI products. It also introduces a live course by industry experts aimed at enhancing PMs' skills in implementing effective evaluation frameworks.
If this 80 minutes podcast doesn't make you an AI PM, I'll delete my channel!
Yes, I know what I just said. But this is too good!
You’ll learn:
- How to become an AI PM without any tech experience
- 2 main categories of AI PMs and how to break into each
- How to build great AI products your users love
- All concepts like Evals, RAG, MCP, etc
- Creating AI PM portfolio
And so much more
----
Brought to you by:
The AI Evals Course for PMs & Engineers: You get $800 with this link
Maven: Get $100 off my curation of their top courses
AI PM Certification: Get $500 with code AAKASH25
----
Timestamps:
Two Types of PMs in the World Right Now – 00:00:00
How Much AI PMs Make in the US & India – 00:01:49
Why Jobs Aren’t Marketed as “AI PM” Roles – 00:04:19
Live Slide Share Session Begins (AI PDLC) – 00:05:35
What People Get Wrong About AI PM Jobs – 00:08:51
Why AI Product Management Is Here to Stay – 00:11:00
Product Development Lifecycle Explained – 00:12:35
Ad 1: AI Evaluations – 00:14:47
Ad 2: Maven Courses – 00:15:46
Understanding PM Canals (Idea Sources) – 00:16:34
How AI Will Transform the Traditional Product Cycle – 00:18:55
The Two Branches of AI: Predictive vs Generative – 00:23:10
Diving Deeper into Generative AI – 00:27:17
Ad 3: AI PM Certification – 00:28:47
How to Build Your Own AI Use Case Database – 00:29:34
Why You Shouldn’t Use AI for Everything – 00:34:10
Understanding Problem Space vs Solution Space – 00:36:51
Most Common Mistake People Make Entering AI PM – 00:40:49
The Building Blocks of AI – 00:47:01
Contextualization (Prompts, Fine-Tuning, RAG) – 00:50:24
The Limitations of AI — Why You Still Need Evals – 00:56:18
Case Study: AI-First Job Search Website – 00:58:49
Prompt Engineering Breakdown for the Case Study – 01:02:09
Understanding and Leveraging AI Agents – 01:03:40
Introducing the MCP Framework – 01:08:07
How to Build Your AI PM Portfolio – 01:11:37
Your Next 7 Steps to Becoming an AI PM – 01:16:04
Closing Thoughts and Final Notes – 01:18:41
----
Key Takeaways:
Takeaways:
01. Stop Shipping Blind. Your AI product isn't truly valuable until you validate it. Go beyond just building; understand user needs deeply with personas, journey maps, and jobs-to-be-done.
02. MOM Test = Your Secret Weapon. The "MOM Test" is about asking questions that even your most supportive friend can't lie about. Don't ask if users "would" use your AI. Ask about their past behaviors and real problems. This helps you define success metrics and avoid building a fancy toy nobody needs.
03. Evaluate Everything, Relentlessly. AI Evals are not just a technical task for engineers, but the most critical tool for Product Managers to build high-quality, trustworthy AI products. Use them to understand, refine, and continuously improve your AI.
04. Passion Won't Land the Job. Proof Will. "I'm passionate"...great I guess, but recruiters want to see what you've done. Your portfolio is your direct line to showing you can actually do the job.
05. Build Your AI Portfolio. Now. Don't wait for experience. Create product teardowns of AI tools, develop case studies, or launch small side projects. This is your living, breathing proof of thinking and skill.
06. Forget the Resume. Add Value. The ultimate job hack? Identify a problem at a target company and propose a solution, or even build a prototype before you apply. This showcases initiative and concrete skills.
07. You’re At Fault (Brutal, I Know). Nailing Prompt Engineering is a direct path to better AI outputs. If your AI misbehaves, it's often your fault for unclear instructions. Refine your prompts for smarter, more reliable AI.
08. Generic Resumes In The Bin! Forget sending generic resumes into the void. There are three distinct approaches: just a resume, adding a portfolio and cover letter, or the ultimate "Value Add" where you solve a company's problem before applying.
09. AI Will Do Your Dishes (Metaphorically). While AI Agents promise incredible autonomy and action, remember they still need clear goals and defined tasks. So, while your AI PM dream is big, maybe don't expect it to clean your dishes (yet) – stick to email automation for now!
10. Don't Trust LLMs Blindly. LLMs are powerful. But they need continuous human validation and evaluation frameworks. Automate grading where possible, but always, always, have a human in the loop for critical judgment.
----
Check out the conversation on Apple or Spotify and the demo on YouTube.
----
Related Podcasts:
If you only have 2 hrs, this is how to become an AI PM
How to Become, and Succeed as, an AI PM | The Marily Nika Episode
Complete Course: AI Product Management
Tutorial of Top 5 AI Prototyping Tools
----
P.S. More than 85% of you aren't subscribed yet. If you can subscribe on YouTube, follow on Apple & Spotify, my commitment to you is that we'll continue making this content better.
----
If you want to advertise, email productgrowthppp at gmail.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe
The AI-powered Podcast Player
