Observability, trace analysis and PM-engineer tension

Today’s Episode

AI PM jobs pay 30-40% more than regular PM jobs.

But here’s the problem: You can’t just slap “AI PM” on your resume.

Todd Olson has spent 28 years in product management, VP of Product at a public company, then founder of Pendo, now a $2.5B product management platform working with everyone from American Cancer Society to Zendesk.

----

Check out the conversation on Apple, Spotify and YouTube.

Brought to you by - Reforge:

Get 1 month free of Reforge Build (the AI prototyping tool built for PMs) with code BUILD

----

Key Takeaways

1. AI PM market exploded - Last year 10% of PM jobs were AI PM jobs. This year it's 20%. They pay 30-40% more because of scarcity and skill level. But Todd warns: "You better damn well be good and know what you're talking about if you're gonna call yourself an AI PM because we are going to interrogate the hell out of it."2. Real requirement is production at scale - Not "I built prototype at 1-person startup." Hiring managers want 20,000 paying B2B customers experiencing your AI feature successfully. To get there: upskill internally at current company by shipping AI features on your roadmap.3. The 5-layer technical pyramid - Foundation: AI/ML fundamentals, data pipelines, prompt engineering. Middle: Observability (trace analysis), cost optimization, evals. Top: Product strategy, stakeholder management, leadership. You need to climb all 5 layers. Most PMs stop at layer 1.4. RAG is table stakes - "RAG is the de facto way to build." You ingest data, create embeddings, feed into vector database, look up relevant context, pass to LLM. Todd: "If you put too much in context window, just like a human, you get confused. You want to give the right context."5. PM-engineering tension is real - At startups, PMs do trace analysis. At large companies, engineering managers push back: "This is my world. I don't want some PM shadowing me." Similar to Data Dog—most PMs don't have login. Know the line. Be fluent but respect boundaries.6. But evals are YOUR domain - Unlike trace analysis, evals are where PMs are the expert. "The PM is probably the best-suited human being to author and manage eval sets." You understand user and business needs. Engineers don't have that context. This is must-have competency now.7. Cost optimization will matter - Some AI companies have sub-15% gross margins. Traditional software is 70-80%. Todd: "It's not a business at sub-15%." Eventually you'll rearchitect systems because infrastructure is too costly. Rule: when something's faster, it's cheaper (both buying compute).8. Solve hard problems, not shiny objects - Todd's test: "Are we gonna do much better job than ChatGPT out of box? Why would we just wrap that and slap Pendo logo on it?" His discovery agent example: hard part isn't interviewing customers—it's finding which to interview, prioritizing, scheduling. Automate that workflow.9. Kill bad features ruthlessly - Todd shipped features couple years ago that weren't great and turned them off. "Too often we hold on to something. Turn them off. Be unafraid. The more stuff in your product, the worse the experience is by default."10. Control the narrative with boards - Don't show up with no story and get crushed with random requests. Todd: "Show them how you actually run your business. I want to see what you're looking at, not something just made for me." Think deeply about how each bet drives shareholder value.

----

Where to Find Todd Olson

* LinkedIn

* Company

* X

----

Related Content

Podcasts:

* How to Become, and Succeed as, an AI PM | The Marily Nika Episode

* If you only have 2 hrs, this is how to become an AI PM

* Complete Course: AI Product Management

Newsletters:

* How to Become an AI Product Manager with No Experience

* How to Write a Killer AI Product Manager Resume

* How to become an AI Product Manager

----

PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!

----

If you want to advertise, email productgrowthppp at gmail.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe