Crazy Wisdom

Episode #495: The Black Box Mind: Prompting as a New Human Art

Oct 6, 2025
Jared Zoneraich, CEO of PromptLayer, delves into the evolving art of AI engineering. He discusses how PromptLayer serves as a workbench for testing LLM products and the tension between implicit and explicit knowledge in AI. The conversation explores the complexities of prompt evaluation, distinguishing between deterministic and probabilistic systems. They also touch on 'vibe coding', which emphasizes the influence of syntax and tone on AI outputs, and how this approach could reshape coding education and practices.
Ask episode
AI Snips
Chapters
Books
Transcript
Episode notes
INSIGHT

AI Workbench For Domain Experts

  • PromptLayer positions itself as an AI engineering workbench to help domain experts build and test LLM products.
  • Jared argues the best AI builders are subject-matter experts, not ML engineers, and the tool scales their work.
ADVICE

Backtest Prompts On Real Production Data

  • Run rigorous evals and backtests on production data before deploying prompt updates.
  • Use heuristics like hallucination checks and format assertions to catch regressions.
ADVICE

Fix Prompt Bugs With Short Sprints

  • Use sprint-style iterations for practical prompt fixes: bootstrap recent production cases and iterate quickly.
  • Run multiple models and focus on hard disagreement cases to build a regression set.
Get the Snipd Podcast app to discover more snips from this episode
Get the app