Experiencing Data w/ Brian T. O’Neill (UX for AI Data Products, SAAS Analytics, Data Product Management)

172 - Building AI Assistants, Not Autopilots: What Tony Zhang’s Research Shows About Automation Blindness

6 snips
Jun 24, 2025
Tony Zhang, a researcher at Fortis with expertise in human-computer interaction, shares fascinating insights on AI design and user interaction. He discusses the risks of automation blindness, particularly in aviation where over-reliance on AI can hinder critical thinking. Zhang presents research showing how forward reasoning improves decision-making compared to traditional recommendations, emphasizing a hybrid approach that combines AI guidance with human intuition. The conversation also touches on enhancing user trust and the need for thoughtful AI integration in investment decisions.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
INSIGHT

Forward Reasoning Improves Decision-Making

  • Forward reasoning in AI interfaces helps users progress through decision-making rather than reverse engineering recommendations.
  • This approach reduces cognitive load compared to recommendation-centric AI that presents direct solutions.
INSIGHT

Flaws of Recommendation-Centric AI

  • Recommendation-focused AI in pilots led to poor choices by accepting seemingly obvious but flawed suggestions.
  • Forward reasoning designs significantly reduced pilots falling into suboptimal AI traps.
ANECDOTE

Airport Congestion Case Example

  • In an emergency scenario, pilots incorrectly followed AI suggesting a close airport without considering congestion caused by traffic from the closed main airport.
  • The AI failed to factor in traffic congestion, leading to poor AI-recommendation reliance.
Get the Snipd Podcast app to discover more snips from this episode
Get the app