The Testing Psychologist Podcast 527. AI for Testing Psychologists, Part 2: Ethics, Supervision, and Protecting Clinical Judgment in the AI Era
Would you rather read the transcript? Click here.
In this episode, I step back from the nuts and bolts of AI tools and dig into the ethical, supervisory, and professional development challenges of using AI in psychological assessment. I share insights from aviation, medicine, and radiology to highlight risks like automation bias, skill decay, and over-reliance on AI. I also discuss how clinicians and trainees can protect their critical thinking, balance workload reduction with deliberate practice, and frame AI as a thought partner rather than a replacement for judgment. This conversation is designed to spark reflection on how AI is shaping not just our workflows, but our ethics, training practices, and professional identity as psychologists.
Main topics covered:
- Does reducing cognitive load always help—or can it erode clinical skill?
- Strategies to prevent skill decay and maintain clinical proficiency
- Over-reliance, automation bias, and how to resist it
- Structured reflection exercises to improve judgment
- The role of deliberate practice in training and supervision
- Using AI as a thought partner versus a worker
- How AI-supported education can preserve skill development pathways
- Frameworks for trainees: explain-to-verify and reflective reasoning
Cool Things Mentioned
- The Testing Psychologist mastermind groups and business consulting
- “Panel on AI & Testing (AAPdN Annual Conference, April 2025)”: https://www.theaapdn.org/
- “We’re All Wrong Sometimes: Blindspots, Biases, & Getting Better w/ Dr. Stephanie Nelson (The Testing Psychologist #276)”: https://thetestingpsychologist.com/276-were-all-wrong-sometimes-blindspots-biases-getting-better-w-dr-stephanie-nelson/
- “Reverb (AI-assisted report writing for psychologists)”: https://reverbreports.com/
- “Automation bias: a systematic review of frequency, effect, mediators, and mitigators (JAMIA, 2011)”: https://pmc.ncbi.nlm.nih.gov/articles/PMC3240751/
- “Humans and automation: use, misuse, disuse, abuse (Parasuraman & Riley, 1997)”: https://web.mit.edu/16.459/www/parasuraman.pdf
- “Automation-induced complacency: NASA review and experiments (Prinzel et al., 2001)”: https://ntrs.nasa.gov/api/citations/20020021642/downloads/20020021642.pdf
- “Cognitive forcing strategies in clinical decisionmaking (Croskerry, 2003)”: https://pubmed.ncbi.nlm.nih.gov/12514691/
- “Effect of availability bias and reflective reasoning on diagnostic accuracy (Mamede et al., JAMA 2010)”: https://jamanetwork.com/journals/jama/fullarticle/186585
- “The role of deliberate practice in the acquisition of expert performance (Ericsson, Krampe, & Tesch-Römer, 1993)”: https://cseweb.ucsd.edu/classes/fa08/cse599/Papers/Ericsson.pdf
- “Care to Explain? AI Explanation Types Differentially Impact Chest Radiograph Diagnostic Performance and Physician Trust in AI (Radiology, 2024)”: https://pubmed.ncbi.nlm.nih.gov/39560483/
- “Does AI help or hurt human radiologists’ performance? (Harvard DBMI summary of Agarwal et al., 2024)”: https://dbmi.hms.harvard.edu/news/does-ai-help-or-hurt-human-radiologists-performance-depends-doctor
Featured Resources
I am honored to partner with two AMAZING companies to help improve your testing practice!
PAR is a long time supporter of the Testing Psychologist podcast, and I will continue to bring you featured items from their catalog! Listen to each episode for specific product recommendations and links. Learn more at www.parinc.com.
TherapyNotes is the leading EHR system for mental health practitioners. I’ve used TherapyNotes for over 10 years, and it just keeps getting better. Use this link and the promo code “testing” to get two free months and try it for yourself! www.therapynotes.com/testing
The Testing Psychologist podcast is approved for CEU’s!
I’ve partnered with At Health to offer CE credits for podcast episodes! Visit this link to access current and past episodes available for CE credit. You can use code “TTP10” for a discount on ALL the course credits you purchase from At Health!
About Dr. Jeremy Sharp
I’m a licensed psychologist and Clinical Director at the Colorado Center for Assessment & Counseling, a private practice that I founded in 2009 and have grown to over 20 clinicians. I earned my undergraduate degree in Experimental Psychology from the University of South Carolina before getting my Master’s and PhD in Counseling Psychology from Colorado State University. These days, I specialize in psychological and neuropsychological evaluation with kids and adolescents.
As the host of the Testing Psychologist Podcast, I provide private practice consulting for psychologists and other mental health professionals who want to start or grow psychological testing services in their practices. I live in Fort Collins, Colorado with my wife (also a therapist) and two young kids.
Ready to grow your testing services? Click below to schedule a complimentary 30-minute pre-consulting call!