Why AI in user research isn’t replacing real people (yet) with Mario Callegaro

Episode web page: https://bit.ly/3KUDMzV

Episode description:

In this episode of Insights Unlocked, host Amrit Bhachu sits down with Mario Callegaro, founder of Callegaro Research and a leading expert in survey methodology and AI-augmented research. Mario brings a wealth of experience from his 15 years at Google and shares practical, thoughtful guidance on how to responsibly integrate AI tools into UX, market, and survey research workflows.

From the evolving role of the researcher to the risks and potential of synthetic users, this episode unpacks what’s changing, what to be cautious of, and what skills are now essential in the age of AI.

What you’ll learn

• How AI supports every phase of the research workflow: planning, execution, and activation

• The role of prompt engineering in getting better insights from AI tools

• Why synthetic users and synthetic data can't yet replace real human participants

• What researchers are gaining—and potentially losing—by relying on AI

• The current limitations of large language models in capturing nuance and variability

• Practical advice on testing and validating AI tools with known datasets

• Why transparency, iteration, and responsible experimentation are key

Resources & links

• Mario on LinkedIn (https://www.linkedin.com/in/mcallegaro/)

• Amrit on LinkedIn (https://www.linkedin.com/in/amritsbhachu/)

• Callegaro Research (https://callegaroresearch.com/)

• Quantitative User Experience Association (https://www.quantuxcon.org/)

• There’s an AI for that (https://theresanaiforthat.com/)

• Mike on LinkedIn (https://www.linkedin.com/in/mikemace/)

• Nathan Isaacs on LinkedIn (https://www.linkedin.com/in/nathanisaacs/)

• Learn more about Insights Unlocked: https://www.usertesting.com/podcast