Shifting Schools: Conversations for K12 Educators

AI Companions: the risks and benefits, and what educators need to know

7 snips

Jul 21, 2025

In this enlightening conversation, Dr. Tomasz Hollanek, a postdoctoral research fellow at the University of Cambridge, dives deep into the ethical complexities of AI companions. He discusses the emotional manipulation risks of AI grief bots and the necessity of ethical design in AI systems. Dr. Hollanek emphasizes the importance of AI literacy for educators to safeguard vulnerable populations and navigate regulation challenges like the EU AI Act. He also shares promising examples of AI's role in preserving cultural histories, urging educators to shape a mindful AI future.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Risks of AI Companions

AI companions might offer temporary comfort but risk replacing human connections and deepening isolation.
Their human-like design increases potential emotional dependency, notably in vulnerable teenagers.

ADVICE

Rethink Consent Design

Implement recurring consent mechanisms for AI interactions rather than one-time agreements.
Design these prompts to be meaningful but not disruptive to ensure informed, ongoing user consent.

ADVICE

Broaden AI Literacy Focus

Focus AI literacy on social and ethical dimensions rather than just skill development.
Encourage active participation and critical thinking about AI's societal impacts, not just tool usage.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

How do we prepare students—and ourselves—for a world where AI grief companions and "deadbots" are a reality?

In this eye-opening episode, Jeff Utecht sits down with Dr. Tomasz Hollanek, a critical design and AI ethics researcher at the University of Cambridge's Leverhulme Centre for the Future of Intelligence, to discuss:

The rise of AI companions like Character.AI and Replika
Emotional manipulation risks and the ethics of human-AI relationships
What educators need to know about the EU AI Act and digital consent
How to teach AI literacy beyond skill-building—focusing on ethics, emotional health, and the environmental impact of generative AI
Promising examples: preserving Indigenous languages and Holocaust survivor testimonies through AI

From griefbots to regulation loopholes, Tomasz explains why educators are essential voices in shaping how AI unfolds in schools and society—and how we can avoid repeating the harms of the social media era.

Dr Tomasz Hollanek is a Postdoctoral Research Fellow at the Leverhulme Centre for the Future of Intelligence (LCFI) and an Affiliated Lecturer in the Department of Computer Science and Technology at the University of Cambridge, working at the intersection of AI ethics and critical design. His current research focuses on the ethics of human-AI interaction design and the challenges of developing critical AI literacy among diverse stakeholder groups; related to the latter research stream is the work on AI, media, and communications that he is leading at LCFI.

Connect with him:

https://link.springer.com/article/10.1007/s13347-024-00744-w

https://www.repository.cam.ac.uk/items/d3229fe5-db87-42ff-869b-11e0538014d8

https://www.desirableai.com/journalism-toolkit

📌 Key Takeaways

Teenagers are vulnerable AI users. Many systems simulate empathy while bypassing meaningful regulation or safeguards.
Consent needs a redesign. Hollanek proposes recurring consent mechanisms—a shift from passive pop-ups to informed, adaptive engagement.
AI literacy ≠ prompt engineering. We must move from tool proficiency to critical awareness of data footprints, systemic manipulation, and long-term impact.
Social AI is the new social media. Without thoughtful intervention, the pitfalls of social media could repeat—and intensify—with AI companions.
AI for cultural preservation. Ethical use of AI offers promise for sustaining languages and stories that might otherwise disappear.

🧠 For Educators: Use This Episode To

Spark classroom discussions on ethics, digital legacy, and emotional AI
Reflect on policy decisions around edtech adoption and student data consent
Explore speculative design as a student project to imagine future AI uses and risks
Guide students in analyzing AI's impact on mental health and community memory

Thank you to our show sponsor, Alongside.

Learn more about their critical research:

Inside the report: > Teens are struggling with sleep more than ever > School-life balance feels out of reach at every age > Boys are looking for new ways to ask for help > Confidential, self-guided tools are resonating deeply with students.

https://www.alongside.care/shifting?utm_campaign=Shifting