Why AI Still Needs Us? A conversation with Olga Megorskaya, CEO of Toloka

In this episode, I sit down with Olga Megorskaya, CEO of Toloka, to explore what true human-AI co-agency looks like in practice. We talk about how the role of humans in AI systems has evolved from simple labeling tasks to expert judgment and co-execution with agents – and why this shift changes everything.

We get into:

- Why "humans as callable functions" is the wrong metaphor – and what to use instead

- What co-agency really means?

- Why some data tasks now take days, not seconds – and what that says about modern AI

- The biggest bottleneck in human-AI teamwork (and it’s not tech)

- The future of benchmarks, the limits of synthetic data, and why it is important to teach humans to distrust AI

- Why AI agents need humans to teach them when not to trust the plan

If you're building agentic systems or care about scalable human-AI workflows, this conversation is packed with hard-won perspective from someone who’s quietly powering some of the most advanced models in production. Olga brings a systems-level view that few others can – and we even nerd out about Foucault’s Pendulum, the power of text, and the underrated role of human judgment in the age of agents.

Did you like the episode? You know the drill:

 📌 Subscribe for more conversations with the builders shaping real-world AI.

 💬 Leave a comment if this resonated.

 👍 Like it if you liked it.

 🫶 Thank you for watching and sharing!

Guest:

 Olga Megorskaya, CEO of Toloka

📰 Want the transcript and edited version?

 Subscribe to Turing Post https://www.turingpost.com/subscribe

Chapters

0:00 – Intro: Humans as Callable Functions?

0:33 – Evolving with ML: From Crowd Labeling to Experts

3:10 – The Rise of Deep Domain Tasks and Foundational Models

5:46 – The Next Phase: Agentic Systems and Complex Human Tasks

7:16 – What Is True Co-Agency?

9:00 – Task Planning: When AI Guides the Human

10:39 – The Critical Skill: Knowing When Not to Trust the Model

13:25 – Engineering Limitations vs. Judgment Gaps

15:19 – What Changed Post-ChatGPT?

18:04 – Role of Synthetic vs. Human Data

21:01 – Is Co-Agency a Path to AGI?

25:08 – How To Ensure Safe AI Deployment

27:04 – Benchmarks: Internal, Leaky, and Community-Led

28:59 – The Power of Text: Umberto Eco and AI

Turing Post is a newsletter about AI's past, present, and future. Publisher Ksenia Semenova explores how intelligent systems are built – and how they’re changing how we think, work, and live.

Sign up: Turing Post: https://www.turingpost.com

If you’d like to keep following

Olga and Toloka:

https://www.linkedin.com/in/omegorskaya/

https://x.com/TolokaAI

Ksenia and Turing Post:

https://x.com/TheTuringPost

https://www.linkedin.com/in/ksenia-se

https://huggingface.co/Kseniase