Building Voice Agents with Luke Harries of ElevenLabs

Discover how to build powerful AI agents that integrate with your personal communication, as Luke Harris, Head of Growth at ElevenLabs, shares his journey and groundbreaking projects. Luke, a true tinkerer, reveals how he built the most popular WhatsApp MTP server and an ingenious iPhone shortcut for superior voice transcription.

What you’ll learn:

• The surprising gap in consumer AI and how to build solutions for it.

• Practical insights into building secure AI agent systems and managing data privacy.

• How ElevenLabs’ new speech-to-text (Scribe) and conversational AI APIs enable advanced voice agents.

• The importance of launching personal projects and the unexpected opportunities they create.

• Emerging growth areas for AI in education and niche industries.

Luke Harris leads growth at ElevenLabs, a leading company in generative voice AI. With a background spanning software engineering, biotech ML, and entrepreneurship, Luke is a prolific builder and a Y Combinator alum.

Key topics covered:

• Building the WhatsApp MTP server and its technical challenges.

• The power of Apple Shortcuts for AI-powered mobile workflows.

• ElevenLabs’ Scribe model for highly accurate speech-to-text and diarization.

• Designing conversational AI agents with low latency and emotional expressiveness.

• AI-first workflows for product development, content creation, and sales intelligence.

This episode of AI Tinkerers One-Shot goes deep under the hood with a builder at the forefront of voice AI.

💡 Resources:

• ElevenLabs - https://elevenlabs.io

• Luke Harris’s Blog - https://harrys.co

• Luke Harris’s LinkedIn - linkedin.com/in/luke-harris-ai

• AI Tinkerers - https://aitinkerers.org

• One-Shot Podcast - https://aitinkerers.org/podcast

👍 Like this video if you found it valuable, and subscribe to AI Tinkerers One-Shot for more conversations with innovators building the future of AI!

00:00 Introduction to Luke Harris

00:01:51 Luke’s Background and Growth at ElevenLabs

00:04:59 The WhatsApp MTP Server Project

00:11:17 Data Privacy and Security in AI Agents

00:16:15 Automation vs. Human-in-the-Loop

00:27:17 ElevenLabs Speech-to-Text and iPhone Shortcut

00:31:34 ElevenLabs Scribe: Advanced Transcription Features

00:35:03 Conversational AI and Voice Agents

00:44:08 Building Voice Agents with ElevenLabs

00:51:41 Unexpected Growth Drivers for Voice AI

01:00:15 Luke’s AI-First Workflow and Resources

01:05:22 Keeping Up with AI & AI Tinkerers Community