Closing the Loop with Andrew Camel

Reinventing Zapier for the LLM era With Mike Knoop, Co-Founder & Head of AI at Zapier

Sep 8, 2023

Mike Knoop, Co-Founder & Head of AI at Zapier, discusses how LLMs are transforming Zapier's business, driving 100x productivity improvements and deflecting customer support requests. They explore the role of LLMs in tool discovery, optimizing apps, and handling customer interactions. They discuss the vision for Zapier in this new world, challenges in building LLM-powered features, and evaluating model outputs. The episode also explores using language models for debugging, generating data, and improving error messages.

48:27

Creator website

Episode guests

Mike Knoop

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

LLMs are already transforming Zapier's business by improving internal processes and reducing customer support requests.

Zapier views LLMs as the "escape hatch" UX paradigm that will enable it to grow from 10M to 100M users.

Zapier has successfully used LLMs for generating zap templates, handling support tickets, and optimizing tool discovery.

Deep dives

The Journey of Zapier's Head of AI: LLMs, Use Cases, and Growth

Mike Neup, co-founder and head of AI at Zapier, discusses the journey of Zapier's adoption of LLMs and their potential impact on the company's growth. He emphasizes the importance of experimenting with use cases and form factors in all aspects of the business. Neup highlights the progress made by his team in finding material ROI in customer support, go-to-market motion, product building, and implementation challenges. He also explores how LLMs can contribute to an evolution in Zapier's UI/UX paradigm, aiming to reach 100 million users in the next decade.

Introduction

4min

Language models and automation

25min

Improving Error Messages and the Role of Language Models

4min

Using Code Interpretation, Validating JSON, and Generating Data with Offline Models

6min

Challenges of Evaluating and Comparing Model Outputs

3min

Building a Grammar for Every Possible Action on Zapier

5min

LLMs are already transforming Zapier's business. They're seeing 100x productivity improvements on internal processes that drive user acquisition, deflecting a large volume of customer support requests by improving the handling of user-facing error messages. But most interesting is that Zapier views LLMs as the "escape hatch" UX paradigm that will enable it to grow from 10M users today, to 100M users over the next decade.

Topics Covered:

(1:27) Mike's journey to taking the role of Head of Zapier AI

(4:16) Fast internal LLM adoption at Zapier

(6:54) What were the specific use cases driving the growth?

(11:05) If AI/bots are the ones trying to do tool discovery, will that change the role of SEO?

(13:09) Is it also important to vendors to optimize their apps to ensure Zapier is picking them to accomplish a given tool?

(14:12) Use case of handling customer interactions; what have you seen there?

(16:46) Summarization of customer info... will this be owned by Intercom/Zendesk, or should it exist as an independent capability?

(19:04) How do you think about the vision for Zapier in this new world?

(22:30) Where do you see LLMs adding most value?

(27:00) Do you think you need to capture users earlier in the funnel?

(28:18) What problems have you encountered building these LLM-powered features, solved or unsolved?

(35:52) Any tools you've found valuable to handle fine tuning, quality management, etc.?

(40:07) How do you actually evaluate the outputs / measure similarity?

(45:16) Do AI agents necessitate lower latency?

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Closing the Loop with Andrew Camel

Reinventing Zapier for the LLM era With Mike Knoop, Co-Founder & Head of AI at Zapier

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

The Journey of Zapier's Head of AI: LLMs, Use Cases, and Growth

Fast Adoption of LLMs and Their Role in Zapier's Growth

Fine-Tuning Models, Quality Management, and the Future of Zapier

Zapier's long-term vision and growth

Language models as a key to Zapier's growth

Remember Everything You Learn from Podcasts