EP23: Building Open Source AI Frameworks: David Mezzetti on TxtAI and Local-First AI

Feb 1, 2026

David Mezzetti, creator of TextAI and solo developer of an open-source AI orchestration library focused on local-first and small-model workflows. He discusses why local-first AI matters, how COVID research led to semantic search, the power of tiny models on CPU, evolving RAG and orchestration, and the trade-offs of resisting then embracing cloud APIs.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Local-First AI Often Matches Cloud

David Mezzetti argues local-first AI often matches or beats cloud APIs for embeddings and speed.
Running models locally preserves control, privacy, and reproducibility for domain-specific data.

ADVICE

Start With Tiny Models On CPU

Try small models first: a 20M-parameter model can solve many embedding tasks on CPU.
Run lightweight models in containers or VMs before opting for complex GPU setups.

ANECDOTE

TextAI Emerged From COVID Research

David launched TextAI during COVID after doing literature research and building extractive QA.
A Reddit demo of semantic search drove early traction by showing results without keyword overlap.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

David Mezzetti, creator of TxtAI, joins us to talk about building open source AI frameworks as a solo developer - and why local-first AI still matters in the age of API-everything.

David's path from running a 50-person IT company through acquisition to building one of the most well-regarded AI orchestration libraries tells you how sometimes constraints breed better design. TextAI started during COVID when he was doing coronavirus literature research and realized semantic search could transform how we find information.

We get into the evolution of the AI framework landscape - from the early days of vector embeddings to RAG to LLM orchestration. David was initially stubborn about not supporting OpenAI's API, wanting to keep everything local. He admits that probably cost him some early traction compared to LangChain, but it also shaped TextAI's philosophy: you shouldn't need permission to build with AI.

We also talk about small models and some genuinely practical insights: a 20-million parameter model running on CPU might be all you need. On the future of coding with AI, David's come around on "vibe coding" and notes that well-documented frameworks with lots of examples are perfectly positioned for this new world.

Takeaways:

Local-first AI gives you control, reproducibility, and often better performance for your domain
Small models (even 20M parameters) can solve real problems on CPU
Good documentation and examples make your framework AI-coding friendly
Open source should mean actually contributing - not just publishing code
Solo developers can compete by staying focused and being willing to evolve

Timeline:

(00:14) Introduction and David's Background

(07:44) TextAI History and Evolution

(12:04) Framework Landscape: LangChain, LlamaIndex, Haystack

(15:16) Can AI Re-implement Frameworks?

(24:14) API Specs: OpenAI vs Anthropic

(26:46) Running an Open Source Consulting Business

(32:51) Origin Story: COVID, Kaggle, and Medical Literature

(43:08) Open Source Philosophy and Giving Back

(47:16) Ethics of Local AI and Developer Freedom

(01:06:44) Human in the Loop and AI-Generated Code

(01:09:31) The Future of Work and Automation

Music:

"Kid Kodi" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.
"Palms Down" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0. Changes: trimmed

About:

The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.