On-device models and offline inference tradeoffs

From applied cryptography and offensive security in France’s defense industry to optimizing nuclear submarine workflows, then selling his e-signature startup to Docusign (https://www.docusign.com/company/news-center/opentrust-joins-docusign-global-trust-network and now running AI as CTO of Superhuman Mail (Superhuman, recently acquired by Grammarly https://techcrunch.com/2025/07/01/grammarly-acquires-ai-email-client-superhuman/), Loïc Houssier has lived the full arc from deep infra and compliance hell to obsessing over 100ms product experiences and AI-native email. We sat down with Loïc to dig into how you actually put AI into an inbox without adding latency, why Superhuman leans so hard into agentic search and “Ask AI” over your entire email history, how they design tools vs. agents and fight agent laziness, what box-priced inference and local-first caching mean for cost and reliability, and his bet that your inbox will power your future AI EA while AI massively widens the gap between engineers with real fundamentals and those faking it.

We discuss:

• Loïc’s path from applied cryptography and offensive security in France’s defense industry to submarines, e-signatures, Docusign, and now Superhuman Mail

• What 3,000+ engineers actually do at a “simple” product like Docusign: regional compliance, on-prem appliances, and why global scale explodes complexity

• How Superhuman thinks about AI in email: auto-labels, smart summaries, follow-up nudges, “Ask AI” search, and the rule that AI must never add latency or friction

• Superhuman’s agentic framework: tools vs. agents, fighting “agent laziness,” deep semantic search over huge inboxes, and pagination strategies to find the real needle in the haystack

• How they evaluate OpenAI, Anthropic, Gemini, and open models: canonical queries, end-to-end evals, date reasoning, and Rahul’s infamous “what wood was my table?” test

• Infra and cost philosophy: local-first caching, vector search backends, Baseten “box” pricing vs. per-token pricing, and thinking in price-per-trillion-tokens instead of price-per-million

• The vision of Superhuman as your AI EA: auto-drafting replies in your voice, scheduling on your behalf, and using your inbox as the ultimate private data source

• How the Grammarly + Coda + Superhuman stack could power truly context-aware assistance across email, docs, calendars, contracts, and more

• Inside Superhuman’s AI-dev culture: free-for-all tool adoption, tracking AI usage on PRs, and going from ~4 to ~6 PRs per engineer per week

• Why Loïc believes everyone should still learn to code, and how AI will amplify great engineers with strong fundamentals while exposing shallow ones even faster

—

Loïc Houssier

• LinkedIn: https://www.linkedin.com/in/houssier/

Where to find Latent Space

• X: https://x.com/latentspacepod

• Substack: https://www.latent.space/

Chapters

• 00:00:00 Introduction and Loïc's Journey from Nuclear Submarines to Superhuman
• 00:06:40 Docusign Acquisition and the Enterprise Email Stack
• 00:10:26 Superhuman's AI Vision: Your Inbox as the Real AI Agent
• 00:13:20 Ask AI: Agentic Search and the Quality Problem
• 00:18:20 Infrastructure Choices: Model Selection, Base10, and Cost Management
• 00:27:30 Local-First Architecture and the Database Stack
• 00:30:50 Evals, Quality, and the Rahul Wood Table Test
• 00:42:30 The Future EA: Auto-Drafting and Proactive Assistance
• 00:46:40 Grammarly Acquisition and the Contextual Advantage
• 00:38:40 Voice, Video, and the End of Writing
• 00:51:40 Knowledge Graphs: The Hard Problem Nobody Has Solved
• 00:56:40 Competing with OpenAI and the Browser Question
• 01:02:30 AI Coding Tools: From 4 to 6 PRs Per Week
• 01:08:00 Engineering Culture, Hiring, and the Future of Software Development