The Crucial Role of Latency in Cloud Performance

Supported by Our Partners

•⁠ Modal⁠ — The cloud platform for building AI applications

•⁠ CodeRabbit⁠⁠ — Cut code review time and bugs in half. Use the code PRAGMATIC to get one month free.

—

What happens when LLMs meet real-world codebases? In this episode of The Pragmatic Engineer,  I am joined by Varun Mohan, CEO and Co-Founder of Windsurf. Varun talks me through the technical challenges of building an AI-native IDE (Windsurf) —and how these tools are changing the way software gets built. 

We discuss: 

• What building self-driving cars taught the Windsurf team about evaluating LLMs

• How LLMs for text are missing capabilities for coding like “fill in the middle”

• How Windsurf optimizes for latency

• Windsurf’s culture of taking bets and learning from failure

• Breakthroughs that led to Cascade (agentic capabilities)

• Why the Windsurf teams build their LLMs

• How non-dev employees at Windsurf build custom SaaS apps – with Windsurf!

• How Windsurf empowers engineers to focus on more interesting problems

• The skills that will remain valuable as AI takes over more of the codebase

• And much more!

—

Timestamps

(00:00) Intro

(01:37) How Windsurf tests new models

(08:25) Windsurf’s origin story 

(13:03) The current size and scope of Windsurf

(16:04) The missing capabilities Windsurf uncovered in LLMs when used for coding

(20:40) Windsurf’s work with fine-tuning inside companies 

(24:00) Challenges developers face with Windsurf and similar tools as codebases scale

(27:06) Windsurf’s stack and an explanation of FedRAMP compliance

(29:22) How Windsurf protects latency and the problems with local data that remain unsolved

(33:40) Windsurf’s processes for indexing code 

(37:50) How Windsurf manages data 

(40:00) The pros and cons of embedding databases 

(42:15) “The split brain situation”—how Windsurf balances present and long-term 

(44:10) Why Windsurf embraces failure and the learnings that come from it

(46:30) Breakthroughs that fueled Cascade

(48:43) The insider’s developer mode that allows Windsurf to dogfood easily 

(50:00) Windsurf’s non-developer power user who routinely builds apps in Windsurf

(52:40) Which SaaS products won’t likely be replaced

(56:20) How engineering processes have changed at Windsurf 

(1:00:01) The fatigue that goes along with being a software engineer, and how AI tools can help

(1:02:58) Why Windsurf chose to fork VS Code and built a plugin for JetBrains 

(1:07:15) Windsurf’s language server 

(1:08:30) The current use of MCP and its shortcomings 

(1:12:50) How coding used to work in C#, and how MCP may evolve 

(1:14:05) Varun’s thoughts on vibe coding and the problems non-developers encounter

(1:19:10) The types of engineers who will remain in demand 

(1:21:10) How AI will impact the future of software development jobs and the software industry

(1:24:52) Rapid fire round

—

The Pragmatic Engineer deepdives relevant for this episode:

• IDEs with GenAI features that Software Engineers love

• AI tooling for Software Engineers in 2024: reality check

• How AI-assisted coding will change software engineering: hard truths

• AI tools for software engineers, but without the hype

—

See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠

—

Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com.