Importance of Latency, Model Evaluation, and Internal Tool Development in Software

On this week’s Unsupervised Learning, Pat and I sat down with CTO and Co-Founder of Sourcegraph, Beyang Liu. Sourcegraph is a leader in the AI coding space, and recently launched AI coding assistant, Cody. Beyang shared with us his view on the current landscape of AI coding and the future of coding and software development. He also shared how Sourcegraph has tried to make RAG better, and their model eval approaches.

(0:00) intro
(0:47) advice for young coders
(3:34) AI products at Sourcegraph
(6:17) the current state of AI coding
(12:33) what happens when a new GPT model comes out?
(20:16) what types of developers benefit from these AI tools?
(30:45) how important is inference cost?
(35:31) how does Sourcegraph structure AI teams?
(41:27) what metrics does Sourcegraph use to evaluate their products?
(50:02) customizing RAG
(56:55) getting ahead of the agentic future
(1:05:05) will there be more or less engineers in the future?
(1:13:50) over-hyped/under-hyped
(1:16:56) surprises during the Sourcegraph journey
(1:18:26) cognition buzz and Devin
(1:26:48) Jacob and Pat debrief

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint