SF Compute: Commoditizing Compute
Latent Space: The AI Engineer Podcast
Intro
This chapter delves into the rise of SF Compute in the San Francisco tech scene, spotlighting CoreWeave's successful IPO and its long-term contract approach. The discussion contrasts GPU and CPU cloud models, showcasing the benefits of commodity hardware for improved financial strategy and risk mitigation.
Evan Conrad, co-founder of SF Compute, joined us to talk about how they started as an AI lab that avoided bankruptcy by selling GPU clusters, why CoreWeave financials look like a real estate business, and how GPUs are turning into a commodities market.
Chapters:
00:00:05 - Introductions
00:00:12 - Introduction of guest Evan Conrad from SF Compute
00:00:12 - CoreWeave Business Model Discussion
00:05:37 - CoreWeave as a Real Estate Business
00:08:59 - Interest Rate Risk and GPU Market Strategy Framework
00:16:33 - Why Together and DigitalOcean will lose money on their clusters
00:20:37 - SF Compute's AI Lab Origins
00:25:49 - Utilization Rates and Benefits of SF Compute Market Model
00:30:00 - H100 GPU Glut, Supply Chain Issues, and Future Demand Forecast
00:34:00 - P2P GPU networks
00:36:50 - Customer stories
00:38:23 - VC-Provided GPU Clusters and Credit Risk Arbitrage
00:41:58 - Market Pricing Dynamics and Preemptible GPU Pricing Model
00:48:00 - Future Plans for Financialization?
00:52:59 - Cluster auditing and quality control
00:58:00 - Futures Contracts for GPUs
01:01:20 - Branding and Aesthetic Choices Behind SF Compute
01:06:30 - Lessons from Previous Startups
01:09:07 - Hiring at SF Compute
Chapters
- 00:00:00 Introduction and Background
- 00:00:58 Analysis of GPU Business Models
- 00:01:53 Challenges with GPU Pricing
- 00:02:48 Revenue and Scaling with GPUs
- 00:03:46 Customer Sensitivity to GPU Pricing
- 00:04:44 Core Weave's Business Strategy
- 00:05:41 Core Weave's Market Perception
- 00:06:40 Hyperscalers and GPU Market Dynamics
- 00:07:37 Financial Strategies for GPU Sales
- 00:08:35 Interest Rates and GPU Market Risks
- 00:09:30 Optimal GPU Contract Strategies
- 00:10:27 Risks in GPU Market Contracts
- 00:11:25 Price Sensitivity and Market Competition
- 00:12:21 Market Dynamics and GPU Contracts
- 00:13:18 Hyperscalers and GPU Market Strategies
- 00:14:15 Nvidia and Market Competition
- 00:15:12 Microsoft's Role in GPU Market
- 00:16:10 Challenges in GPU Market Dynamics
- 00:17:07 Economic Realities of the GPU Market
- 00:18:03 Real Estate Model for GPU Clouds
- 00:18:59 Price Sensitivity and Chip Design
- 00:19:55 SF Compute's Beginnings and Challenges
- 00:20:54 Navigating the GPU Market
- 00:21:54 Pivoting to a GPU Cloud Provider
- 00:22:53 Building a GPU Market
- 00:23:52 SF Compute as a GPU Marketplace
- 00:24:49 Market Liquidity and GPU Pricing
- 00:25:47 Utilization Rates in GPU Markets
- 00:26:44 Brokerage and Market Flexibility
- 00:27:42 H100 Glut and Market Cycles
- 00:28:40 Supply Chain Challenges and GPU Glut
- 00:29:35 Future Predictions for the GPU Market
- 00:30:33 Speculations on Test Time Inference
- 00:31:29 Market Demand and Test Time Inference
- 00:32:26 Open Source vs. Closed AI Demand
- 00:33:24 Future of Inference Demand
- 00:34:24 Peer-to-Peer GPU Markets
- 00:35:17 Decentralized GPU Market Skepticism
- 00:36:15 Redesigning Architectures for New Markets
- 00:37:14 Supporting Grad Students and Startups
- 00:38:11 Successful Startups Using SF Compute
- 00:39:11 VCs and GPU Infrastructure
- 00:40:09 VCs as GPU Credit Transformators
- 00:41:06 Market Timing and GPU Infrastructure
- 00:42:02 Understanding GPU Pricing Dynamics
- 00:43:01 Market Pricing and Preemptible Compute
- 00:43:55 Price Volatility and Market Optimization
- 00:44:52 Customizing Compute Contracts
- 00:45:50 Creating Flexible Compute Guarantees
- 00:46:45 Financialization of GPU Markets
- 00:47:44 Building a Spot Market for GPUs
- 00:48:40 Auditing and Standardizing Clusters
- 00:49:40 Ensuring Cluster Reliability
- 00:50:36 Active Monitoring and Refunds
- 00:51:33 Automating Customer Refunds
- 00:52:33 Challenges in Cluster Maintenance
- 00:53:29 Remote Cluster Management
- 00:54:29 Standardizing Compute Contracts
- 00:55:28 Unified Infrastructure for Clusters
- 00:56:24 Creating a Commodity Market for GPUs
- 00:57:22 Futures Market and Risk Management
- 00:58:18 Reducing Risk with GPU Futures
- 00:59:14 Stabilizing the GPU Market
- 01:00:10 SF Compute's Anti-Hype Approach
- 01:01:07 Calm Branding and Expectations
- 01:02:07 Promoting San Francisco's Beauty
- 01:03:03 Design Philosophy at SF Compute
- 01:04:02 Artistic Influence on Branding
- 01:05:00 Past Projects and Burnout
- 01:05:59 Challenges in Building an Email Client
- 01:06:57 Persistence and Iteration in Startups
- 01:07:57 Email Market Challenges
- 01:08:53 SF Compute Job Opportunities
- 01:09:53 Hiring for Systems Engineering
- 01:10:50 Financial Systems Engineering Role
- 01:11:50 Conclusion and Farewell
Remember Everything You Learn from Podcasts
