Fluid Compute: Vercel’s Next Step in the Evolution of Serverless?

Feb 13, 2025

Mariano Cocirio, Staff Product Manager at Vercel, dives into Fluid Compute, an innovative cloud computing model that revolutionizes serverless applications. He discusses how this model tackles AI workload challenges by optimizing resource management and reducing costs related to idle time. Mariano highlights that developers can adopt Fluid Compute without major changes, while still benefiting from improved performance and scalability. The conversation also sheds light on the role of observability tools in maximizing efficiency and managing costs.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Fluid Compute Solves AI Idle Cost Problem

Traditional serverless struggles with AI workloads due to idle costs during long processing times.
Fluid Compute offers a solution by combining serverless scalability with server efficiency, enabling single instances to handle multiple requests.

INSIGHT

Advantages Over Servers

Fluid Compute addresses the limitations of serverless for AI without reverting to traditional servers.
The key benefit of Fluid Compute is the combination of infinite scaling and scaling to zero.

ANECDOTE

Significant Cost Savings with Fluid Compute

Some Vercel customers have reported up to 85% cost savings after enabling Fluid Compute.
This is achieved by efficiently utilizing idle time during long AI processing tasks by allowing instances to handle concurrent requests.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this episode of the Modern Web Podcast, hosts Rob Ocel and Danny Thompson sit down with Mariano Cocirio, Staff Product Manager at Vercel, to discuss Fluid Compute, a new cloud computing model that blends the best of serverless scalability with traditional server efficiency. They explore the challenges of AI workloads in serverless environments, the high costs of idle time, and how Fluid Compute optimizes execution to reduce costs while maintaining performance. Mariano explains how this approach allows instances to handle multiple requests efficiently while still scaling to zero when not in use. The conversation also covers what developers need to consider when adopting this model, the impact on application architecture, and how to track efficiency gains using Vercel’s observability tools.Is Fluid Compute the next step in the evolution of serverless? Is it redefining cloud infrastructure altogether?

Keypoints

Fluid Compute merges the best of servers and serverless – It combines the scalability of serverless with the efficiency and reusability of traditional servers, allowing instances to handle multiple requests while still scaling down to zero.
AI workloads struggle with traditional serverless models – Serverless is optimized for quick, stateless functions, but AI models often require long processing times, leading to high costs for idle time. Fluid Compute solves this by dynamically managing resources.
No major changes required for developers – Fluid Compute works like a standard Node or Python server, meaning developers don’t need to change their code significantly. The only consideration is handling shared global state, similar to a traditional server environment.
Significant cost savings and efficiency improvements – Vercel’s observability tools show real-time reductions in compute costs, with some early adopters seeing up to 85% savings simply by enabling Fluid Compute.

Chapters

0:00 – Introduction and Guest Welcome

1:08 – What is Fluid Compute? Overview and Key Features

2:08 – Why Serverless Compute Struggles with AI Workloads

4:00 – Fluid Compute: Combining Scalability and Efficiency

6:04 – Cost Savings and Real-world Impact of Fluid Compute

8:12 – Developer Experience and Implementation Considerations

10:26 – Managing Global State and Concurrency in Fluid Compute

13:09 – Observability Tools for Performance and Cost Monitoring

20:01 – Long-running Instances and Post-operation Execution

24:02 – Evolution of Compute Models: From Servers to Fluid Compute

29:08 – The Future of Fluid Compute and Web Development

30:15 – How to Enable Fluid Compute on Vercel

32:04 – Closing Remarks and Guest Social Media Info

Follow Mariano Cocirio on Social Media:Twitter:https://x.com/mcocirio

Linkedin:https://www.linkedin.com/in/mcocirio/