S5E02 - Topic Smorgasbord
Spring Office Hours
00:00
Prompt caching to save token costs
Dan outlines caching system prompts, tool lists, and other stable payloads to reduce token usage in high-throughput apps.
Play episode from 37:55
Transcript
Dan outlines caching system prompts, tool lists, and other stable payloads to reduce token usage in high-throughput apps.
