A team caches a 200-token system prompt. Cache hits return zero cost reduction in their metrics. What is the most likely explanation?
Answer: The prompt is below the minimum cacheable size threshold
Prompt caching has a minimum input size (~1024 tokens depending on model). Caching a 200-token prompt produces no benefit because the prompt is too small to qualify. The mitigation is to either cache only when the prefix exceeds the threshold or use a larger reference template.