Cascading Strategy
A cost optimization strategy in LLM production systems that operates sequentially: requests are first handled by a small model, and if a verifier judges the result unreliable, they are escalated to a stronger model for retry. Cost is amortized by reducing calls to the large model.
Core Facts
Timeline (last 90 days)
When committing to 90% accuracy, a cascade strategy with a verifier costs only 28% of what always using the large model costs.
Cascading is a sequential execution strategy in which a request first goes to a low-cost model, and if its confidence or a verifier determines the result is unreliable, the same request is escalated to a stronger model for a retry.
All Facts (2)
Cascading is a sequential execution strategy in which a request first goes to a low-cost model, and if its confidence or a verifier determines the result is unreliable, the same request is escalated to a stronger model for a retry.
75%UnverifiedWhen committing to 90% accuracy, a cascade strategy with a verifier costs only 28% of what always using the large model costs.
50%