Token Bucket
A rate limiting algorithm that adds tokens to a bucket at a fixed rate; each request consumes one token, and the bucket capacity sets the upper bound on permitted bursts, making it suited to scenarios that tolerate short spikes.
Core Facts
Timeline (last 90 days)
The token bucket allows overdraft: when two threads take tokens simultaneously, the second sleeps longer, thereby shortening lock hold time and maintaining the correct pace.
External API rate limiting manifests as returned 429 status codes or specific quota errors, requiring token-bucket-style rate control on the client side.
The token bucket rate limiting algorithm refills tokens at a fixed rate, allowing a certain degree of burst traffic.
In distributed deployments, token state must be stored in shared storage such as Redis and updated with atomic operations; otherwise, multiple instances counting separately will cause the actual allowed volume to far exceed expectations.
For multi-agent frameworks such as LangGraph, AutoGen, and CrewAI, agent node calls are often triggered concurrently within the same event loop, making it especially necessary to introduce a rate limiter based on the token bucket or leaky bucket algorithm.
All Facts (7)
The core difference between token bucket and leaky bucket is that the token bucket allows short bursts when sufficient tokens have accumulated, while the leaky bucket outputs requests at a constant rate.
70%Unverified令牌桶（Token Bucket）算法在AI SaaS场景中最为常用，因为它既能限制平均速率又允许合理的突发请求
70%UnverifiedThe token bucket allows overdraft: when two threads take tokens simultaneously, the second sleeps longer, thereby shortening lock hold time and maintaining the correct pace.
50%UnverifiedExternal API rate limiting manifests as returned 429 status codes or specific quota errors, requiring token-bucket-style rate control on the client side.
50%UnverifiedThe token bucket rate limiting algorithm refills tokens at a fixed rate, allowing a certain degree of burst traffic.
50%UnverifiedIn distributed deployments, token state must be stored in shared storage such as Redis and updated with atomic operations; otherwise, multiple instances counting separately will cause the actual allowed volume to far exceed expectations.
50%UnverifiedFor multi-agent frameworks such as LangGraph, AutoGen, and CrewAI, agent node calls are often triggered concurrently within the same event loop, making it especially necessary to introduce a rate limiter based on the token bucket or leaky bucket algorithm.
50%