Unverified50% confidenceFactExact time
In LLM API scenarios, 502/503 usually correspond to gateway backend unavailability or overload, 504 corresponds to inference queue timeout, and 500 is mostly an internal exception.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe failure modes of LLM services are broader than those of traditional services, including overall provider unavailability, regional rate limiting, exhausted key quota, silent model upgrades causing output quality drift, and latency slowly creeping up rather than directly erroring; most of these scenarios do not trigger traditional 5xx alerts.65% similarUnverifiedHaving an LLM call REST APIs directly leads to three practical problems: increased token consumption and latency, context pollution, and unfriendly error semantics (e.g., HTTP 422 status codes are hard for LLMs to understand).63% similarUnverifiedFor large model APIs, retryable errors include 429 rate limiting, 5xx errors such as 500/503, and connection timeouts; non-retryable errors include 400 parameter errors and 401 authentication failures.63% similarUnverifiedA 200 status code combined with a stream interruption or parse failure can be attributed to overload of the relay station's billing/filtering middleware; 502/504 can be attributed to failure of the relay station's gateway nodes; 429/401 can be attributed to failure of concurrency control or exhaustion of the upstream key's balance.63% similarUnverifiedLength bands divide the request token count into discrete intervals (such as <50, 50-500, >500) rather than continuous values, because the routing decision's sensitivity to length is step-like.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1104096API
curl https://kongchang.com/api/v1/knowledge/claims/1104096MCP
get_claim(id=1104096)