Telnyx AI Gateway
An AI gateway product from Telnyx that provides model access control, retry and fallback policies, budget management and observability features, and is compatible with the OpenAI API.
Timeline (last 90 days)
Telnyx AI Gateway centralizes retry and fallback policies at the gateway layer and makes request processing observable through response headers and a usage ledger.
Telnyx AI Gateway's architecture divides the system into a management plane, an inference plane, and an observation plane; the management plane uses account API keys to manage token groups, token keys, budgets, and usage reports, while the inference plane is OpenAI API compatible and uses only token keys.
A token group is a named collection of a set of inference access rules and budgets in Telnyx AI Gateway, which can declare allowed model aliases, fallback relationships, retry parameters, and budget limits and periods.
A token key is a key issued to a token group, which applications use to access the OpenAI-compatible inference endpoint without holding an account-level API key; the token group simultaneously serves as a permission boundary and a policy container.
When using Telnyx AI Gateway, the OpenAI client should set max_retries=0 to disable its own retries, because SDK retries would stack on top of gateway attempts, inflating the request count and distorting the attempt count.
Telnyx AI Gateway retries the same model for definite no-work failures (such as pre-dispatch errors, 429, 529); moves to the next fallback model for uncertain failures (such as timeouts, 408, 500, 502, 503, 504); and terminates directly on context-length rejections.
Telnyx AI Gateway provides x-ltg-served-model and x-ltg-attempts in response headers, identifying the model that actually handled the request and the number of attempts, respectively.
Telnyx AI Gateway's usage ledger events include fields such as requested_model, model, attempts, status, and cache_hit, providing a persistent request-level record corresponding to the response headers.
The GET /usage/summary endpoint requires the account to have the llm_token_gateway.usage.read permission to return totals and per-model summaries; a 403 should be treated as a non-fatal warning, because the usage event ledger still contains request-level proof.
Telnyx AI Gateway supports request-level overrides for attempt counts, latency, backoff, and timeouts (such as the x-ltg-request-timeout and x-ltg-max-attempts request headers), allowing a single request to be adjusted without rewriting the token group policy.
1 more timeline events
All Facts (11)
Telnyx AI Gateway centralizes retry and fallback policies at the gateway layer and makes request processing observable through response headers and a usage ledger.
50%UnverifiedTelnyx AI Gateway's architecture divides the system into a management plane, an inference plane, and an observation plane; the management plane uses account API keys to manage token groups, token keys, budgets, and usage reports, while the inference plane is OpenAI API compatible and uses only token keys.
50%UnverifiedA token group is a named collection of a set of inference access rules and budgets in Telnyx AI Gateway, which can declare allowed model aliases, fallback relationships, retry parameters, and budget limits and periods.
50%UnverifiedA token key is a key issued to a token group, which applications use to access the OpenAI-compatible inference endpoint without holding an account-level API key; the token group simultaneously serves as a permission boundary and a policy container.
50%UnverifiedWhen using Telnyx AI Gateway, the OpenAI client should set max_retries=0 to disable its own retries, because SDK retries would stack on top of gateway attempts, inflating the request count and distorting the attempt count.
50%UnverifiedTelnyx AI Gateway retries the same model for definite no-work failures (such as pre-dispatch errors, 429, 529); moves to the next fallback model for uncertain failures (such as timeouts, 408, 500, 502, 503, 504); and terminates directly on context-length rejections.
50%UnverifiedTelnyx AI Gateway provides x-ltg-served-model and x-ltg-attempts in response headers, identifying the model that actually handled the request and the number of attempts, respectively.
50%UnverifiedTelnyx AI Gateway's usage ledger events include fields such as requested_model, model, attempts, status, and cache_hit, providing a persistent request-level record corresponding to the response headers.
50%UnverifiedThe GET /usage/summary endpoint requires the account to have the llm_token_gateway.usage.read permission to return totals and per-model summaries; a 403 should be treated as a non-fatal warning, because the usage event ledger still contains request-level proof.
50%UnverifiedTelnyx AI Gateway supports request-level overrides for attempt counts, latency, backoff, and timeouts (such as the x-ltg-request-timeout and x-ltg-max-attempts request headers), allowing a single request to be adjusted without rewriting the token group policy.
50%UnverifiedTelnyx AI Gateway's cache is best-effort, and the example verifies cache behavior by sending a repeated prompt and checking the cache_hit field, rather than fabricating cache hits.
50%