LLM API

API billing typically distinguishes between input tokens and output tokens; in multi-turn conversations, the outputs from earlier turns become inputs in later turns and are billed again.

The request-response cycle handled by a traditional API gateway usually completes within 50 to 200 milliseconds, whereas an LLM API often needs to keep the connection open for 30 to 120 seconds.

LLM APIs are billed by token, with approximately 1–2 tokens per English word and approximately 1–2 tokens per Chinese character.

Since 2023, API prices for mainstream LLMs have fallen by more than 90%

调用第三方LLM API的AI产品数据流向通常经历「用户设备→产品服务器→LLM服务商」三个节点，每个节点都存在独立的安全风险面

A local agent places the execution layer on the user's own computer; after command parsing it directly calls the local file system and operating system interfaces, so data need not leave the machine, but limited by local compute it usually relies on an external LLM API for semantic understanding.

The LLM API is the source of the desktop pet's conversational ability; the program uses an API Key to call the LLM to generate reply text, then feeds the motion intent into the T2M model to generate skeletal animation.

In LLM API scenarios, 502/503 usually correspond to gateway backend unavailability or overload, 504 corresponds to inference queue timeout, and 500 is mostly an internal exception.

The starting method for building a minimum viable Agent: choose a timer-triggered scenario, hook up a free LLM API to make simple judgments, and then output the results through a notification tool.

For LLM APIs, retryable errors include 429, 5xx and timeouts, while non-retryable errors include 400 parameter errors and 401 authentication failures.

Free-tier LLM API limits are usually distributed across four levels: request quota, response size, model capability and usage tracking.

Learning Agentic AI should start with directly using LLM APIs, mastering parameter controls such as temperature and max tokens, structured prompt engineering, and function calling mechanisms.

When calling an LLM API, the marginal cost grows linearly with the number of requests, whereas the marginal cost per execution of locally compiled decision logic is close to zero, with the main cost shifting to one-time compilation and validation investment.

Prompt caching charges only one-tenth the price for input tokens that have already been processed, making it one of the highest-leverage cost optimizations on modern LLM APIs.

Calling LLM APIs in production requires adding retries and clear failure paths, for example exponential backoff retries for HTTP 429 status codes.

The author's canary costs about $0.4 per month in LLM calls, with 720 runs, each processing just 10 tokens with one cheapest model.

The LLM API availability probe sends a 10-token prompt "reply OK" to the cheapest model and requires the response to arrive within 30 seconds, capturing overall API outages, expired keys, and slow degradation.

The UplinkSwitch aggregation platform extracted 66 chat models from 11 developers with official list prices from the real-time catalog of its own API gateway; all 66 models are below official list prices on both input and output sides, with the largest price difference reaching 95%.

LLM APIs are priced separately for input and output tokens, and the per-token price does not change with the temperature and seed parameters.

LLM APIs rate-limit on two independent dimensions, requests per minute and tokens per minute; even with monthly budget to spare, an endpoint can still return 429 during traffic peaks.