Always Running Out of Tokens? This Five-Layer Methodology Saves You Real Money

Why Your Tokens Keep Slipping Away

Have you ever done this calculation: of all the money you spend on AI models each month, how much actually turns into useful output? Overseas models are already expensive, and after OpenAI's recent model releases, some analyses suggest that token quotas for high-frequency usage tiers may face adjustments — costs will only keep rising.

A developer systematically broke down the question of "how to save and efficiently use tokens" in a Bilibili livestream. He attributed token waste to several root causes: redundant prompt information, oversized context, over-reasoning (cranking the reasoning level up to very high or even Ultra), and unnecessarily verbose output.

There's a simple metric for measuring token efficiency: useful tokens ÷ total consumption. The closer this ratio is to 1, the better — though hitting exactly 1 isn't realistic, since the model itself isn't precise enough and platforms won't let you use it that way.

Two things must be established first. First, output tokens are usually far more expensive than input tokens. Take common pricing as an example: 1 million input tokens costs about 2 yuan, while output typically runs around 8 to 10 yuan — a difference of several times over. So limiting unnecessary output has a much higher ROI than optimizing input. Second, loading too many tools eats up a lot of context. Every Skill and MCP you install has its description preloaded into the system prompt — convenient on the surface, but with high hidden costs.

The Five-Layer Methodology: From Awareness to System

The creator divides the optimization path into five layers: the awareness layer (understand where the money goes); the input/output layer (streamline); the model and tool layer (match on demand); the context layer (control bloat); and the system layer (build a habit of continuous measurement).

You don't need to do everything at once — just find the layer wasting the most right now and start there.

Layer 1: Turn "Guessing" into "Executing"

The core of input optimization isn't shorter-is-better — it's more deterministic is better.

Take calculating retention rate as an example. A vague question is "help me calculate the retention rate for each group" — the model has to guess whether it's next-day or 7-day retention, what the denominator is, how activity is defined. A clear version: 7-day retention = active users within day 7 ÷ new users on the current day; active defined as at least one app open; new defined as first registration; grouped by group id; date range specified. With this kind of input, the model doesn't need to probe and trial-and-error — what you save is real money.

System prompts also need slimming down. Phrases like "You should be professional, accurate, and organized, while answers should be concise, not verbose, and free of unnecessary explanations..." look professional, but to the model most of it is noise. Change it to "You are a data analysis assistant; output format is Markdown; language is Chinese" — shorter, clearer, and the output is actually more stable.

Structured input is also worth making a habit: list three questions with numbering, or use JSON instead of long natural-language descriptions — the model understands at lower cost. On the output side, limit fields, format, and length — if 300 words can explain it, there's no need to have the model write 600.

System Prompt is the fixed instruction the model receives first at every inference, usually preset by the platform or customized by the user, defining the model's role, output format, and behavioral boundaries. It doesn't change across conversation turns but continuously occupies input tokens. Therefore, a system prompt should retain only the necessary role definition and hard constraints, rather than piling on generic adjectives like "professional, accurate, organized" — such descriptions have weak influence on model behavior yet raise the base cost of every call. Trimming the system prompt is one of the most directly rewarding actions in input optimization.

Layer 2: Model Tiering and Tool Streamlining

Not every task deserves the strongest model. Use strong models for complex problems, medium models for moderate difficulty, and small models — or even one or two generations back — for simple tasks. Strong models consume tokens faster and cost more per unit, with prices decreasing from strong to weak.

Deterministic operations — queries, filtering, computation, caching — don't need the model to read everything and compute each time. Have the model generate a one-off execution script embedded in the pipeline; subsequent data sources are handled directly by code — faster and cheaper.

At the tool level there's a counterintuitive conclusion: more capabilities doesn't equal higher efficiency. The Skills or MCPs you use frequently day to day, by rule of thumb, usually don't exceed 10 — the rest can be turned off or uninstalled by default. Specifying a fixed toolset saves considerable input overhead compared to mounting everything.

MCP (Model Context Protocol) is an open protocol that lets models call external tools and data sources in a standardized way. Each MCP service exposes a list of tools and their descriptions to the model, and these descriptions are preloaded into the system prompt. The more MCPs mounted, the more bloated the input context — and the more the model hesitates or picks wrong when selecting tools. Therefore, enabling only the necessary MCPs per task type and disabling long-unused services is a key means of controlling hidden token overhead.

Layer 3: Context Bloat Is the Real Enemy

Many people assume context only contains their own input and output, but in reality the model continuously retrieves information from browsers, knowledge bases, and code repositories into context while processing tasks — a process that itself consumes enormous tokens.

One task, one session is the easiest practice to adopt. Code implementation, requirements analysis, and deployment all strung together in one conversation means every interaction re-inserts the entire history — yet the deployment step doesn't need the large chunks of code generation at all. Split them up and the effect is immediate.

Reusing cache depends on stable prefixes and tool ordering. If you use a Skill called ABC and midway change it to ABCD, the cache is invalidated, the model must reload everything, and prior information gets inserted again. Keep names, prefixes, and tool ordering as unchanged as possible.

When a very long task approaches the context limit, have the model do summary compression first, retain the conclusions, then open a new session to continue — rather than replaying the whole process over and over.

KV Cache (Key-Value Cache) is the mechanism that reuses already-processed context during inference: when a new request's prefix exactly matches a previous one, the model doesn't need to recompute that portion, reducing latency and cost. This is why "stable prefixes" save money — once the system prompt, tool ordering, or Skill names change, the cache is invalidated and the model must recompute from scratch. Most platforms have short cache validity (commonly 5 to 10 minutes), so consecutive follow-up questions are more likely to hit the cache than asking again much later, while frequently changing the opening content zeroes out cache benefits.

A Few Tools Worth Installing

The tools the creator recommends are all designed for automatic invocation by the model — users only need to install them, not operate them manually.

Context compression tools are represented by kintestmode. Official data shows it keeps raw data out of context as much as possible, with refined results reducing context usage by about 98%; post-compression recovery efficiency is six times that of traditional methods (knowledge base tracking, file editing, error recovery); and having the model generate computation scripts rather than putting entire datasets into context yields a 100x improvement. It suits "large input, small conclusion" scenarios like Playwright browser calls, CSV tables, and log output.

Code and knowledge retrieval tools are represented by CodeGraph. The traditional approach is repeated grep reads and character-by-character traversal, with tokens bleeding away heavily during code search. CodeGraph builds fast query capability through a local index — information isn't uploaded, and after installation you just run initialization; it provides CLI and MCP capabilities by default for local Agents. It's most valuable for fixing bugs in legacy projects and structural Q&A in large repositories (e.g., "how does the order flow work," "where does the cache get invalidated"). Note: static analysis isn't runtime truth — dynamic imports, reflection, and similar mechanisms have blind spots, so code involving these still needs human judgment.

Output compression tools like RTK and headroom compress diffs, tests, logs, and other output; consumption observability tools like tokenlizer help you see budget distribution and cache hit rates, pinpointing the worst waste.

30-Second Self-Check: Which Layer Is Your Waste On

The livestream gave a four-item self-check list:

1. Context bloat — a single conversation exceeding 10 turns, or a single task exceeding 15k tokens, or mixing multiple tasks
2. Tool overload — low cache hit rate, too many Skills and MCPs installed
3. Verbose output — output not in the specified format, loaded with explanatory text, when you actually only needed a third of the content
4. Model overuse — simple tasks calling the strongest model

In the livestream's real-time feedback, context bloat (item one) was mentioned most. The corresponding fix is straightforward: don't mix multiple tasks; if a task really is long, summarize first and open a new session. The fix for tool overload is to disable non-frequent tools by default. For cache hits, mind the timing — some models' cache is valid for 5 or 10 minutes, so follow up quickly after the previous answer to hit the cache; if it times out, it recomputes.

Make Optimization a Habit

The final layer is systematization. Optimization isn't a one-off action but a measurable, repeatable process. Continuously observe cache hit rate, tool calls, and token distribution; spend about 20% of your time solving the most painful point — capturing most of the benefit is enough.

There are three concrete practices: prompt templating — distill high-frequency scenarios into fixed templates, replacing only variables like time and metrics each time; batch merging — rather than repeatedly asking for "one more name," have it generate 10 candidates at once; continuous measurement — let the data tell you what to optimize next round.

The core mantra of the whole talk is just six words: feed less, emit less, reuse more. Low consumption doesn't mean reduced capability — it means spending tokens where they truly create value.

Common Questions

Do these tools require familiarity to use? No. Except for observability tools like tokenlizer that humans look at, the rest are for the model to call — just install them. They aren't limited to specific models; overseas models adapt better, and domestic models will also call them automatically once installed. Savings for image generation and video scenarios can likewise apply the templating approach: fix size parameters and distill styles like ink-wash, cyberpunk, and anime into reusable prompt templates.

One-line Summary

By optimizing input, output, context, and tools in layers, spend tokens where they matter most.

Paragraph Summary

The article systematically breaks down the root causes of token waste in AI usage and the optimization path. The author identifies redundant prompts, oversized context, over-reasoning, and verbose output as the main sources of waste, and uses "useful tokens ÷ total consumption" to measure efficiency. Two core insights: output tokens usually cost several times more than input, so limiting unnecessary output has higher ROI; and loading too many tools implicitly occupies the system prompt. Optimization falls into five layers: input should aim for determinism rather than brevity, tier models and tools on demand, control context bloat (one task per session, stable prefixes for cache reuse, summarize long tasks first), leverage context compression and code retrieval tools, and finally build a continuous measurement habit. The self-check list helps pinpoint the worst waste layer, and the core mantra is "feed less, emit less, reuse more."