QQ Bot Burning Through Tokens Too Fast? Engineering Layering and Cost-Saving Tips

Why Is Your Bot Such a Token Hog?

QQ bots have become a hot topic recently, and many developers are jumping in—only to hit the same bottleneck almost immediately: token consumption far exceeds expectations. In a sharing session, Bilibili creator "Da Fei Yu" (Big Fat Fish) put it bluntly: many people have their bots read every single message in full, carrying all context and memory from the start. This approach is "absurd," he said—"it will definitely need optimization later; no normal AI is used this way."

The root of the problem isn't the model itself but how it's invoked. Loading a complete persona, historical memory, and knowledge base with every message naturally multiplies token usage. This pitfall is especially easy to fall into when users want the bot to be "more capable" and are unwilling to weaken it. Below, we break down actionable ways to speed things up and cut costs, from two angles: engineering layering and cost optimization.

Engineering Layering: Separate "Reading" from "Thinking"

Separating Deep Memory from Shallow Memory

The creator's core idea is a two-layer memory architecture: deep memory holds the persona card, memes, emoji packs, and long-term memory—this part can be loaded heavily; shallow memory keeps only the basic personality and basic memory, kept as small as possible.

He used concrete numbers to illustrate the gap: shallow memory at around 1000 tokens is already on the high side, and ideally should be compressed to under 300 tokens. Each message first uses this lightweight context to make a judgment—whether a reply is needed and what the general direction of the reply should be; only once a reply is confirmed does it load the full context.

This "judge first, load later" flow keeps the high-frequency reading step extremely cheap, while only the reply step bears the cost of the full context. By his estimate, this single optimization alone can save "a huge amount" of tokens—possibly 10x to 20-30x.

---

Token is the basic unit of text processed by a large language model, roughly understandable as a "word fragment." Each time a model is invoked, both input (prompt) and output are billed by token, and the context window has an upper limit. In Chinese, one character typically corresponds to 1-2 tokens, so carrying a full persona, history, and knowledge base causes token counts to balloon quickly. Understanding how tokens are counted helps you judge which content is worth including in each request and which should be externalized or loaded on demand—this is precisely the measurement basis for layered memory optimization.

Why Full Loading Is Unsustainable

The problem with reading everything isn't just cost—it also dilutes the model's attention. Every irrelevant message carries the full persona and history, and the model must locate the current task within a large amount of redundant information, which may actually degrade performance. Layered design is essentially about matching "the minimum information set required for the task," and this is the watershed between engineering thinking and "just calling the API directly."

Controlling Costs: Don't Just Stare at the Official API

Switch to Coding Plan / Engineer Plan

Many bot developers are used to going straight through the official API, but the creator points out that, from the perspective of someone who develops frequently, there's "not much difference" between various tokens—so-called exclusive personas are mostly added later and are not a difference in the model's foundation. What really creates a cost gap is the type of plan.

He mentioned Volcano Engine's coding plan, which can deliver about 4-5x the usage for bot-style workloads; plans like opencode go also fall under the coding plan or engineer plan category and can bring at least 2-3x the usage. For bot developers just getting started, this is industry common knowledge but an easily overlooked piece.

---

Coding Plan is a type of plan offered by cloud providers for developers, usually providing large model API call quotas on a monthly or volume basis, with unit prices significantly lower than the official pay-as-you-go rate. These plans were originally designed for high-frequency, lightweight scenarios like code completion and IDE integration, but bot developers' invocation patterns (high-frequency short requests) are similar, so they can borrow the low-cost quota. Note that plans may have limits on concurrency, model versions, or usage scenarios; before choosing, confirm whether bot-type applications are permitted to avoid quota freezes due to policy violations.

Channels for Getting Free Tokens

If your interest is more technical, you can also explore sources of free tokens. The creator briefly listed a few: some AI services give away quotas, and you can extract the API keys for your own use; suites like Zhipu and opencode offer free tokens; AMD or Volcano Engine also give away roughly a few million tokens. These methods have a higher barrier and are cumbersome to operate, but they can genuinely lower costs further. He suggests just having your own AI search for the specific channels.

RAG: Give Your Character an External Knowledge Base

For developers who care especially about character settings, the creator recommends using the most basic RAG (Retrieval-Augmented Generation) as an external knowledge base. The implementation isn't complicated—just telling the AI "use the most basic RAG for the bot" is basically enough.

RAG's value lies in decoupling "knowledge" from the context window: retrieve relevant fragments when needed instead of stuffing all materials into every request. This aligns with the layered memory approach—both reduce the amount of information carried in a single call. Combined, the savings are amplified further.

---

RAG (Retrieval-Augmented Generation) typically works as follows: first split knowledge base documents into fragments and vectorize them into a vector database; when a user asks a question, vectorize the question with the same embedding model, retrieve the most relevant fragments, then splice these fragments as context into the prompt and hand it to the large model to generate an answer. This way the model doesn't need to memorize all knowledge in its parameters, and avoids carrying all materials in every request. For bot character settings, you can put persona details and world-building documents into RAG and only retrieve them when relevant conversations occur, complementing layered memory.

The Real-World Impact of the Optimization Combo

Looking at layered memory, plan selection, and RAG together, the creator's judgment is: if each optimization is done well individually, savings can reach dozens or even hundreds of times, and he stresses that these "are all the most basic optimization approaches." In other words, most token waste comes from architectural sloppiness, not insufficient model capability.

For bot developers just getting started, the recommended priority is: first change the layered loading logic, then switch to a coding plan-type package, and finally introduce RAG as needed. The most direct path the creator offers is to forward this passage directly to your own AI and have it understand and implement the changes. He puts it clearly: first separate the knowledge and memory needed for reading from those needed for replying, and don't load them in equal amounts—then the problem of running out of tokens will basically be alleviated.

One-line Summary

Layered memory + plan selection + RAG can save bots tens to hundreds of times the tokens.

Paragraph Summary

The article addresses the common problem of excessive token consumption among QQ bot developers, pointing out that the root cause is the sloppy invocation pattern of loading the persona, history, and knowledge base in full with every message. Bilibili creator "Da Fei Yu" proposes an engineering layering approach: split memory into deep (persona, long-term memory) and shallow (basic personality, kept under 300 tokens), use the shallow layer to first judge whether to reply, and only load the full context after confirming—this alone can save 10x to 20-30x tokens. On cost, he suggests switching from the official API to plans like Volcano Engine's coding plan and opencode go for several times the usage; free token channels are also worth exploring. In addition, a basic RAG external knowledge base decouples knowledge from the context window and stacks well with layered memory. With combined optimization, savings can reach dozens or even hundreds of times, and all are the most basic engineering measures.