Local LLMs with MTP Suffix: 27% Faster on Short Q&A, 14% Slower on Long-Doc First Tokens

Title

Local LLMs with MTP Suffix: 27% Faster on Short Q&A, 14% Slower on Long-Doc First Tokens

Meta Description

Bilibili creator benchmarks local LLM MTP suffix: Qwen 3.6-35B-A3B short Q&A 27% faster end-to-end, 14% faster token output, but long-doc first-token latency up 14%. Data plus a 3-step selection guide — just scan the filename before downloading.

Article Content

For anyone running large models locally, there's a widely held assumption: to go faster, you either upgrade hardware or lower quantization. But a set of benchmarks from Bilibili creator 「AIPC遇森」 points to a nearly zero-cost path — no parameter changes, no new machine, no different model, just a single suffix in the filename when you download. Short Q&A responses complete 27% faster. The suffix is called MTP.

What MTP Is: The Model's Built-In "Guess First, Answer Later" Assistant

MTP stands for Multi-Token Prediction. In plain terms, the model has a built-in little assistant that guesses ahead: it predicts the next several tokens in one shot, then verifies them all at once. If the guesses are right, they're output directly — skipping the time cost of generating token by token.

The creator stresses in the video that MTP is not a plugin, nor a toggle requiring extra tuning. It's hidden right in the model file's name — files with the three letters MTP are the multi-token-prediction-enabled version; files without are the original. Scanning the filename at download time is the entire "configuration."

The test subject is the same Qwen 3.6-35B-A3B model. The B in the name stands for billion parameters — 35B means 35 billion parameters. Parameters are like pre-tuned knobs inside the model; more knobs means more memory, and this is unrelated to whether MTP is enabled — both versions have identical parameter counts.

First-Token Latency (TTFT) and Token Output Speed (TPOT)

The recurring "first-token latency" and "token output speed" in this article are the two core metrics for LLM inference experience. First-token latency (Time To First Token, TTFT) is the time from sending a request to the model emitting its first token, and it's mainly governed by the prefill phase — the model must read all the input context at once and compute attention, and the longer the context, the longer this phase. Token output speed (the inverse of Time Per Output Token, TPOT) corresponds to the decode phase: the model runs a forward pass for every token it generates, and speed depends on compute and model size. MTP can "save on both ends" because it simultaneously optimizes the decode phase (guessing multiple tokens at once, reducing forward passes) and changes how prefill and decode connect; the longer first-token latency in long-document scenarios is precisely because the prefill phase must additionally handle the "guessing" logic. Understanding these two metrics is what makes sense of the data structure in this article: "short Q&A gains on both ends, long documents lose at the start."

Short Q&A Benchmark: 27% Faster End-to-End, 14% Faster Output

The short Q&A group's input is 400 characters plus a small question. Comparison data for the two versions:

Metric	Original	With MTP	Change
First-token latency	5.9s	4.3s	1.6s less waiting
Output speed	57.98 tok/s	66.14 tok/s	~14% faster
Full response	85.9s	62.3s	27% faster

Same question, 23.6 seconds saved. The creator's description is vivid: go pour a glass of water and come back, the answer is already written. The key is that only one variable changed — same machine, same model, same downloaded archive; only the filename suffix differs.

A 14% output speed gain alone isn't explosive, but the creator notes that writing articles, writing code, and organizing spreadsheets often run thousands of tokens, so the speed advantage compounds; and this tier "saves on both ends": opening time drops from 5.9s to 4.3s, output is 14% faster, and stacked together the full response is 23.6 seconds shorter.

The Flip Side for Long Documents: 6 More Seconds to First Token, 14% Slower

The short Q&A advantage doesn't carry over linearly to long-text scenarios. Switching to a long-document test — stuffing in 32,000 characters for the model to read before answering — the result reverses: original first-token latency 42s, with MTP 48s, 6 more seconds, 14% slower.

The reason is that same "guess first" assistant: before guessing, it must go through the material once, and the longer the material, the longer that pass takes. So MTP is not a universal gain. For users who "just want to see the first few tokens ASAP" and often feed long materials, it actually holds them back.

Context Length and "Prefill" Overhead

The long-document test stuffs in 32,000 characters, and that magnitude directly determines the first-token latency magnitude. An LLM's context window is measured in tokens; 30,000+ Chinese characters roughly correspond to tens of thousands of tokens, and the model must fully "read through" this entire material before generating its first token — that step is prefill. Prefill compute scales roughly proportionally with context length, which is why original first-token latency jumps from 5.9s in short Q&A to 42s. The MTP version additionally does "guess-ahead" preparation in this phase, turning 42s into 48s. This also explains why MTP's benefit is highly scenario-dependent: the shorter the input, the lighter the prefill and the higher the decode share, making MTP's saved decode time more visible; the longer the input, the heavier the prefill, and the more MTP's extra overhead gets amplified.

Long Documents Overall: 6 Seconds Lost Up Front, 22 Seconds Still Saved

Don't rush to turn off MTP. In the same long-document test group, full-response time is: original 142s, with MTP 120s — 22 seconds saved, 15% faster. In other words, long-document scenarios are "6 more seconds at the start, 22 seconds saved later" — the total is still a net gain.

The creator gives the only scenario where you should turn MTP off: you only need the opening section, reading that much is enough, and you won't wait for it to finish. In that usage, the extra 6 seconds is pure loss, and the original is more suitable.

Three-Step Decision: Scan the Filename Before Downloading

Compressing the benchmark conclusions into three actionable points:

1. Check the filename: files with the three letters MTP are the multi-token-prediction-enabled version; without are the original.
2. If you want full-response speed, choose MTP: 27% faster on short Q&A, 15% faster on long documents — a gain on both ends.
3. If you want first-token speed and often feed long materials, choose the original: otherwise you wait 6 extra seconds every time you start, and the more you wait the more annoying it gets.

The value of these conclusions is that they carry almost zero migration cost. Most people's first instinct when optimizing local inference is to study quantization levels, context length, and VRAM usage, yet overlook that the filename itself at model download time already labels the inference mode. All of the creator's data comes from benchmarks on the same machine with no parameter changes, giving it high credibility; however, he also teases at the end of the video a more counterintuitive topic — that the more expensive machine sometimes loses on certain projects, showing that hardware specs and inference efficiency aren't simply positively correlated. That's worth following up on.

For local deployment folks, the MTP suffix is essentially a free "scenario switch": turn it on for short-interaction-dense daily Q&A, writing, and code completion; turn it off for long-document summarization and retrieval-style Q&A that needs extremely fast first response. A one-letter difference determines whether you wait 4.3 or 5.9 seconds, and whether the full task finishes in 62 seconds or 85.

One-line Summary

Benchmarks show that just using the MTP-suffixed filename makes local LLMs 27% faster on short Q&A and 15% faster on long documents.

Paragraph Summary

Bilibili creator 「AIPC遇森」 ran comparative benchmarks on the same machine with the same Qwen 3.6-35B-A3B model, finding that downloading the version with the MTP (Multi-Token Prediction) suffix alone speeds things up without tuning or hardware changes. Short Q&A first-token latency drops from 5.9s to 4.3s, output speed rises ~14%, and full response goes from 85.9s to 62.3s — 27% faster. Long documents reverse: first-token latency rises from 42s to 48s (14% slower), but full response still drops from 142s to 120s (15% faster). The only case for disabling MTP is long-material input where you only read the opening section and don't wait for the full answer. The article accordingly offers a three-step decision: check the filename, choose MTP for full-response speed, choose the original for first-token speed with long inputs.