Qwen3 Flash Next
A large Mixture-of-Experts (MoE) language model from Alibaba's Qwen series, with approximately 125 billion total parameters, of which around 6 billion are activated per inference pass. The model supports local deployment in GGUF format and demonstrates strong programming capabilities.
Core Facts
Timeline (last 90 days)
Developer Primitive reported a generation speed of 142.6 tokens/s for Qwen3 Flash Next running locally.
Unsloth has released a GGUF quantized version of Qwen3 Flash Next, available for download through the Model Hub of the Unsloth desktop application.
Developer Primitive reported: 91.2 tokens/s with speculative decoding off, reaching 142.6 tokens/s after enabling 3 speculative tokens, approximately a 56% speedup under the same configuration.
Even with the most aggressive quantization, a model with a total of 177B parameters still occupies approximately 72.5GB.
The UP host ran a 177B-parameter model occupying approximately 85GB locally using two budget graphics cards (24GB VRAM in total) plus 32GB of system memory (approximately 50GB total available).
Qwen3 Flash Next is regarded as the next-generation open-weight flagship model, with an intelligence index comparable to frontier closed-source models.
The deployment uses a llama.cpp server with 4-bit KV cache, context length set to 100,000, 32 layers offloaded to system memory, and lazy mode enabled to force the Ngram table to be read from SSD.
Running the model on Windows is approximately 20% slower than on Linux, with approximately 16 tokens/s for short contexts and approximately 10 tokens/s for long contexts.
All Facts (8)
Developer Primitive reported a generation speed of 142.6 tokens/s for Qwen3 Flash Next running locally.
65%UnverifiedDeveloper Primitive reported: 91.2 tokens/s with speculative decoding off, reaching 142.6 tokens/s after enabling 3 speculative tokens, approximately a 56% speedup under the same configuration.
50%UnverifiedUnsloth has released a GGUF quantized version of Qwen3 Flash Next, available for download through the Model Hub of the Unsloth desktop application.
50%UnverifiedEven with the most aggressive quantization, a model with a total of 177B parameters still occupies approximately 72.5GB.
50%UnverifiedQwen3 Flash Next is regarded as the next-generation open-weight flagship model, with an intelligence index comparable to frontier closed-source models.
50%UnverifiedThe deployment uses a llama.cpp server with 4-bit KV cache, context length set to 100,000, 32 layers offloaded to system memory, and lazy mode enabled to force the Ngram table to be read from SSD.
50%UnverifiedRunning the model on Windows is approximately 20% slower than on Linux, with approximately 16 tokens/s for short contexts and approximately 10 tokens/s for long contexts.
50%UnverifiedThe UP host ran a 177B-parameter model occupying approximately 85GB locally using two budget graphics cards (24GB VRAM in total) plus 32GB of system memory (approximately 50GB total available).
50%