Unverified50% confidenceFactExact time
The NVFP4 version of Qwen3-27B uses sensitivity-aware mixed precision: feed-forward layers run in FP4 (E2M1) with FP8 block scaling, while attention projections, DeltaNet recurrent layers, the vision encoder, and multi-token prediction heads remain in FP8 or BF16.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
Unverified工程实践中建议将路由器层、注意力投影矩阵及门控网络保留在 BF16/FP16 精度，仅对专家 FFN 权重进行激进量化70% similarUnverified开发者采取混合精度策略，将对质量敏感的层保留在BF16，仅对能从FP8获益的部分做转换69% similarUnverifiedQwen3.8-Flash-Next adopts a hybrid attention mechanism of GDN plus QSA, with a total of 125B parameters plus 51B N-gram embeddings, activating only approximately 6B parameters per token.68% similarUnverifiedMixed-precision training was jointly proposed by NVIDIA and Baidu in 2018; its core idea is to use FP16 in forward and backward propagation to reduce VRAM usage and accelerate computation, while retaining an FP32 master weight copy for parameter updates.68% similarVerifiedQuantization compresses neural network weights from high-precision floating-point numbers (such as FP32 and FP16) to low-bit integers (such as INT8 and INT4), with modern quantization schemes including GPTQ, AWQ, and GGUF's K-quant series67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/907273API
curl https://kongchang.com/api/v1/knowledge/claims/907273MCP
get_claim(id=907273)