NVFP4
NVFP4 is a 4-bit floating-point (FP4) numerical quantization format introduced by NVIDIA for the Blackwell architecture, representing a low-precision numerical standard. Designed for deep learning inference and training scenarios, it reduces memory usage and computational overhead by lowering the bit width of numerical representations. Compared to conventional integer quantization formats, NVFP4 offers advantages in numerical range coverage and precision loss control, and is primarily used for efficient deployment and accelerated computation of large-scale neural network models.
Core Facts
NVFP4 is a 4-bit floating-point (FP4) quantization format introduced by NVIDIA, designed specifically for the hardware acceleration capabilities of Blackwell architecture GPUs
NVFP4 is a 4-bit floating-point quantization format used to reduce memory usage and improve inference throughput while preserving model accuracy.
NVIDIA designed a dedicated Tensor Core instruction mma.kind::mxf4 for NVFP4 in the Blackwell architecture (SM 100/103, such as the B100 and B200 series), and adopted an FP8 scaling factor mechanism with one scale per group of 16 elements.
Timeline (last 90 days)
NVFP4 uses block-level shared scaling factors to extend the representable dynamic range, making it suitable for scenarios where weight and activation distributions are relatively concentrated.
The weights of Nemotron 3.5 Transcribe are released in three precisions: BF16, FP8, and NVFP4.
The NVFP4 decode path on DGX Spark differs from that on data-center GPUs, so it requires separate test configurations and test-case selection logic.
This version includes a CI fix: it corrects the outdated NVFP4 decode test selector on the DGX Spark platform (PR #64107).
NVFP4, with its finer block granularity and floating-point scaling, better preserves small-value information in layers where activation distributions span a wide range, so at the same 4-bit width it often incurs less perplexity loss than INT4.
NVFP4 requires a GPU with fifth-generation Tensor Core FP4 support to deliver hardware acceleration; older-generation GPUs can only emulate it via dequantization, and the speed gains are greatly diminished.
Someone ran the Qwen3.8-Flash-Next model at NVFP4 precision on a single NVIDIA RTX 6000 Pro, achieving an inference speed of 184 tok/s.
The MLX framework refuses to directly create Flint's NVFP4 weights before conversion; Flint's NVFP4 weights require a specific conversion process rather than being instantiated directly at the import stage.
The framework chooses to assemble the expert banks first and then perform the NVFP4 conversion, to avoid instantiating the target format directly when data is incomplete, which would cause precision loss or conversion failure.
This version refuses to directly create Flint's NVFP4 model before conversion is complete.
40 more timeline events
All Facts (20)
NVFP4 is a 4-bit floating-point (FP4) quantization format introduced by NVIDIA, designed specifically for the hardware acceleration capabilities of Blackwell architecture GPUs
90%VerifiedNVFP4 is a 4-bit floating-point quantization format used to reduce memory usage and improve inference throughput while preserving model accuracy.
75%VerifiedNVIDIA designed a dedicated Tensor Core instruction mma.kind::mxf4 for NVFP4 in the Blackwell architecture (SM 100/103, such as the B100 and B200 series), and adopted an FP8 scaling factor mechanism with one scale per group of 16 elements.
75%VerifiedIn an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
70%VerifiedNVFP4 is a 4-bit floating-point format (FP4) introduced by NVIDIA, using floating-point representation (including sign bit, exponent bit, and mantissa bit) to cover the numerical space in a non-uniform manner.
65%VerifiedThe Unsloth team released a new Qwen3.8-27B GGUF model based on the Dynamic v3 quantization scheme, achieving an additional 10% improvement in top-1% accuracy while maintaining the same model size.
85%UnverifiedNVFP4量化需要50系列NVIDIA显卡才能使用
85%Unverified在非50系显卡上使用NVFP4会退化为软件模拟，导致显存占用大、精度差，无任何优势
85%UnverifiedNVFP4量化在Blackwell架构上相比FP16吞吐量提升约40-50%
80%Partially VerifiedThe 816B-parameter Muse Spark Open occupies about 493GB of memory after NVFP4 quantization, while FP16 storage would require more than 1.6TB.
80%Partially VerifiedNVFP4 occupies 4 bits per parameter, i.e., 0.5 bytes, theoretically halving weight memory usage and load bandwidth compared with FP8.
75%UnverifiedNVFP4 uses block-level shared scaling factors to extend the representable dynamic range, making it suitable for scenarios where weight and activation distributions are relatively concentrated.
50%UnverifiedThe weights of Nemotron 3.5 Transcribe are released in three precisions: BF16, FP8, and NVFP4.
50%UnverifiedThe NVFP4 decode path on DGX Spark differs from that on data-center GPUs, so it requires separate test configurations and test-case selection logic.
50%UnverifiedNVFP4 requires a GPU with fifth-generation Tensor Core FP4 support to deliver hardware acceleration; older-generation GPUs can only emulate it via dequantization, and the speed gains are greatly diminished.
50%UnverifiedSomeone ran the Qwen3.8-Flash-Next model at NVFP4 precision on a single NVIDIA RTX 6000 Pro, achieving an inference speed of 184 tok/s.
50%UnverifiedNVFP4, with its finer block granularity and floating-point scaling, better preserves small-value information in layers where activation distributions span a wide range, so at the same 4-bit width it often incurs less perplexity loss than INT4.
50%UnverifiedThe MLX framework refuses to directly create Flint's NVFP4 weights before conversion; Flint's NVFP4 weights require a specific conversion process rather than being instantiated directly at the import stage.
50%UnverifiedThe framework chooses to assemble the expert banks first and then perform the NVFP4 conversion, to avoid instantiating the target format directly when data is incomplete, which would cause precision loss or conversion failure.
50%UnverifiedThis version refuses to directly create Flint's NVFP4 model before conversion is complete.
50%