MXFP4
A microscaling floating-point quantization format defined by the Open Compute Project (OCP) that compresses each model parameter to 4 bits. It maintains numerical precision at low bit-widths through intra-group shared scaling factors, and is used to reduce cost and improve efficiency in large-model inference workloads.
Core Facts
Timeline (last 90 days)
The OCP-standard MXFP4 uses E8M0 (pure exponent) scaling factors with a block size of 32.
FlyDSL covers core operations such as dense and grouped GEMM, fused multi-head attention (FMHA/Attention), MXFP8/MXFP4 scaled GEMM, RMSNorm and TopK.
The solution uses BF16 precision uniformly for the KV cache, and MXFP4 quantization for the model weights.
In the DeepSeek R1 670B (MXFP4 precision) test, Jalapeño achieved approximately 1.7× throughput per watt (19,641 vs. GB300's 11,781), and end-to-end latency was reduced from 5.99 seconds to 1.65 seconds.
This optimization focuses on the vLLM inference framework and the MXFP4/NVFP4 quantization paths, involving writing or adjusting dedicated compute kernels.
MXFP4 belongs to the MX (Microscaling) series of standards and is a block floating-point quantization specification in which a group of weights shares a single scaling factor.
MXFP8 belongs to the microscaling floating-point format family, whose core concept is sharing a single scaling factor within a small block of data.
All Facts (7)
MXFP8 belongs to the microscaling floating-point format family, whose core concept is sharing a single scaling factor within a small block of data.
75%UnverifiedIn the DeepSeek R1 670B (MXFP4 precision) test, Jalapeño achieved approximately 1.7× throughput per watt (19,641 vs. GB300's 11,781), and end-to-end latency was reduced from 5.99 seconds to 1.65 seconds.
60%UnverifiedThe OCP-standard MXFP4 uses E8M0 (pure exponent) scaling factors with a block size of 32.
50%UnverifiedFlyDSL covers core operations such as dense and grouped GEMM, fused multi-head attention (FMHA/Attention), MXFP8/MXFP4 scaled GEMM, RMSNorm and TopK.
50%UnverifiedThe solution uses BF16 precision uniformly for the KV cache, and MXFP4 quantization for the model weights.
50%UnverifiedThis optimization focuses on the vLLM inference framework and the MXFP4/NVFP4 quantization paths, involving writing or adjusting dedicated compute kernels.
50%UnverifiedMXFP4 belongs to the MX (Microscaling) series of standards and is a block floating-point quantization specification in which a group of weights shares a single scaling factor.
50%