Unverified50% confidenceFactExact time
Deduplicating the Q/K/V weights saved 488 MiB of VRAM, and keeping the text embedding table on the CPU saved 594 MiB, enabling the two models Gemma 4 E4B and Qwen3-TTS 1.7B to be loaded simultaneously, with the graphics card showing 7,448 MiB used out of 8,188 MiB.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
UnverifiedSome ONNX exports labeled q4 are larger than the q4f16 versions because the embedding table retains 32-bit precision; for example, SmolLM2-360M's q4 is 390MB and q4f16 is 275MB, and the two produce identical outputs.68% similarUnverifiedOn a single Mac Studio with 512GB of memory, the 128B Mistral Medium 3.5 (FP8) and the 27B Qwen 3.8 (BF16) occupy about 158GB and 68GB of memory respectively at a 65536-token context.68% similarUnverifiedA user runs the 397B model in FP8 on 8× GB10 units at a speed 20% faster than INT4, and published the first solution for MiMo 2.5 Pro and Kimi 2.6 on the NVIDIA forums.67% similarVerifiedMemory requirements are at least 37.6 GB for Q2_0, at least 39.2 GB for IQ2_XS, and at least 47 GB for IQ3_XXS, with the vision encoder occupying an additional 0.91 GB.67% similarUnverifiedThe Qwen 9B model has a peak memory usage of approximately 5.1 GB when loaded in MLX format.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1043814API
curl https://kongchang.com/api/v1/knowledge/claims/1043814MCP
get_claim(id=1043814)