Unverified50% confidenceFactExact time
MOSS-VL-Realtime offers a 4-bit quantized version designed for a single RTX 4090; after 4-bit quantization, the weight memory footprint of the 11-billion-parameter model can be reduced to approximately 5.5 GB of VRAM
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/10/2026
First Seen
Valid until: 12/9/2026
Sources
Related Entities
Related Claims
VerifiedQLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.74% similarVerifiedA model with 70 billion parameters (70B) can run on a single RTX 4090 (24GB VRAM) after quantization.69% similarUnverifiedA ~25 GB Q3 model running entirely within the RTX 4090's VRAM (bandwidth ~1 TB/s) theoretically achieves ~40 tokens per second, whereas running it in DDR5 memory (~80 GB/s) drops throughput to 1–3 tokens per second.69% similarUnverifiedAfter quantization, the VRAM requirement of a 7-billion-parameter model is compressed from approximately 28 GB to approximately 4–6 GB, allowing consumer-grade GPUs such as the RTX 3060/4060 to run 7-billion-parameter-class models like Llama 3, Qwen2.5, and Mistral smoothly.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/885998API
curl https://kongchang.com/api/v1/knowledge/claims/885998MCP
get_claim(id=885998)