Verified70% confidenceFactTime unknown
The rank r of the low-rank matrices in LoRA is typically set to 8 or 16, reducing the number of trainable parameters from d×k to r×(d+k), a reduction of up to hundreds of times.
4
Sources
70%
Confidence
Long-term
Relevance
6/1/2026
First Seen
Sources
4-bit QLoRA微调LLaMA 3实战：消费级GPU训练80亿参数大模型指南
githubCre4T3Tiv3
Related Entities
Related Claims
VerifiedLoRA decomposes the fine-tuning weight change ΔW into the product of two low-rank matrices, with a parameter count of only 0.1%–1% of the original model, achieving a parameter compression ratio as high as 256× when r=8 and d=4096.74% similarUnverifiedTaking a 1000×1000 weight matrix as an example, if the LoRA rank is set to 8, only approximately 16,000 parameters need to be trained instead of the original 1,000,000 parameters.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/29159API
curl https://kongchang.com/api/v1/knowledge/claims/29159MCP
get_claim(id=29159)