Verified70% confidenceFactExact time
LoRA attaches two low-rank matrices A and B in parallel alongside each target weight matrix, where A is randomly initialized and B is initialized to zero; only these two small matrices are trained, and during inference the product A×B is added back to the original weights.
4
Sources
70%
Confidence
Long-term
Relevance
7/24/2026
First Seen
Sources
零基础入门大模型微调：原理、场景与实战路径
Related Claims
VerifiedLoRA achieves fine-tuning by inserting low-rank decomposition matrices alongside the model's weight matrices, typically requiring training of only 0.1%–1% of the original parameter count.84% similarUnverifiedThe values in LoRA's low-rank matrices A and B are dense, with each parameter carrying mixed functional signals, so it is difficult to read what the model has learned by directly inspecting the adapter weights.77% similarUnverifiedFor a frozen base matrix W0, LoRA represents the learned update as W' = W0 + (alpha/r)BA, where A and B are trainable compact matrices with rank r much smaller than the dimensions of the original matrix.75% similarUnverifiedLoRA (Low-Rank Adaptation) constrains the weight update matrix ΔW to BA via low-rank decomposition, where B∈ℝ^{d×r}, A∈ℝ^{r×k}, and rank r≪min(d,k), thereby reducing the number of trainable parameters from d×k to r×(d+k).75% similarVerifiedDuring LoRA training, the pre-trained model weights W₀ are frozen and only the two matrices B and A are trained, with the forward propagation formula H = W₀X + (α/r)BAX.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/607136API
curl https://kongchang.com/api/v1/knowledge/claims/607136MCP
get_claim(id=607136)