Concept网络秩 / 秩
rank
The intermediate dimension of the low-rank decomposition matrices, which determines the feature capacity the LoRA model can express. A larger rank yields higher capacity but also increases the risk of overfitting.
Timeline (last 90 days)
Oct 16
LoRA's parameter count, memory usage and optimization difficulty grow linearly with the rank r, rather than quadratically with the model width.
Unverified50%