ConceptGS投影 / Gram-Schmidt正交化
Gram-Schmidt Projection
An orthogonalization method in linear algebra used to project vectors onto an orthogonal basis; in optimizers it can be used to remove the radial component of an update direction.
Timeline (last 90 days)
Oct 28
MuonGS uses Gram-Schmidt projection to remove the radial component in each Muon update without fixing the weight norm.
Unverified50%