Unverified50% confidenceOpinionExact time
HoS's gating parameters are very small in scale and do not touch the main weights, making training cost far lower than full-parameter fine-tuning and avoiding catastrophic forgetting.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe cost of targeted optimization is far lower than that of large-scale training improvements, and it is very difficult for users to detect, especially when users evaluate models using only a small number of the most popular prompts.68% similarUnverifiedSMD's repulsion mechanism does not require the discriminator commonly used in adversarial training, nor does it require online score estimation; it is merely an additional constraint on the training objective, and the training overhead is almost zero.67% similarUnverifiedA model with fewer parameters but fully trained on more tokens, though it may appear over-trained during training, can significantly reduce costs at inference time, since inference overhead is primarily determined by parameter count.67% similarVerifiedParameter-efficient fine-tuning (PEFT) methods such as LoRA and Adapter can compress the number of trainable parameters to less than 1% of the total parameters.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106048API
curl https://kongchang.com/api/v1/knowledge/claims/1106048MCP
get_claim(id=1106048)