Unverified50% confidenceOpinionExact time
The VICO paper points out that when model capability improves, fixed training tasks deviate from its learning frontier, causing the learning signal to collapse.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAs the scale of pretrained models increases, many previously effective explicit knowledge injection techniques have gradually become ineffective or even produced negative effects.78% similarUnverifiedDuring large model training, operations such as gradient accumulation, momentum updates, and weight decay are essentially additions, and when weights are large while gradient updates are extremely small, some learning signals may be silently discarded.76% similarVerifiedDuring the post-training phase, an excessively low loss may correspond to overfitting, where the model memorizes the training data and loses generalization ability.75% similarUnverifiedWhen the distribution of online data deviates from the training data, model performance degrades (data drift and model decay), requiring retraining to be triggered.74% similarUnverifiedTraditional draft models are trained on narrow distributions targeting a single model, causing acceptance rates to drop sharply when workloads change, and requiring the entire pre-training process to be redone each time a target model is replaced.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990696API
curl https://kongchang.com/api/v1/knowledge/claims/990696MCP
get_claim(id=990696)