Unverified60% confidenceSolutionExact time
SFT fine-tuning is effective but causes a decline in general knowledge in small models—i.e., the phenomenon of catastrophic forgetting—due to the prominent capacity bottleneck of small models.
2
Sources
60%
Confidence
Long-term
Relevance
8/29/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSupervised fine-tuning (SFT) tends to cause more severe catastrophic forgetting than reinforcement fine-tuning (RFT).77% similarUnverifiedWhen the context approaches its limit, the model may forget critical early information or produce lower-quality output.76% similarUnverifiedA severely overspecified model parameter count will cause an OOM (out-of-memory) error and prevent startup, while a slightly overspecified count will borrow regular memory and cause a significant drop in speed74% similarUnverifiedFine-tuning carries the risk of catastrophic forgetting, where fine-tuning for a specific behavior may degrade the model's performance on other tasks.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/819750API
curl https://kongchang.com/api/v1/knowledge/claims/819750MCP
get_claim(id=819750)