Unverified50% confidenceFactExact time
JOLT's absolute accuracy on MATH, the model scale at which it matches GRPO, whether the 6.5x token reduction comes at the cost of more training steps or a more complex teacher training phase, and whether there is a risk of capability degradation or mode collapse under shared parameters are all open questions still to be answered.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/24/2026
First Seen
Valid until: 11/7/2026
Sources
Related Entities
Related Claims
Unverified小模型通过知识蒸馏通常比直接在标注数据上训练精度高出2-5个百分点71% similarUnverifiedCompared to MoE architectures, dense models have more uniform gradient flow during training, are less prone to expert collapse, and exhibit more deterministic inference behavior.71% similarUnverifiedDuring fine-tuning, model weights, gradients, and optimizer states must be stored simultaneously, making VRAM consumption typically 3–4 times that of inference.71% similarVerifiedAlignment training optimizes the model's statistical behavior in known scenarios rather than establishing formally verifiable security boundaries, and may fail when faced with rare instruction combinations or role-playing frameworks outside the training distribution.71% similarUnverifiedDuring fine-tuning, the prompt format in the training phase must strictly match the inference phase; otherwise, performance degradation will occur without any error being reported.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094396API
curl https://kongchang.com/api/v1/knowledge/claims/1094396MCP
get_claim(id=1094396)