Unverified50% confidenceOpinionExact time
When training small code models, the choice of tokenizer vocabulary size is often treated as a secondary hyperparameter, but experiments show its impact is significant.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFor domains where code errors are extremely costly and high-quality training data is scarce, purpose-trained small-to-medium models may be more cost-effective and reliable than calling the most powerful general-purpose models.77% similarUnverifiedRecursive training makes extensive use of task completion rates and code execution results as reward signals, lacking explicit supervision for narrative consistency, which may lead to catastrophic forgetting of general language modeling capabilities, and this is more pronounced in models with small parameter counts.74% similarUnverified更宽泛的分词器扩大了词表规模或覆盖范围，会增加嵌入层参数量和内存占用并提高训练成本74% similarUnverified模型能力高度依赖训练数据的分布，法律文本充足而空间推理题的文本化表示相对稀少72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019334API
curl https://kongchang.com/api/v1/knowledge/claims/1019334MCP
get_claim(id=1019334)