Model
TinyBERT
华为提出的超轻量BERT蒸馏模型，通过两阶段知识蒸馏大幅压缩参数量，适合资源受限环境下的NLP任务部署
Timeline (last 90 days)
Sep 28
After adopting a frequency-aware training schedule (sampling rare tokens at a rate higher than uniform sampling), the model matches the parameter-matched BERT and TinyBERT baselines on the rarest token-frequency bin.
Unverified50%
Sep 21
Models such as TinyBERT and MobileBERT further combine distillation with architecture search to improve parameter compression ratios.
Unverified50%
All Facts (2)
Unverified
After adopting a frequency-aware training schedule (sampling rare tokens at a rate higher than uniform sampling), the model matches the parameter-matched BERT and TinyBERT baselines on the rarest token-frequency bin.
50%UnverifiedModels such as TinyBERT and MobileBERT further combine distillation with architecture search to improve parameter compression ratios.
50%Source Articles
意图识别面试题答法：三层漏斗架构拆解与答题模板Jul 11