MiniLM
A lightweight Transformer architecture compressed from larger models such as BERT through knowledge distillation. The L-6 in the name denotes six layers. It is widely used in Cross-Encoder ranking models and embedding models.
Timeline (last 90 days)
MiniLM is a lightweight Transformer architecture compressed from large models such as BERT via knowledge distillation, where L-6 denotes 6 layers and v2 is the second version.
Using lightweight embedding models such as sentence-transformers' MiniLM requires tens of milliseconds per inference, while large embedding models can reach hundreds of milliseconds.
all-MiniLM 是 Sentence-Transformers 项目中的轻量级模型，基于微软的MiniLM架构蒸馏而来
All Facts (3)
MiniLM is a lightweight Transformer architecture compressed from large models such as BERT via knowledge distillation, where L-6 denotes 6 layers and v2 is the second version.
50%UnverifiedUsing lightweight embedding models such as sentence-transformers' MiniLM requires tens of milliseconds per inference, while large embedding models can reach hundreds of milliseconds.
50%Unverifiedall-MiniLM 是 Sentence-Transformers 项目中的轻量级模型，基于微软的MiniLM架构蒸馏而来
50%