mBERT
A multilingual BERT model developed by Google, jointly pre-trained on Wikipedia corpora in 104 languages. It supports cross-lingual transfer learning tasks across the languages covered in its training data.
Timeline (last 90 days)
A study based on a corpus containing 131,000 sentences covering 11 Indian scripts compared the tokenizer performance of models such as TinyLlama-1.1B, mBERT, and MuRIL.
mBERT achieves a fertility of 1.10 tokens/word on Odia, but at the same time discards 87.9% of the text as UNK.
Among the six language pairs, LaBSE's ranking results agreed with mBERT's ranking results on 4 of them.
Compared to multilingual BERT (mBERT), AraBERT has significant optimizations for Arabic morphological segmentation and semantic representation, especially in morphological restoration and root analysis.
Existing multilingual models such as mBERT and XLM-R perform far worse on low-resource languages than on English.
mBERT and PaLM 2 are Google's multilingual pre-trained models.
All Facts (7)
mBERT achieves a fertility of 1.10 tokens/word on Odia, but at the same time discards 87.9% of the text as UNK.
50%UnverifiedA study based on a corpus containing 131,000 sentences covering 11 Indian scripts compared the tokenizer performance of models such as TinyLlama-1.1B, mBERT, and MuRIL.
50%UnverifiedAmong the six language pairs, LaBSE's ranking results agreed with mBERT's ranking results on 4 of them.
50%UnverifiedCompared to multilingual BERT (mBERT), AraBERT has significant optimizations for Arabic morphological segmentation and semantic representation, especially in morphological restoration and root analysis.
50%UnverifiedExisting multilingual models such as mBERT and XLM-R perform far worse on low-resource languages than on English.
50%UnverifiedmBERT and PaLM 2 are Google's multilingual pre-trained models.
50%Unverified现代多语言模型如mT5、BLOOM和Llama系列在数百种语言的文本上联合训练，通过共享词汇表和跨语言注意力机制构建跨语言语义地图
50%