Verified80% confidenceFactTime unknown
The vocabulary of a standard autoregressive language model typically contains 32,000 to 128,000 tokens.
4
Sources
80%
Confidence
Long-term
Relevance
6/1/2026
First Seen
Sources
Related Claims
Unverified现代语言模型的词汇表规模通常在32,000到100,000个token之间，使用BPE或SentencePiece等分词方法78% similarUnverifiedBenchmark datasets for sign language translation (such as PHOENIX-2014T, CSL-Daily, and How2Sign) typically contain only thousands to tens of thousands of samples, far smaller than the millions of sentence pairs in parallel corpora in the machine translation field.70% similarUnverifiedThe English version of Layla has 420 million parameters, the multilingual version has 320 million parameters and covers hundreds of languages, and there is an additional version fine-tuned for workflows.68% similarUnverifiedA study based on a corpus containing 131,000 sentences covering 11 Indian scripts compared the tokenizer performance of models such as TinyLlama-1.1B, mBERT, and MuRIL.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/21216API
curl https://kongchang.com/api/v1/knowledge/claims/21216MCP
get_claim(id=21216)