Ngram
A novel inference mechanism introduced in DeepSeek v4.1 Flash. It enables fast vocabulary lookup through a conditional memory table based on N-gram sequence matching, implemented as a non-computational large-scale lookup table structure rather than a learned computation.
Timeline (last 90 days)
v4.1 Flash introduces a mechanism called Ngram, where 2 out of 40 layers contain conditional memory tables, with 196 billion parameters used for a lookup table that matches sequences of up to 4 tokens against a vocabulary of 16 million.
Converting the 8-bit Ngram table to 4-bit can save approximately 100 GB, reducing the model size from approximately 475 GB to approximately 385 GB.
All Facts (2)
v4.1 Flash introduces a mechanism called Ngram, where 2 out of 40 layers contain conditional memory tables, with 196 billion parameters used for a lookup table that matches sequences of up to 4 tokens against a vocabulary of 16 million.
50%UnverifiedConverting the 8-bit Ngram table to 4-bit can save approximately 100 GB, reducing the model size from approximately 475 GB to approximately 385 GB.
50%