Skip to main content
โ† Back to glossary

N-gram embedding

Architectures

A mechanism in which the model indexes short runs of consecutive tokens (bigrams, trigrams) through an embedding table, instead of treating them one by one. It cuts computational cost and memory for very long contexts, and is the scaling axis used by Qwen3.8-Flash-Next (51B of dedicated parameters).

This term in other languages

๐Ÿ‡ฎ๐Ÿ‡นEmbedding n-gram
๐Ÿ‡ซ๐Ÿ‡ทEmbedding n-gram