Unverified50% confidenceFactExact time
The model natively supports a 256K token context and can be scaled to approximately 1 million tokens via 4x scaling using YaRN technology.
1
Sources
50%
Confidence
Long-term
Relevance
9/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCombined with interpolation techniques such as YaRN and Dynamic NTK, models trained with a context length of 4K can be extended to context windows of 128K or even 1M.72% similarUnverifiedWhen the context expands from 4K tokens to 128K tokens (a 32× increase), the theoretical computation grows by approximately 1024×.69% similarUnverifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.68% similarUnverifiedThe expansion of large model context windows from the early 4K tokens to today's 100K+ or even longer is one of the technical foundations of agentic programming.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/863821API
curl https://kongchang.com/api/v1/knowledge/claims/863821MCP
get_claim(id=863821)