Unverified50% confidenceFactExact time
Models such as Mistral and Mixtral adopt an architecture that alternates between Full Attention and SWA, and vLLM's Group partitioning mechanism supports this pattern.
1
Sources
50%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGQA将注意力头分组共享KV，已被LLaMA 3、Mistral等主流模型采用78% similarUnverifiedGQA（Grouped-Query Attention）方案已被LLaMA 3、Mistral等主流模型采用以缓解HBM带宽瓶颈72% similarUnverifiedLlamaFactory supports fine-tuning of models including LLaMA, Qwen, ChatGLM, and Mistral through a unified interface70% similarVerifiedThe key technical innovations of Mistral 7B include Sliding Window Attention (SWA) and Grouped Query Attention (GQA).68% similarUnverifiedThe Mistral model family is a typical representative of SWA, using a window size of 4096 tokens at each layer.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/887215API
curl https://kongchang.com/api/v1/knowledge/claims/887215MCP
get_claim(id=887215)