ConceptMQA / Multi-Query Attention

Multi-Query Attention

A variant of the attention mechanism in which all query heads share a single key/value head, greatly reducing the KV cache, though it may affect model quality.

Timeline (last 90 days)

Oct 18

In multi-query attention (MQA), Hkv = 1, and the cache is even smaller than with GQA.

Unverified50%

All Facts (1)

Unverified

In multi-query attention (MQA), Hkv = 1, and the cache is even smaller than with GQA.

50%

Source Articles

Local LLM VRAM Budget: Weights, KV Cache, and Runtime OverheadOct 18