Expired50% confidenceFactTime unknown
In the Gemma 4 26B MOE model, the model contains 128 internal 'experts' but only 8 are activated per inference.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 9/30/2026(expired)
Sources
Google Gemma 4 Hands-On Review: Offline on Smartphones + Ollama Deployment Tutorial
bilibiliAGI_Ananas
Related Claims
VerifiedGemma 4's 26B MoE model uses a Mixture of Experts architecture, with 26 billion total parameters but only approximately 4 billion parameters active at any given moment.71% similarUnverified在MoE模型中每个token只激活少量Expert（通常2-4个，Top-K路由），实际计算量可能仅相当于6-7B的Dense模型68% similarUnverifiedMoE模型中'35B'指总参数量，每个token只激活少量Expert（通常2-4个），实际计算量可能仅相当于6-7B的Dense模型68% similarUnverifiedKimi K2.7 Code uses a MoE architecture with 384 experts, selecting 8 experts plus 1 shared expert per token, with actual computation approximately 2.3% of that of a dense model.67% similarUnverifiedThe Qwen3.6 35B model contains 256 experts, with each token activating only 8 routed experts and 1 shared expert, resulting in approximately 3B activated parameters during actual inference.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/57177API
curl https://kongchang.com/api/v1/knowledge/claims/57177MCP
get_claim(id=57177)