Gemma 4
Gemma 4 is an open-source large language model series released by Google, with a parameter scale of 31 billion (31B). Built on the Transformer architecture, it supports tasks including text understanding, generation, and multi-step reasoning. Model weights are made available to researchers and developers for local deployment and fine-tuning on downstream tasks. As an iteration of the Gemma series, it is designed to deliver strong general-purpose language processing capabilities while maintaining relatively low resource requirements.
Core Facts
Gemma 4's 26B MoE model uses a Mixture of Experts architecture, with 26 billion total parameters but only approximately 4 billion parameters active at any given moment.
Gemma 4 adopts the Apache 2.0 license, allowing completely unrestricted commercial use
Gemma 4的1B/2B模型量化后仅需4GB显存即可运行
Timeline (last 90 days)
Change #19247 is titled 'preserve small-image detail for compact Gemma 4 vision towers', aiming to preserve detail information in small-sized images when Gemma 4's compact vision encoders process images.
TF-IDF+logistic regression never predicts the two rarest categories, and on the type field it is statistically on par with Gemma 4.
The MLX backend of Ollama v0.40.10 supports speculative decoding using a DFlash draft model for the Gemma 4 target model.
Ollama v0.40.10 adds a gemma4-nemotron renderer in the renderers module.
Users can explicitly select this renderer via RENDERER gemma4-nemotron, while the automatic detection logic for gemma4 remains unchanged, so existing users will not be affected by behavior changes.
Some models fine-tuned on Gemma 4 (such as the Nemotron series) adopt an assistant-content-first layout during training, and using the default renderer may cause tool call format misalignment or parsing failures.
When training Qwen3-MoE, Gemma 3, and Gemma 4 on GPUs that do not support bf16, Unsloth reduces attention memory usage by up to 7 times (requires torch 2.13 or earlier).
The gemma4-nemotron renderer targets Gemma 4 variants that use the Nemotron-style chat format.
Gemma 4's ecosystem adaptation is still in the paving stage, this batch of changes in the rc version is still being validated, and the official release may still see adjustments.
v0.40.10-rc0 exposes hidden-state taps and raw logits for Gemma 4 to support speculative decoding.
40 more timeline events
All Facts (20)
Gemma 4's 26B MoE model uses a Mixture of Experts architecture, with 26 billion total parameters but only approximately 4 billion parameters active at any given moment.
80%VerifiedGemma 4 adopts the Apache 2.0 license, allowing completely unrestricted commercial use
75%VerifiedGemma 4的1B/2B模型量化后仅需4GB显存即可运行
65%Unverified全部四种Gemma 4模型都支持文本和图片输入，Edge模型（E2B和E4B）还额外支持音频输入
90%UnverifiedGemma 4支持完全离线状态下的图像文字提取（OCR），采用视觉-语言多模态架构
90%UnverifiedGemma 4大模型上下文窗口可达256,000 Token
90%Unverified所有Gemma 4模型原生支持多模态（文本、图像、最多30秒音频输入），上下文窗口为128K Token，支持超过40种语言
85%UnverifiedGemma 4引入了多模态能力，Qwen3支持超长上下文，DeepSeek采用MoE（混合专家）架构
85%Unverified使用Gemma 4系列模型验证了模型权重与Harness状态的联合学习效果，发现单独的监督微调和离线强化学习预热都不能让模型取得里程碑进展，只有联合学习循环才能产生持续提升
85%UnverifiedGemma 4的1B/2B模型自带语音识别功能
80%UnverifiedGoogle在Gemma 4中实际应用了MTP技术
80%UnverifiedGemma 4使用滑动窗口注意力（SWA）架构，该架构在处理数千行混淆JavaScript时导致模型遗忘早先读入的关键上下文
80%Partially VerifiedEmbeddingGemma 2 is a multimodal embedding model built by Google on the Gemma 4 architecture, capable of encoding text, images, audio, and video into a shared 768-dimensional vector space.
75%Unverified使用Gemma 4 2B版本本地运行Codex时，处理速度大约每秒30到40个Token
75%Unverifiedv0.40.10-rc0 exposes hidden-state taps and raw logits for Gemma 4 to support speculative decoding.
70%Partially VerifiedGoogle says Gemini 4 Argon helped refactor Chrome's massive codebase, completing months of engineering in just a few weeks, and increased internal agent code submissions by more than 35%.
65%UnverifiedChange #19247 is titled 'preserve small-image detail for compact Gemma 4 vision towers', aiming to preserve detail information in small-sized images when Gemma 4's compact vision encoders process images.
50%UnverifiedTF-IDF+logistic regression never predicts the two rarest categories, and on the type field it is statistically on par with Gemma 4.
50%UnverifiedUsers can explicitly select this renderer via RENDERER gemma4-nemotron, while the automatic detection logic for gemma4 remains unchanged, so existing users will not be affected by behavior changes.
50%UnverifiedSome models fine-tuned on Gemma 4 (such as the Nemotron series) adopt an assistant-content-first layout during training, and using the default renderer may cause tool call format misalignment or parsing failures.
50%