Gemini 3.7 Flash

Gemini 3.7 Flash calls Nano Banana when generating games to produce character and item textures as well as character animations.

Gemini 2.5 Flash adopts a Mixture of Experts (MoE) architecture, activating only a subset of parameters during inference.

The models tested include Google's Gemini 3.7 Flash, Z.ai's GLM-5, Alibaba's Qwen 3 235B A22B Instruct, and OpenAI's gpt-oss-120b, all running on Kaggle.

The test covered four vision models: Claude Sonnet 5 (Anthropic), Gemini 3 Flash (Google), Gemini 3.1 Flash-Lite (Google), and GPT-5.4 nano (OpenAI).

In Bench-Corp tests, the guardrail missed twice out of 520 runs, both on the same task with Gemini 3.7 Flash.

Gemini 3.7 Flash answered all 169 returned items correctly in the sufficiency task, but 23 calls yielded no result after three attempts; counting those as failures reduces the number of passing families from a perfect score to 28 out of 48.

On two-step reasoning questions, Claude Haiku 4.5 scored 6/30, GPT-5.4 mini scored 3/30, while Claude Sonnet 5.5, Gemini 3.8 Flash, and Gemma 4 all scored 30/30.

Across nine revised runs, the median completion speedup was 5.35× for Gemini, 3.69× for Claude, and 1.00× for GPT-5.4, with all three passing automated checks.

After restoring flushing, the Claude candidate achieved a completion speedup of 2.75× and a time-to-first-byte speedup of 2.80×, while Gemini achieved a completion speedup of 3.44× and a time-to-first-byte speedup of 3.54×, and GPT-5.4 achieved a completion speedup of 2.16× and a time-to-first-byte speedup of 2.19×.

The revised experiments use Gemini 3.7 Flash, Claude Sonnet 5.5, and GPT-5.4, with each model independently performing three optimization attempts, and prompts using temperature 0 and seed 0.

Qwen3-235B performed the worst on injection tests, failing 5 out of 8 attacks; Gemini 3.7 Flash, Claude Haiku 5.5, and Claude Opus 4.5 achieved zero failures across all 8 injection attacks.

On the author's faded receipt, Gemini completed ~ato as potato and ~cicum as capsicum, filling in a full nine lines in this manner and labeling all of them with 95% confidence, while also misreading faded SUGAR 1KG as MOONG 1KG.

Both models masked hidden digits in PNR or IFSC codes every time (0/25 fabrications), but filled in missing letters in words such as UMBRELLA or KITCHEN more than half the time.

Both models expressed almost identical confidence when fabricating versus when answering correctly (Gemini 93.7% vs 95.5%, Gemma 96.3% vs 99.0%), indicating that confidence scores cannot distinguish correct answers from fabrications.

When asked about a restaurant bill's GSTIN without any image attached, Gemini responded with 100% confidence with a perfectly formatted yet entirely fabricated tax number 29ADQFS1573H1Z1, answering 11 out of 12 times.

On clear text both models performed nearly perfectly (Gemini 89/96, Gemma 88/96), and neither corrected deliberately misspelled storefront signs.

In the easy-to-verify category, seven models scored a perfect 40/40 across all six bait types, and Claude Sonnet 5.5, Gemini 3.7 Flash, and GPT-6.1 Sol answered every question correctly across all six tasks in both the easy and hard categories.

The content creator speculates that Google may have introduced an IP-based quota mechanism, with the core logic being that rising demand for Gemini 3.7 Flash and a relatively fixed total quota have led to stricter quota management.

On Automation Bench, the old model scored 17%, while Gemini 3.7 Flash reached 30.4%.

On the Deep Squeeze test, the old model scored 49%, while Gemini 3.7 Flash jumped to 65.3%.