[KongchangAI]
· 3 min read· 1,580 words

Gemini 4 Neon Hands-On Test: Best for Coding but Nearly Twice as Slow — How Do You Account for That?

Gemini 4 Neon Hands-On Test: Best for Coding but Nearly Twice as Slow — How Do You Account for That?

Neon ranks first in coding agents with per-task cost only half of Claude, but takes nearly twice as long as the median.

Overseas reviewers conclude: Gemini 4 Neon ranks first on the Coding Agent Index with 59 points, but its overall intelligence index of 61 places it third. Per-task cost averages $6.86, about half of Claude, but the cheapness comes from low unit price, not economical usage — output tokens per task are 15% more than the previous generation, totaling 283 million. Speed looks fast at 110 tokens per second, yet end-to-end takes 21.5 minutes, nearly twice the 12.1-minute median, and only 40% faster than the previous generation's 36 minutes. Capability showcases include pure code-generated pixel worlds and quadruped robot dogs. The conclusion: Neon is good enough and cheap — it saves the bill but costs you waiting, and whether it's worth it depends on whether your work is batch runs or time-sensitive.

Title

Gemini 4 Neon Hands-On Test: Best for Coding but Nearly Twice as Slow — How Do You Account for That?

Meta Description

Overseas reviewers put Gemini 4 Neon to the test: ranked first on the Coding Agent Index with per-task cost about half that of Claude, but each task takes nearly twice as long as the median of its peers. This article breaks down third-party data on cost, speed, and capability to help you decide whether the bill is worth it.

Article Content

The overseas review community has finished running the numbers on the new model "Neon" (Gemini 4 Neon), and the conclusion surprised quite a few people: it took first place in the coding agent category, yet it's nearly twice as slow as the median of its peers. Tonight we won't talk about the buzz of the launch event — we're just going to do the math: cheap or fast, which one do you pick?

Reviewers Agree: First in Coding, Upper-Middle Overall

According to the intelligence index from third-party organization AA, Neon's top tier scored 61 points, placing third, behind Fable 5.2 at 65 and Astrid at 62. On the coding agent ranking, it placed sixth among 96 models. That's upper-middle among flagship models — not weak, but not the strongest either, and the two ranked above it are both more expensive.

What's more telling is the Coding Agent Index: Neon ranks first with 59 points, while Fable 5.2 and Opus 5.5 both score 58 and Carbon scores 55. According to Google's official blog citation, Neon scores 58.6 on that leaderboard, also first. It's worth clarifying that this is first place in the coding category, not first place overall.

No external assets

Coding Agent Index is a type of evaluation specifically designed to measure how models perform on "autonomous coding tasks." Unlike ordinary code completion or single-turn generation, it requires the model to independently complete tasks within loops of multi-step tool calls, file reading and writing, test execution, and error fixing, ultimately scored by task pass rate or quality standards. These leaderboards are closer to real engineering scenarios, so developers consider them more grounded than traditional indicators like HumanEval and SWE-bench. The AA Coding Agent Index mentioned here is exactly this kind of evaluation; Neon tops it with 59 points, showing an advantage in "end-to-end getting the job done." But that is not equivalent to rankings on general-purpose leaderboards covering reasoning, math, or multilingual ability — readers need to distinguish a "single-category champion" from an "overall first place."

The Cost Math: Per-Task Cost About Half of Claude

What really got the review community excited was the price. Official pricing is $3 input and $15 output. Third-party AA measured the average cost per task using coding tools: Neon comes in at $6.86, while Opus 5.5 and Fable 5.2 are considerably higher — roughly half of Claude, meaning the competitor pays twice as much for the same work.

But there's a counterintuitive reason behind the cheapness: AA points out that Neon uses 15% more output tokens per task than the previous generation, with total output reaching 283 million tokens versus a median of just 65 million. It's not more economical — it's just that the unit price is low and the usage is actually higher. So don't assume it's frugal with usage; what it saves is the bill.

Average cost per task

The pricing difference between output tokens and input tokens is key to understanding the cost math. In mainstream API pricing, output tokens typically cost several times more than input tokens (Neon is $3 input / $15 output, a 1:5 ratio), because the generation process involves more computation. Therefore, a model that "uses more output tokens per task" will significantly inflate the bill even if the unit price is cheap. In this case, Neon outputs 283 million tokens per task — more than four times the median of 65 million — yet still keeps per-task cost down to $6.86, showing that its unit-price advantage is enough to offset its usage disadvantage. Conversely, if a task is sensitive to output volume (such as long-form generation or large-scale code refactoring), Neon's cost advantage will be diluted. When estimating your own bill, you should calculate "unit price × usage" together rather than looking only at unit price.

The Speed Math: Fast at Typing, Slow at Finishing the Job

The speed data deserves the closest look. AA measured Neon outputting 110 tokens per second, which looks fast — the previous generation was 57, nearly twice as fast. But each task takes 21.5 minutes, while the median across all models is 12.1 minutes — nearly twice as slow as the median. The original text also notes: it's actually 40% faster than the previous generation's 36 minutes. And you have to wait 56 seconds for the first token.

In one sentence: slower than the peer median, faster than the previous generation. Typing fast and finishing the job fast are two different things, and the waiting time is real. If you want speed, you'll have to pay more.

In one sentence: slower than the peer median, faster than the previous generation

Tokens per second (TPS) and end-to-end task duration are two indicators at different levels. TPS measures the instantaneous speed at which the model "spits out words," affecting the fluency users perceive; end-to-end duration also includes first-token latency (TTFT), tool-call round trips, multi-turn reasoning and retries, and other overhead. In coding agent scenarios, models often need dozens of tool calls and self-corrections, and the accumulation of these serial steps dominates total time — no matter how high the TPS, it's hard to make up for that. This explains Neon's "contradiction": 110 TPS is about twice the previous generation, yet 21.5 minutes per task is still slower than the 12.1-minute median. When evaluating model speed, task-level duration should be the basis for decisions, with TPS as reference only.

Capability: Raw Footage of Pure Code Generation

On the capability showcase side, someone used Neon to build a pixel wizard world, running on pure code with no external assets — every visual was written by the model itself. Someone else gave it just one image and it built a quadruped robot dog, with the gait entirely computed. The official blog also mentions it made an introductory animation for itself, purely code-generated with not a single external asset, and could build a full set of operable 3D historical flip weather vane bars from a single image. Neither clip is long, but you can see what it can do.

All written by itself

According to Google's official blog citing third-party leaderboards, Neon took first place in two categories: automated testing at 56.3% and business simulation at $16,843; coding tests improved from 12.8% to 39% after tools were used — tools matter more than the model itself. Another third-party leaderboard shows 70.9%, ranking second. These are leaderboards set up by the third-party organizations themselves, which is not the same thing as the official ones run in-house.

The improvement in coding test scores from tool use deserves separate explanation. The article mentions coding tests improving from 12.8% to 39% after tools were used, revealing an often-overlooked fact: the performance of modern coding agents depends largely on "whether they can use tools correctly" (running tests, reading files, checking documentation) rather than the model's raw ability. The same model may perform mediocrely in pure generation mode but multiply its results several times over once connected to tools. This also means comparability between leaderboards is limited — each organization configures different tool sets, retry counts, and timeout strategies for models, so scores cannot be directly compared across them. When readers see "first place on some leaderboard," they should first confirm whether its evaluation protocol matches their own way of using the model.

Should You Use It: It Depends on What Type of Work You Have

Running batches and can't afford to ship? It's a good deal. In a hurry and need speed? Don't skimp on this. What Neon is going after isn't being the strongest — it's being good enough and cheap: what it saves is the bill, what it costs you is waiting. What type of work you have on hand determines whether it's worth it. Cheap or fast — which do you choose? The evidence is all in the footage.

One-line Summary

Neon ranks first in coding agents with per-task cost only half of Claude, but takes nearly twice as long as the median.

Paragraph Summary

Overseas reviewers' conclusion on Gemini 4 Neon: it ranks first on the Coding Agent Index with 59 points, but its overall intelligence index of 61 points places it only third. On cost, the average per-task cost is $6.86, about half of Claude — but the cheapness comes from a low unit price rather than economical usage: output tokens per task are 15% more than the previous generation, totaling 283 million. On speed, 110 tokens per second looks fast, but end-to-end it takes 21.5 minutes, nearly twice as slow as the 12.1-minute median across all models, and only 40% faster than the previous generation's 36 minutes. Capability showcases are backed by raw footage of pure code-generated pixel worlds, quadruped robot dogs, and more. The article's conclusion: Neon is positioned as good enough and cheap — it saves the bill but costs you waiting, and whether it's worth it depends on whether your task is batch runs or time-sensitive work.

#Gemini 4 Neon#Coding Agent Index#model evaluation#API pricing#output tokens#TPS#end-to-end task duration#Claude comparison
Share:

Related articles