Unverified50% confidenceBenchmarkExact time
In a comparative test on 29 real blunders, the bare Gemma model made assertions not confirmed by the engine 18 out of 29 times.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedGemma 4 26B terminated with a runtime error before producing a score, because the scoring code assumed at least one sentence would be returned in the correct structured format.69% similarUnverifiedAfter changing the header identification to have Gemma 4 E2B fill only the three required gaps of date, quantity, and product, all 12 header sets were correct with 0 errors, but the author cautions that the same batch of data was also used for debugging, so 12/12 should be regarded as an optimistic estimate.68% similarUnverifiedWhen Gemma wrote entire messages directly, it dropped all numbers in 4 out of 4 tests and completely reversed the meaning twice.64% similarUnverifiedAcross a range of question sizes spanning 256x, the error judgment rate of a single run is almost unchanged: 0.531 at T=16 and 0.534 at T=4096.64% similarUnverifiedAfter re-evaluation, Gemma 12B in non-thinking mode achieved a substantive pass rate of 12/16, while the strict pass rate remained at 4/16.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053951API
curl https://kongchang.com/api/v1/knowledge/claims/1053951MCP
get_claim(id=1053951)