Unverified50% confidenceBenchmarkExact time
A controlled benchmark containing 24 worlds (each with exactly two possible outcomes) reproduces the failure of the complementary consistency test on all three models (Qwen3-4B, Llama-3.2-3B, Gemma-3-4B).
1
Sources
50%
Confidence
Long-term
Relevance
10/26/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe test covered 8 local models: Qwen3's 0.6B, 1.7B, 4B, 8B, and 14B, as well as Gemma3's 1B, 4B, and 12B, all deployed via Ollama with thinking mode disabled.71% similarUnverifiedThe author noted that other models previously tested, such as 35B and 3B variants, actually performed worse than the Q2 version of Qwen3 27B on the Kanban task.71% similarUnverifiedThis test is a single-source personal test; the model designation is labeled as Qwen 3.8 27B with naming ambiguity, and details such as the actual configuration, quantization method, and generation time have not been disclosed, leaving reproducibility to be verified.70% similarUnverifiedAmong the three tested models, only Qwen2.5-3B can reliably refuse to answer when the evidence does not support an answer.70% similarUnverifiedIn the first round of tests (23 synthetic cases × 3 runs), qwen2.5:1.5b achieved only a 30% correct-judgment rate and qwen2.5:7b only 46%, both failing; qwen2.5:7b incorrectly judged "covered" in 14% of cases.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1098435API
curl https://kongchang.com/api/v1/knowledge/claims/1098435MCP
get_claim(id=1098435)