Unverified50% confidenceBenchmarkExact time
In a test of 48 synthesized programming requests, under the default four-option question Laya matched the team's annotation on only 21 of them, with a maximum confidence of only 0.26.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedThe limitations of this test include: limited task scale (50 tasks, 3 trials per prompt), style versions written by a single author, length confounding between the florid and normal versions, testing only English and local models below 14B, and the perplexity check covering only three smaller Qwen models and using full-precision weights while actual inference used 4-bit quantization.69% similarUnverifiedOf the 46 models, 41 completed all 28 questions, 3 did not finish due to service errors, GPT-5.6 Terra stopped at 24/28 due to quota limits, and GPT-6 Astra only ran 1 question.68% similarUnverifiedThe test results showed that 27 out of 30 tickets were routed to the correct team.68% similarUnverifiedIn a PR review comparison test, Opus found 23 valid issues (including 8 nits), Muse found 16, and Sol found only 9.67% similarUnverifiedIn a small-scale test published by the PartsPlan developer, all 30 chart values were answered correctly.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037626API
curl https://kongchang.com/api/v1/knowledge/claims/1037626MCP
get_claim(id=1037626)