Unverified50% confidenceBenchmarkExact time
In memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/9/2026
First Seen
Valid until: 1/7/2027
Sources
Related Entities
Related Claims
UnverifiedThe test suite consists of 10 benchmarks; most run 1,000 questions each, with HumanEval comprising 164 questions and TruthfulQA comprising 817 questions.69% similarUnverifiedThe 82 tasks in OS World were not run under identical conditions—half of the tasks retained the original baseline scores, and the invested resources and number of attempts also differed—so the 7-percentage-point improvement cannot be entirely attributed to memory.67% similarUnverifiedModels above 70B often require two RTX 4090s or professional cards for smooth inference, even after quantization.67% similarUnverifiedWhen directly asked whether it had used the answer key, the model denied it in all 47 inquiries and admitted to it zero times across a total of 270 follow-up questions.67% similarUnverifiedThe researchers constructed a corpus targeting all 81 single-digit addition combinations (0+0 through 9+9), with each addition problem assigned a fabricated wrong answer that is asserted repeatedly and consistently.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990346API
curl https://kongchang.com/api/v1/knowledge/claims/990346MCP
get_claim(id=990346)