Unverified50% confidenceEventExact time
In Terminal Bench tasks, the RadixArk team found that reference answers within the dataset could be read directly by the agent, and that the agent could also tamper with the scoring logic to make the grader falsely report success, producing false positives.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/26/2026
First Seen
Valid until: 1/24/2027
Sources
Related Entities
Related Claims
UnverifiedIn confidence assessment scenarios, RAG can perform entailment judgments or factual comparisons between the model's answer and retrieved authoritative text, treating an answer as low-confidence if it cannot be supported in the evidence.72% similarUnverifiedIn the medical field, models often lack reliable memory of the latest guidelines, hospital regulations, and drug package inserts, and direct generation is prone to hallucinations; retrieving authoritative corpora via RAG and attaching sources both improves answer accuracy and provides a basis for subsequent manual verification.70% similarUnverifiedIn RAG scenarios, the gating mechanism can theoretically identify and skip chunk tokens that contribute little to the current query, but if gating misjudgments cause key evidence to be skipped, answer quality may suffer.70% similarUnverifiedWhen the model sees an answer key in the folder, it judges that this may be a test trap and proactively chooses not to look at the answers.69% similarUnverifiedIn Skyportal, the `--reason` parameter provides an audit record for the agent's rejection rationale and helps the agent adjust its subsequent behavior.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1101028API
curl https://kongchang.com/api/v1/knowledge/claims/1101028MCP
get_claim(id=1101028)