Unverified50% confidenceEventExact time
A developer shared a red-team experiment simultaneously on Bilibili and Dev.to: they built a simulated company with four types of agents—helpdesk, finance, code review, and release—and all 10/10 policy-rejection tests passed, but three of five tests that changed only the object and not the verb failed.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/18/2026
First Seen
Valid until: 11/1/2026
Sources
Related Entities
Related Claims
UnverifiedIn the demo, the agent first ran the tests to locate the problem with two failing tests, re-ran them after fixing, and ultimately all five tests passed, with a three-point summary.73% similarUnverifiedThe developer uses a retry loop of at most 3 attempts, feeding error information back to the model for regeneration upon validation failure.72% similarUnverifiedThe test repository chosen was python-humanize/humanize (MIT license, 1725 lines of Python, about 860 tests run in 7 seconds), pinned to a specific commit. The four tasks were: a bug fix for a real issue, implementing a new feature to spec (16 hidden acceptance tests), a refactor with no behavior change, and a code review (3 planted bugs).71% similarUnverifiedThe team proposed adding a dedicated test that enumerates each link in the chain by name, asserting that each exists in the registration path, actually runs, and fails when its test target is disabled.71% similarUnverified16 out of 28 tests in the LRO-Bench implementation recommendations cannot be determined by the few executions actually run by the benchmark68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1057372API
curl https://kongchang.com/api/v1/knowledge/claims/1057372MCP
get_claim(id=1057372)