Unverified50% confidenceFactExact time
Open-loop evaluation compares the model's predicted trajectory with the dataset's human trajectory point by point; it is cheap to compute and reproducible, but suffers from "causal confusion" and may not truly respond to state changes caused by its own actions.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper finds a reversal in model dependence, suggesting that conclusions about cross-model transfer should be treated with caution and validated on a small scale on one's own model and data.74% similarUnverified对模型输出进行schema校验，一旦不符合预期结构就触发重试或降级路径73% similarUnverified多项实验表明，模型的表面推理与其实际决策机制之间存在显著偏差，通过激活修补可发现真正影响输出的内部路径与书面推理不吻合72% similarUnverifiedRun evals before switching models; judging by intuition that a small model is sufficient is very risky, and evaluation data is the basis for switching.72% similarUnverifiedEmbodied evaluation is having a model complete tasks in a simulated or real physical environment, involving action sequence planning, changes in environmental state, and causal inference, where the correctness of the task depends on the final environmental state rather than intermediate answers.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114478API
curl https://kongchang.com/api/v1/knowledge/claims/1114478MCP
get_claim(id=1114478)