Cognitive Demand Assessment
A method that decomposes a task into cognitive dimensions, such as knowledge recall, quantitative reasoning, logical reasoning and working memory, and analyzes the contribution of each dimension to model performance. It is used to reveal the cognitive mechanisms underlying model behavior.
Timeline (last 90 days)
Model evaluation experiments based on cognitive demands show that quantitative reasoning and logical reasoning are not the main drivers of performance on temporal reasoning benchmarks; knowledge is the core driver.
Cognitive demand assessment is a method for understanding model performance by analyzing the cognitive abilities required by a task, typically decomposing the task into multiple cognitive dimensions and experimentally determining each dimension's contribution to model performance.
All Facts (2)
Model evaluation experiments based on cognitive demands show that quantitative reasoning and logical reasoning are not the main drivers of performance on temporal reasoning benchmarks; knowledge is the core driver.
50%UnverifiedCognitive demand assessment is a method for understanding model performance by analyzing the cognitive abilities required by a task, typically decomposing the task into multiple cognitive dimensions and experimentally determining each dimension's contribution to model performance.
50%