Task Success Rate
A core metric in agent evaluation that measures the proportion of ordinary tasks whose produced artifacts match a frozen oracle byte for byte. When used alone, it may conceal failures on refusal slices, so it must be read together with metrics such as abstention accuracy and invalid completion rate.
Timeline (last 90 days)
Task success rate is the most commonly used aggregate metric in robot learning benchmarks, usually defined as the proportion of trials in which a predefined goal is achieved.
Task success rate has an attribution blind spot: it cannot distinguish whether the agent succeeded by chance or by anticipating and verifying before succeeding, nor can it distinguish perception errors from planning failures.
The industry commonly measures agent capability by task success rate rather than single-demo performance: repeat the same task many times and see how many times it can deliver a qualified result in one pass, and whether it can self-detect and correct when it fails.
Task success must be a verifiable business end state (the database was actually written, the file was actually created on disk), not just the model claiming it has completed the task.
All Facts (4)
Task success rate is the most commonly used aggregate metric in robot learning benchmarks, usually defined as the proportion of trials in which a predefined goal is achieved.
50%UnverifiedTask success rate has an attribution blind spot: it cannot distinguish whether the agent succeeded by chance or by anticipating and verifying before succeeding, nor can it distinguish perception errors from planning failures.
50%UnverifiedThe industry commonly measures agent capability by task success rate rather than single-demo performance: repeat the same task many times and see how many times it can deliver a qualified result in one pass, and whether it can self-detect and correct when it fails.
50%UnverifiedTask success must be a verifiable business end state (the database was actually written, the file was actually created on disk), not just the model claiming it has completed the task.
50%