Task completion rate
A metric measuring whether an agent successfully satisfies the user's original request. It is a core quantitative criterion for evaluating agent utility and often stands in tension with attack success rate.
Timeline (last 90 days)
The difficulty of agent evaluation lies in the fact that traditional static benchmarks struggle to measure the quality of dynamic decision-making in multi-step interactions; the dimensions the community currently focuses on include task completion rate, tool-calling accuracy, long-term planning consistency and robustness in adversarial environments.
Task completion rate refers to the proportion of goals completed independently without human intervention, and the resource completion ratio equals value divided by cost.
Task completion rate is defined as the proportion of given programming tasks completed without human intervention, and better reflects real-world usability than traditional metrics such as BLEU scores.
All Facts (3)
The difficulty of agent evaluation lies in the fact that traditional static benchmarks struggle to measure the quality of dynamic decision-making in multi-step interactions; the dimensions the community currently focuses on include task completion rate, tool-calling accuracy, long-term planning consistency and robustness in adversarial environments.
50%UnverifiedTask completion rate refers to the proportion of goals completed independently without human intervention, and the resource completion ratio equals value divided by cost.
50%UnverifiedTask completion rate is defined as the proportion of given programming tasks completed without human intervention, and better reflects real-world usability than traditional metrics such as BLEU scores.
50%