τ²-Bench
A benchmark for evaluating the task-completion capabilities of tool-using LLM agents in real-world scenarios, including subsets such as telecommunications. It assesses agent accuracy and efficiency across multi-turn tool-calling interactions.
Timeline (last 90 days)
On τ²-Bench, SpecAgent's success rate is within less than one percentage point of the strong model, while memory traffic and wall-clock time are roughly halved.
SkillScaffold was evaluated on four benchmarks—ALFWorld, WebShop, ScienceWorld, and τ²-Bench—and surpassed strong agentic RL baselines and skill-driven RL baselines on both short-horizon and long-horizon tasks.
τ-bench is an agent evaluation benchmark proposed by the Sierra team, used to measure a model's policy compliance in "tool-agent-user" three-party interaction scenarios, covering two domains: airline customer service and retail e-commerce customer service.
The paper validates the effectiveness of CUE on the τ²-Bench multi-turn interaction evaluation benchmark.
τ²-Bench is an evaluation benchmark designed specifically for multi-turn task-oriented dialogue, using final task success as the core metric and recording complete dialogue trajectories.
Compared with other persona-based simulation methods, CUE-calibrated simulators produce fewer simulator-side errors (false failures).
CUE's experiments are conducted primarily on τ²-Bench with fitting data dominated by customer service interactions, and its calibration effectiveness in more specialized domains such as healthcare and law remains to be verified in subsequent work.
All Facts (7)
On τ²-Bench, SpecAgent's success rate is within less than one percentage point of the strong model, while memory traffic and wall-clock time are roughly halved.
50%UnverifiedSkillScaffold was evaluated on four benchmarks—ALFWorld, WebShop, ScienceWorld, and τ²-Bench—and surpassed strong agentic RL baselines and skill-driven RL baselines on both short-horizon and long-horizon tasks.
50%Unverifiedτ-bench is an agent evaluation benchmark proposed by the Sierra team, used to measure a model's policy compliance in "tool-agent-user" three-party interaction scenarios, covering two domains: airline customer service and retail e-commerce customer service.
50%UnverifiedThe paper validates the effectiveness of CUE on the τ²-Bench multi-turn interaction evaluation benchmark.
50%Unverifiedτ²-Bench is an evaluation benchmark designed specifically for multi-turn task-oriented dialogue, using final task success as the core metric and recording complete dialogue trajectories.
50%UnverifiedCompared with other persona-based simulation methods, CUE-calibrated simulators produce fewer simulator-side errors (false failures).
50%UnverifiedCUE's experiments are conducted primarily on τ²-Bench with fitting data dominated by customer service interactions, and its calibration effectiveness in more specialized domains such as healthcare and law remains to be verified in subsequent work.
50%