CoSA-Bench
A benchmark for evaluating the CoSA skill-composition attack on LLM agents. Built on real-world skills and controlled services, it covers 288 tasks across four domains—recruitment, finance, IT, and business—and assesses three risk categories: data leakage, unauthorized state modification, and unsafe execution.
Timeline (last 90 days)
The researchers built CoSA-Bench based on real-world Skills and controlled services, covering four domains—recruitment, finance, IT and business—with 288 tasks in total, involving three risk types: data leakage, unauthorized state modification, and unsafe execution.
In tests across five LLMs and three Agent harnesses, with newly added Skills capped at only 5% of each original pool, the policy violation rate rose from 35.02% to 58.45%, an increase of 23.43 percentage points.
All Facts (2)
The researchers built CoSA-Bench based on real-world Skills and controlled services, covering four domains—recruitment, finance, IT and business—with 288 tasks in total, involving three risk types: data leakage, unauthorized state modification, and unsafe execution.
50%UnverifiedIn tests across five LLMs and three Agent harnesses, with newly added Skills capped at only 5% of each original pool, the policy violation rate rose from 35.02% to 58.45%, an increase of 23.43 percentage points.
50%