AdvBench
A benchmark dataset for evaluating adversarial attacks, particularly jailbreak attacks, on large language models. It contains a variety of harmful instructions and adversarial prompts.
Timeline (last 90 days)
On AdvBench against DeepSeek-V4-Flash, SEMA-Auto achieves 76.0% ASR@1 within an average of 3.0 turns.
On AdvBench against DeepSeek-V4-Flash, the best static baseline achieves only 54.8% ASR@1 with a fixed 7 turns.
On AdvBench against DeepSeek-V4-Flash, the best interactive baseline achieves 43.8% ASR@1 in 3.7 turns.
AdvBench, released by Zou et al., contains over 500 harmful behavior instructions and adversarial suffixes.
All Facts (4)
On AdvBench against DeepSeek-V4-Flash, SEMA-Auto achieves 76.0% ASR@1 within an average of 3.0 turns.
50%UnverifiedOn AdvBench against DeepSeek-V4-Flash, the best static baseline achieves only 54.8% ASR@1 with a fixed 7 turns.
50%UnverifiedOn AdvBench against DeepSeek-V4-Flash, the best interactive baseline achieves 43.8% ASR@1 in 3.7 turns.
50%UnverifiedAdvBench, released by Zou et al., contains over 500 harmful behavior instructions and adversarial suffixes.
50%