RareTrap
A sampling-based black-box adversarial attack strategy that randomly samples latent variables in a continuous latent space and decodes them into prompts, searching for samples that trigger anomalous behavior in the target model.
Timeline (last 90 days)
This study evaluated the dLLM mapping method under two attack settings: sampling-based RareTrap and optimization-based ThinkTrap.
In the sampling-based RareTrap, using dLLM mapping reduced character-level perplexity from 10^3–10^7 to 3–5.
RareTrap is sampling-based: it randomly samples latent variables in a continuous latent space and decodes them into prompts, searching for samples that can trigger abnormal behaviors such as overthinking; ThinkTrap is optimization-based: it uses gradient estimation or evolutionary strategies to optimize latent variables so that the model produces high-risk internal reasoning.
All Facts (3)
This study evaluated the dLLM mapping method under two attack settings: sampling-based RareTrap and optimization-based ThinkTrap.
50%UnverifiedIn the sampling-based RareTrap, using dLLM mapping reduced character-level perplexity from 10^3–10^7 to 3–5.
50%UnverifiedRareTrap is sampling-based: it randomly samples latent variables in a continuous latent space and decodes them into prompts, searching for samples that can trigger abnormal behaviors such as overthinking; ThinkTrap is optimization-based: it uses gradient estimation or evolutionary strategies to optimize latent variables so that the model produces high-risk internal reasoning.
50%