[KongchangAI]
· 1 min read· 963 words

New Breakthrough in Multi-Agent System Security: SecMAS Defends Against Response Manipulation Attacks

New Breakthrough in Multi-Agent System Security: SecMAS Defends Against Response Manipulation Attacks

The paper proposes the Response Manipulation Attack ReMA and designs the SecMAS aggregation rule based on semantic similarity to achieve secure defense.

The article introduces a study on the security of the aggregation stage in multi-agent systems (MAS). MAS handle complex problems through task decomposition and result aggregation, but each agent's output can influence the final decision, so malicious agents can provide malicious responses to manipulate the final solution. The researchers propose the Response Manipulation Attack ReMA, modeling untargeted and targeted attacks as optimization problems respectively and iteratively refining malicious responses with feedback from a surrogate system, which can be carried out without knowledge of the system's internal details. To defend against this attack, the paper proposes the SecMAS aggregation rule, which leverages semantic similarity among responses and selects the response most consistent with a sufficiently large subset of responses as output, and theoretically proves its security guarantees against both types of attacks. Experiments show that existing aggregation rules are generally vulnerable, while SecMAS can resist all evaluated attacks, including adaptive variants, while maintaining utility comparable to existing rules.

Multi-agent systems solve complex problems through task decomposition and result aggregation, but their distributed nature also introduces new attack surfaces. A recent paper proposes the Response Manipulation Attack (ReMA) and designs the SecMAS aggregation rule to defend against such threats.

Multi-agent system security design

Vulnerability of Multi-Agent Systems: Response Manipulation Attacks

Multi-agent systems (MAS) decompose complex tasks into subtasks, assign them to multiple agents, and then aggregate the results through an aggregation rule. This architecture improves parallelism and flexibility in task processing, but it also means that each agent's output can influence the final decision. The paper points out that malicious agents can manipulate the final solution by providing malicious responses to their assigned subtasks.

To this end, the researchers propose the Response Manipulation Attack (ReMA). This attack iteratively optimizes malicious responses to maximize the attack effect. Specifically, the researchers model untargeted and targeted attacks as optimization problems respectively, and iteratively refine malicious responses based on feedback from a surrogate multi-agent system to approximately solve these problems. This attack method does not rely on complete knowledge of the system's internal details, making it practically feasible.

Untargeted attacks and targeted attacks are standard categories in adversarial attack research. An untargeted attack only requires the final output to deviate from the correct solution or cause the system to fail, without specifying a particular result; a targeted attack requires the system to output a specific answer predetermined by the attacker. The two impose different capability requirements on the attacker and differ in defense difficulty—targeted attacks require finer control over response content but also lead to more severe consequences. The paper models the two types of attacks as optimization problems separately precisely to uniformly measure the robustness of aggregation rules under both threat models within the same framework.

Iterative optimization based on feedback from a surrogate system draws on the idea of transfer attacks in black-box adversarial attacks. The attacker does not directly access the target multi-agent system's aggregation rule or internal prompts; instead, it first builds a structurally similar surrogate system, repeatedly submits candidate malicious responses on it, observes the aggregation results, and updates the responses accordingly, so that the malicious responses gradually approach a form capable of manipulating the target system. Since different MAS are often isomorphic in task decomposition and aggregation logic, responses optimized on the surrogate system are transferable, which explains why the attack remains practically feasible without complete knowledge of the system's internal details.

SecMAS: A Secure Aggregation Rule Based on Semantic Similarity

To defend against ReMA attacks, the paper proposes SecMAS—a secure aggregation rule. Its core idea is to leverage semantic similarity among agent responses and select the response most consistent with a sufficiently large subset of responses as the final output. This means that for a malicious response to be consistent with the majority of responses, it must be extremely cleverly disguised semantically, thereby greatly increasing the difficulty of attack.

The researchers theoretically prove that SecMAS provides security guarantees against both untargeted and targeted attacks. This formal guarantee provides a theoretical foundation for the design of secure aggregation rules, making them not only effective in practical evaluations but also supported at the theoretical level.

Semantic similarity here refers to mapping agent responses into high-dimensional vectors via text embeddings and then measuring the closeness between responses using metrics such as cosine similarity. SecMAS does not compare literal wording but compares semantic vectors, so responses that are paraphrased, differently worded, or phrased differently but similar in meaning will be regarded as consistent.

The key to the criterion of "most consistent with a sufficiently large subset of responses" lies in the threshold: a response can be selected as output only when its semantic neighborhood contains more than a certain proportion of agent responses. The implicit assumption of this mechanism is that honest agents tend to be semantically consistent, while malicious responses are difficult to align with the majority without being captured by semantic similarity—this is precisely the source of its security guarantee.

Experimental Evaluation: Effective Defense and Utility Preservation

Extensive evaluations show that existing aggregation rules are vulnerable to ReMA attacks, while SecMAS can effectively defend against all evaluated attacks, including adaptive variants of ReMA. More critically, SecMAS maintains a level of utility comparable to existing aggregation rules while providing security. This means that the improvement in security does not come at the expense of system performance, which is crucial for practical deployment.

This research reveals the security risks of multi-agent systems at the aggregation stage and provides a design approach that balances security and utility. As multi-agent systems are increasingly applied to complex tasks, such "secure-by-design" methods will provide important references for system builders.

One-line Summary

The paper proposes the Response Manipulation Attack ReMA and designs the SecMAS aggregation rule based on semantic similarity to achieve secure defense.

Paragraph Summary

The article introduces a study on the security of the aggregation stage in multi-agent systems (MAS). MAS handle complex problems through task decomposition and result aggregation, but each agent's output can influence the final decision, so malicious agents can provide malicious responses to manipulate the final solution. The researchers propose the Response Manipulation Attack ReMA, modeling untargeted and targeted attacks as optimization problems respectively and iteratively refining malicious responses with feedback from a surrogate system, which can be carried out without knowledge of the system's internal details. To defend against this attack, the paper proposes the SecMAS aggregation rule, which leverages semantic similarity among responses and selects the response most consistent with a sufficiently large subset of responses as output, and theoretically proves its security guarantees against both types of attacks. Experiments show that existing aggregation rules are generally vulnerable, while SecMAS can resist all evaluated attacks, including adaptive variants, while maintaining utility comparable to existing rules.

#multi-agent systems#response manipulation attack#ReMA#SecMAS#semantic similarity#secure aggregation#adversarial attacks
Share:

Related articles