ARBITRA: A Multi-Expert Argumentation Framework for Robust Document-Level ATT&CK Extraction

Cyber threat intelligence (CTI) reports typically describe attacker behavior in natural language, but converting them into structured MITRE ATT&CK mappings remains a challenge. Existing methods often decompose reports into sentence-level granularity for extraction, while modern large language models (LLMs), though capable of processing an entire report in a single pass, suffer from significant performance degradation. To address this, researchers propose ARBITRA—a novel framework for robust document-level ATT&CK extraction via multi-expert argumentation.

Limitations of Existing Methods and ARBITRA's Breakthrough

Current ATT&CK extraction mainly faces a trade-off between two technical routes. Sentence-level pipeline methods decompose reports into sentences and extract entities sentence by sentence, sacrificing the document's global context and causing cross-sentence relational information to be lost. Directly using an LLM to process the full report preserves context, but the model's extraction accuracy on long documents drops significantly, especially when the report contains ambiguous statements or contradictory claims.

ARBITRA's core innovation is the introduction of a multi-role expert LLM collaboration mechanism. Multiple role-specialized LLM experts propose candidate ATT&CK entities from the full report, and these candidates are then validated through a per-candidate quantitative bipolar argumentation framework (QBAF) based on document evidence and reasoning rationales. This design preserves document-level context while improving extraction precision through the argumentation process.

MITRE ATT&CK is a public knowledge base of adversarial tactics and techniques that distills real attack behaviors into a two-level taxonomy of tactics (e.g., initial access, lateral movement) and techniques (e.g., phishing attachments, credential dumping), assigning each technique a unique identifier (e.g., T1566). If natural-language descriptions in threat intelligence reports are mapped to this identifier system, they can be integrated with security operations platforms, detection rules, and threat hunting workflows to enable automated correlation analysis. Therefore, ATT&CK extraction is essentially an entity linking task that aligns unstructured text to a controlled ontology. Its difficulty lies in the fact that the same attack behavior can be expressed in many ways, a single technique often spans multiple sentences, and report authors may not use ATT&CK's standard terminology.

Multi-Expert Argumentation and QBAF Validation Mechanism

ARBITRA's workflow is divided into two stages: proposal and validation. In the proposal stage, multiple role-specialized LLM experts each identify possible ATT&CK entities from the full report, forming a candidate set. Different expert role settings make them focus on different dimensions of attack behavior, thereby covering a more comprehensive candidate space.

The validation stage employs a quantitative bipolar argumentation framework (QBAF). Each candidate entity undergoes an argumentative evaluation based on document evidence and reasoning rationales—supporting evidence and opposing evidence are quantified and weighed to ultimately decide whether the candidate is adopted. This per-candidate argumentation approach enables the system to handle contradictory statements in reports and reduce noise interference.

To enhance the effectiveness of expert participation in the validation process, the researchers also constructed a reasoning-specialized corpus for fine-tuning, enabling expert models to better play their argumentative roles.

The quantitative bipolar argumentation framework (QBAF) originates from computational argumentation theory and is used to compute evaluations of arguments in the presence of supporting and opposing opinions. Its basic structure is as follows: each argument is assigned an initial base score, arguments are connected via "support" edges and "attack" edges, and iterative semantic rules (such as weighted summation or the Euler formula) aggregate the positive and negative influences from other arguments to ultimately yield each argument's strength. In ARBITRA, a candidate ATT&CK entity is the argument to be evaluated, evidence fragments in the document constitute sub-arguments that support or attack it, and the framework produces comparable confidence values, thereby transforming "whether to adopt this candidate" into a quantifiable, interpretable argumentation result rather than a black-box model score.

Performance and Robustness Advantages

On the ATT&CK extraction benchmark, ARBITRA achieves a stronger precision-oriented operating point than sentence-level pipelines, recent LLM-based extraction systems, and frontier models including GPT-5.5. This means that in scenarios pursuing high precision, ARBITRA can provide more reliable extraction results.

Robustness tests further demonstrate ARBITRA's advantages. Under adversarial perturbations simulating conflicting TTP statements, cross-report contamination, and semantic poisoning, ARBITRA limits average precision degradation to approximately half that of frontier model baselines. This result indicates that the multi-expert argumentation mechanism can effectively resist adversarial interference and maintain stable extraction quality.

The precision-oriented operating point refers to selecting a threshold position on the precision-recall trade-off curve that prioritizes high precision. In threat intelligence scenarios, this orientation is practically justified: incorrect mappings can mislead detection rule writing and incident response, incurring high verification costs, whereas missing a few techniques can usually be compensated in subsequent analysis. Therefore, evaluation not only looks at F1 but also focuses on system performance under the condition of "better to under-report than misreport," and ARBITRA is compared with sentence-level pipelines and frontier models precisely at this operating point.

Significance for Threat Intelligence Automation

ARBITRA's value lies in reconciling the tension between document-level context utilization and precise extraction through an argumentation mechanism. For threat intelligence analysts, automatically extracting accurate ATT&CK mappings from lengthy CTI reports can greatly reduce the manual annotation burden, while improved robustness means the system can still provide trustworthy output when facing noisy or adversarial data.

The framework's multi-expert argumentation approach also offers a reference for other structured information extraction tasks—when a single model struggles to balance contextual completeness and extraction precision, introducing multi-role collaboration and formal argumentation validation may be an effective path.

As LLMs are applied more deeply in cybersecurity, how to ensure the precision and robustness of extraction results will remain a focus of ongoing attention. ARBITRA's proposal marks an important step toward the practical application of document-level ATT&CK extraction.

One-line Summary

ARBITRA combines multi-expert LLM proposals with QBAF argumentation validation to achieve robust document-level ATT&CK extraction.

Paragraph Summary

The article introduces ARBITRA, a document-level MITRE ATT&CK extraction framework for cyber threat intelligence (CTI) reports. Existing methods face a dilemma: sentence-level pipelines that extract sentence by sentence lose cross-sentence context, while directly using large language models to process full reports leads to degraded extraction precision, especially with ambiguous or contradictory statements. ARBITRA has two stages—proposal and validation: multiple role-specialized LLM experts propose candidate ATT&CK entities from the full report, then a quantitative bipolar argumentation framework (QBAF) weighs supporting and opposing opinions for each candidate based on document evidence and reasoning rationales to decide adoption; the researchers also built a reasoning-specialized corpus to fine-tune expert models. Experiments show it outperforms sentence-level pipelines, LLM extraction systems, and even frontier models such as GPT-5.5 at a high-precision operating point, and under adversarial perturbations such as conflicting statements, cross-report contamination, and semantic poisoning, it limits precision degradation to about half that of frontier baselines, balancing contextual completeness and extraction precision.