Selective Retrieval Routing: A New Paradigm for Multi-Violation Knowledge Graph Repair

Knowledge graphs serve as carriers of structured knowledge, and their integrity directly affects the reliability of downstream applications. However, real-world knowledge graphs often contain missing or contradictory facts, leading to biases in applications built on them. Traditional schema validation can detect violations but cannot identify the most intent-aligned repair among multiple valid options. A recent paper proposes a selective retrieval framework that uses intelligent routing to control the acquisition of external evidence, achieving more efficient repair in iterative graph repair.

Challenges in Knowledge Graph Repair and Limitations of Existing Methods

Knowledge graph repair is not simply error detection. When schema validation finds a violation, there are often multiple repairs that satisfy the constraints, but only one truly restores the graph's original intent. For example, a missing attribute value may have multiple candidate fillers that all meet schema requirements but differ greatly in semantics.

External corpora can provide necessary evidence to identify the correct repair, but retrieval is costly and not always successful. Existing methods tend to either rely entirely on external retrieval, wasting resources, or ignore external information entirely, limiting repair quality. This dilemma has created a need for more fine-grained retrieval control.

Schema Validation is a foundational means of ensuring knowledge graph quality. Its principle is to check the consistency of triples in the graph against predefined constraint rules (such as ShEx, SHACL, or OWL axioms). These constraints can specify property cardinality (e.g., "each building must have at least one address"), value ranges (e.g., "temperature values must be numeric"), or type hierarchies. When the graph violates a constraint, the validator reports the violation location but usually does not provide repair suggestions. The advantage of schema validation is that it is automatable and interpretable, but its limitation is also obvious: it can only determine "whether it is compliant," not "which repair better matches the original intent." For example, for a violation stating "Building A's temperature is 'hot'," one could change the value to a number or delete the property; both are compliant but semantically different. Therefore, schema validation is the starting point of the repair process, not the endpoint; subsequent disambiguation requires external evidence or reasoning.

Core Design of the Selective Retrieval Routing Framework

The framework proposed in the study contains two key components: an LLM-based repair agent responsible for exploring and editing the graph, and a router that dynamically controls the repair agent's access to a researcher agent. The researcher agent retrieves evidence from external corpora.

The router's core role is to decide when external evidence is needed. During the repair agent's iterative process, the router decides whether to invoke the researcher agent based on the current violation type and context. This selective access mechanism avoids unnecessary retrieval overhead while retaining the ability to obtain external support at critical moments.

To train deployment-specific routers, the researchers extended the violation injection framework to generate test cases containing multiple compatible violations with known reference repairs. The repair agent's performance on these cases provides supervision signals for optimizing the routing policy.

The LLM Repair Agent plays the role of "graph editor" in this framework. Unlike traditional rule-based repair, an LLM agent can understand natural-language descriptions of violations and local graph context, thereby proposing candidate edit operations (such as inserting, deleting, or modifying triples). The agent works iteratively: each round it observes the current violation state, decides the next action, and determines whether external evidence is needed. This LLM-based agent design gives the system generalization ability to handle unseen violation types, because the LLM's pretrained knowledge can serve as a prior. However, the LLM itself may hallucinate or be overconfident, so the router needs to introduce external evidence at critical moments for calibration. The repair agent's exploration process also provides behavioral trajectories for router training, enabling the routing policy to learn "under which violation patterns retrieval is most valuable."

Experimental Design and Evaluation Results

On two building automation datasets, the researchers evaluated the effectiveness of selective retrieval routing. The experiments revealed an interesting phenomenon: researcher access improves repair for some violations but causes degradation for others, and this effect varies across deployments.

This shows that external retrieval is not always beneficial, and blindly enabling it may backfire. Across all six evaluation suites, the router consumed fewer repair time tokens than the always-enable-access approach. Meanwhile, in five of the six suites, the router achieved a higher repair F1 score than the always-disable-access approach.

This result validates the value of selective retrieval: it can achieve repair quality close to or even better than always-enable-access without excessive resource consumption. The deployment specificity of the routing policy also indicates that routers customized for specific scenarios can more precisely balance retrieval benefits and costs.

Repair F1 Score is a core metric for evaluating knowledge graph repair quality, computed by combining the precision and recall of repairs. Specifically, precision measures how many of the repairs proposed by the system are correct and necessary, while recall measures how many of all violations that should be repaired are successfully repaired. The F1 score is the harmonic mean of the two, balancing "repairing accurately" and "repairing comprehensively." In this study, the F1 comparison revealed the advantage of selective retrieval: the router's F1 is higher than the always-disable-access approach, indicating that on-demand retrieval indeed supplements key evidence; and the router's F1 is close to always-enable-access, indicating that it avoids noise from ineffective retrieval. Note that the F1 score is sensitive to the definition of repair granularity—whether an edit is treated as one repair unit or the complete resolution of a violation is treated as one unit—different definitions affect absolute values, but relative comparisons remain informative.

Repair Time Tokens is a proxy metric for the computational overhead of the repair process, usually referring to the total number of tokens consumed by the LLM during reasoning and retrieval. In LLM-based systems, token consumption is directly tied to API call costs and response latency, making it a key dimension for evaluating practicality. This metric includes not only the repair agent's own reasoning tokens but also the tokens consumed by the researcher agent for retrieving and summarizing evidence. The router reduces token consumption across all six evaluation suites, meaning it significantly lowers overall overhead by skipping unnecessary retrieval calls. This result echoes the finding that "external retrieval is not always beneficial": blind retrieval may not only introduce noisy evidence leading to repair degradation but also waste computational resources. Reducing token consumption is especially important for long-term maintenance of large-scale knowledge graphs, because repair costs grow linearly with graph size and number of violations.

Implications for Knowledge Graph Maintenance

This study offers new ideas for automated knowledge graph maintenance. Traditionally, repair systems either rely on manual rules or adopt full retrieval strategies, whereas selective routing introduces a more economical middle path. By letting an LLM agent explore autonomously and combining on-demand retrieval, the system can respond more flexibly to complex violation scenarios.

Repair in multi-violation scenarios is especially challenging because different violations may be coupled. The framework generates multi-violation test cases, providing a more realistic benchmark for training and evaluation. This data generation method itself is also instructive and can be used for other agent training tasks that require supervision signals.

From an engineering perspective, the router's lightweight design makes it easy to integrate into existing repair pipelines. Although deployment-specific training adds some complexity, it yields more precise retrieval decisions; for long-term maintenance of large-scale knowledge graphs, this investment is worthwhile.

Future Directions and Open Questions

Although the framework shows potential, several questions remain to be explored. For example, how generalizable is the router? Can it make reasonable retrieval decisions on unseen violation types? In addition, the researcher agent's retrieval strategy itself can be optimized, such as introducing more efficient query generation or evidence filtering mechanisms.

Another direction worth attention is extending selective retrieval to broader knowledge graph tasks, such as link prediction or entity alignment. These tasks also face insufficient information, and the idea of on-demand retrieval may bring new breakthroughs.

Overall, this study provides a pragmatic and effective solution for knowledge graph repair, balancing repair quality and computational cost, and laying a solid foundation for future research.

One-line Summary

A selective retrieval routing framework dynamically controls access to external evidence to balance quality and cost in knowledge graph repair.

Paragraph Summary

This article introduces a selective retrieval routing framework for knowledge graph repair. To address the problem that schema validation finds violations with multiple compliant repairs but it is difficult to identify the optimal one, the framework introduces an LLM-based repair agent and a dynamic router that invokes an external researcher agent to retrieve evidence on demand. Experiments on building automation datasets show that the effect of external retrieval on repair varies by violation type and deployment; the router consumes fewer tokens than always-enable-access across all six suites and achieves higher repair F1 than always-disable-access in five suites. The study also extends the violation injection framework to generate multi-violation test cases, providing supervision signals for router training. This work offers a pragmatic solution for automated knowledge graph maintenance that balances repair quality and computational cost.