[KongchangAI]
· 2 min read· 1,154 words

Termination Lag in Coding Agents: Solving Early, Finishing Late

Termination Lag in Coding Agents: Solving Early, Finishing Late

Coding agents often "solve early, finish late," idling extensively after the first correct patch or even breaking the solution.

A study proposes the concept of "termination lag," revealing that coding agents continue running many steps after a task is actually completed. Covering five models, two frameworks, and three SWE-bench datasets, per-patch testing found that most successful runs already produce correct solutions at the first edit, yet post-solution steps can account for 35–60% of the median share. These extra steps are mainly used to verify and explore the repository, and in a small fraction of cases they undo correct patches and lead to final errors. This phenomenon indicates that current agents lack effective termination judgment mechanisms, and design must optimize stopping strategies while improving task capabilities to reduce unnecessary computational waste.

Introduction: Why Coding Agents "Solve Early, Finish Late"

A recent study introduced the concept of "termination lag": coding agents often keep running many steps after the task has actually been completed. The study covered five models, two agent frameworks, and three SWE-bench datasets. By recording and testing every distinct repository patch generated during execution, it systematically characterized the number of steps between the first correct patch and the end of the run. This finding reveals a neglected efficiency problem: agents can solve problems early but often fail to stop in time.

Termination Lag research illustration

SWE-bench is a benchmark for evaluating coding agents' ability to solve real software engineering problems. It collects real issues and corresponding fix PRs from open-source Python repositories on GitHub, requiring agents to generate patches given a codebase and problem description. SWE-bench has multiple variants: SWE-bench Full is the complete test set; SWE-bench Verified consists of 500 high-quality, reproducible samples manually curated and is currently the most commonly used evaluation subset; SWE-bench Multilingual extends tasks to multiple programming languages. Researchers typically judge task success by whether the patch passes tests. The "two agent frameworks" mentioned here refer to the scaffolding that drives model-environment interaction, such as ReAct-style loops: the model observes repository state, decides to execute commands or edit files, and continues after receiving feedback. Different frameworks differ in tool sets, context management, and termination conditions, so the same model may exhibit different termination behavior under different frameworks. Understanding how these benchmarks and frameworks are structured helps clarify the experimental conditions under which "termination lag" is measured.

Core Finding: Most Successful Runs Are Correct at the First Edit

The study shows that the vast majority of successful runs already produced correct solutions in their initial source code edits. Specifically, a substantial proportion of successful runs (the original reports a range of 38–82%) achieved correct fixes on the first patch. This result indicates that agents already possess the capability to solve tasks in early steps.

The problem, however, is that agents do not stop there. In successful runs, many steps remain between the first correct patch and the end of the run, and these "post-solution steps" account for a considerable proportion. For Qwen3-Coder, the median proportion is 35–37%; for Laguna-XS.2, it reaches as high as 54–60% on SWE-bench Verified and Multilingual. This means more than half of the work occurs after the problem has already been solved.

Patch in software engineering refers to a set of modifications to source code files, usually represented as a diff recording which lines were added, deleted, or modified. In SWE-bench-style tasks, each file edit operation by the agent may produce a new repository state; accumulating all edits forms a candidate patch. "Recording and testing every distinct repository patch generated during execution" in the study means that after each step of the agent's run, the current repository state is exported as a patch and tests are run to determine whether a correct solution already exists at that moment. The so-called "first correct patch" is the earliest intermediate state that passes all relevant tests. This method can precisely measure the time gap between "solving" and "finishing" rather than only looking at the final submitted patch. Note that intermediate patches may be incomplete or non-submittable, but passing tests is sufficient to show that the logic for a correct fix already exists at that moment.

What Are Post-Solution Steps Doing?

So what exactly are agents doing in "post-solution steps"? The study's analysis points out that these extra steps are mainly spent on two things: verification work and repository exploration. Agents repeatedly check whether their modifications are correct or further inspect other parts of the codebase to ensure nothing is missed. Although this "overly cautious" behavior may be intended to improve reliability, it also brings obvious waste of computational resources.

More seriously, continued activity is not always harmless. In a small fraction of cases (the original reports 1–8%), this continued operation actually caused the agent to undo a previously correct patch, ultimately making the solution incorrect. This "gilding the lily" phenomenon exposes current agents' deficiencies in termination judgment.

Self-verification is the process in which an agent, after generating modifications, proactively checks their correctness, commonly by rereading code, running tests, performing static analysis, or having the model self-critique. In coding agents, verification steps are important for improving final success rates because models may produce plausible but actually incorrect patches. However, verification also has costs: each test run or rereading of large code sections consumes tokens and tool-call turns. When verification becomes repeated confirmation of the same thing, redundant steps arise. On the other hand, exploration refers to the agent searching the repository for relevant files, understanding call relationships, or looking for potential side effects. Moderate exploration helps discover missed modification points, but excessive exploration prolongs runtime. A key cause of termination lag is that agents lack confidence judgments about "good enough" and therefore default to continuing verification and exploration. Understanding the diminishing returns of verification and exploration is a prerequisite for designing better stopping strategies.

Implications for Coding Agent Design

This study has important implications for the design of coding agents. The existence of termination lag means that focusing only on whether a task succeeds is insufficient; attention must also be paid to when the agent stops. An efficient agent should terminate promptly after confirming that the problem is solved, rather than continuing to consume steps unnecessarily.

Current mainstream frameworks may lack effective termination judgment mechanisms, causing agents to continue executing the default loop after reaching a correct solution. Future improvement directions include introducing more precise self-verification mechanisms so that agents can confidently judge that a task is complete, or designing more reasonable stopping strategies to avoid excessive exploration.

Conclusion

The "termination lag" phenomenon reveals a common but neglected efficiency problem in coding agents. Most successful runs are already correct early on, but agents tend to continue running many steps and sometimes even destroy existing correct solutions. This finding reminds researchers and developers that while improving agents' task capabilities, they also need to pay attention to optimizing their termination behavior.

One-line Summary

Coding agents often "solve early, finish late," idling extensively after the first correct patch or even breaking the solution.

Paragraph Summary

A study proposes the concept of "termination lag," revealing that coding agents continue running many steps after a task is actually completed. Covering five models, two frameworks, and three SWE-bench datasets, per-patch testing found that most successful runs already produce correct solutions at the first edit, yet post-solution steps can account for 35–60% of the median share. These extra steps are mainly used to verify and explore the repository, and in a small fraction of cases they undo correct patches and lead to final errors. This phenomenon indicates that current agents lack effective termination judgment mechanisms, and design must optimize stopping strategies while improving task capabilities to reduce unnecessary computational waste.

#termination lag#coding agents#SWE-bench#agent design#self-verification#AI efficiency
Share:

Related articles