[KongchangAI]
· 2 min read· 1,316 words

CarryOn: Teaching Long-Horizon LLM Agents to Retain Critical Context

CarryOn: Teaching Long-Horizon LLM Agents to Retain Critical Context

CarryOn replaces context compression with execution state plus source-linked evidence, boosting long-horizon agent success and cutting costs.

The article introduces CarryOn, a context management method for long-horizon LLM agents. Existing summarization, pruning, and history selection assume shorter context is better but may lose precise details or key evidence. CarryOn splits context into a compact execution state and fine-grained source-linked evidence, using a learned selector and adaptive retention to keep relevant evidence available without extra LLM calls. With two complementary metrics (S_α and NCS) balancing success and cost, it achieves 75.7% macro-average success on three benchmarks—6.7 points above the strongest baseline.

Title

CarryOn: Teaching Long-Horizon LLM Agents to Retain Critical Context

Meta Description

CarryOn introduces a lightweight context manager that replaces simple summary-based pruning with compact execution state plus source-linked evidence, achieving 75.7% task success on benchmarks like AppWorld—6.7 percentage points above the strongest baseline—while balancing success and cost.

Article Content

When long-horizon LLM agents coordinate tools and applications to complete complex multi-step tasks, the ballooning interaction history makes context management a bottleneck. Existing methods compress context through summarization, pruning, or history selection, but shorter context does not necessarily lead to better decisions—summaries may omit precise details needed for subsequent actions, while retaining all interactions carries a large amount of irrelevant information. To address this tension, researchers propose CarryOn, a lightweight context manager that pairs a compact execution state with fine-grained, source-linked evidence, and maintains the availability of relevant evidence through a learned selector and adaptive retention policy—without requiring additional LLM calls to manage context.

Screenshot of the CarryOn paper title

Limitations of Existing Context Compression Methods

A long-horizon agent's interaction history grows continuously as the task progresses. Directly concatenating the full history not only consumes a large number of tokens but may also interfere with decisions due to irrelevant information. Common countermeasures include summarization, pruning, and history selection. Summarization compresses past interactions into brief descriptions but easily loses precise parameters or intermediate results that later steps depend on; pruning discards seemingly unimportant fragments and may accidentally delete key evidence; history selection relies on similarity or heuristic rules and struggles to guarantee that the selected content truly supports the current decision. These methods share an implicit assumption: the shorter the context, the better. The authors of CarryOn point out that shorter context does not necessarily support better decisions—what matters is what to retain and in what form.

Long-horizon LLM agents refer to large language model agents that need to continuously execute dozens or even hundreds of steps, calling external tools or application interfaces at each step. Typical scenarios include cross-application scheduling, multi-table data organization, and customer service ticket routing. Unlike single-turn question answering, each tool result returned to such an agent is appended to the interaction history, causing context length to grow approximately linearly with the number of steps. Although mainstream LLMs have context windows on the order of hundreds of thousands of tokens, the effective usable length is often limited by attention dilution and loss of intermediate information (the so-called "lost in the middle" phenomenon), and token-based billing significantly raises the inference cost of long contexts. For this reason, maintaining information truly useful for decision-making within a limited budget has become one of the core engineering problems for long-horizon agents.

CarryOn's Core Design: Execution State Plus Source-Linked Evidence

CarryOn splits context into two complementary parts: a compact execution state and fine-grained evidence with source links. The execution state summarizes the task's current progress—such as completed steps, pending items, and key constraints—and is small in size and clearly structured; the evidence retains specific fragments from the original interactions, accompanied by links pointing to their sources, so the agent can trace back to precise details when needed. This separation allows the agent to quickly grasp the big picture while drawing on reliable grounds when executing specific actions.

To make evidence retention more precise, CarryOn introduces a learned selector that determines which evidence is relevant to the current decision; it also works with an adaptive retention policy that dynamically adjusts the retention granularity according to the task phase. The entire context management process does not rely on additional LLM calls, so it does not add inference overhead for managing context itself.

Source-linked evidence borrows the idea of "citation/traceability" from retrieval-augmented generation: each evidence fragment not only stores text content but also carries an identifier pointing to the original interaction record (such as a dialogue turn number, tool call ID, or message offset), enabling the agent to locate, expand, or verify the original content on demand in later steps. Compared with simply truncating history fragments, this mechanism decouples "what to retain" from "how to retrieve it"—only a small amount of highly relevant evidence is placed in context, while the rest of the details are accessed on demand through links. The cost is the introduction of retrieval overhead and the problem of choosing evidence granularity, so it needs to be paired with a selector to determine which evidence is worth residing in context and which only needs a retained link.

Evaluation Metrics: Balancing Task Success and Resource Consumption

The researchers propose two complementary evaluation metrics. S_α prioritizes task success while accounting for resource usage, avoiding pursuing success at any cost; Normalized Cost per Success (NCS) measures the resources consumed per successful task. Combined, the two metrics reflect the practical value of a context management strategy more comprehensively, rather than looking only at task completion rate.

Normalized Cost per Success (NCS) is usually defined as: the total resources consumed on a given task set (such as token count, number of tool calls, or actual cost) divided by the number of successfully completed tasks, then normalized against some reference baseline so that different methods can be compared directly. Unlike simply reporting average token consumption, NCS includes the cost of failed tasks in the success count outside the denominator, thereby penalizing strategies that "trade resource stacking for success rate"; when used together with success rate, it can characterize a position on the success rate–cost Pareto frontier. S_α, meanwhile, uses a weight α to strike a weighted balance between success rate and resources; different α values reflect different deployment preferences, and the two are complementary and can avoid the misleading effects of a single metric.

Experimental Results: Superior in Both Task Success Rate and Cost

On three benchmarks—AppWorld, OfficeBench, and Tau2 Telecom—tested with three LLM Actors, CarryOn achieves a 75.7% macro-average task success rate, 6.7 percentage points higher than the strongest baseline, and obtains the best average scores on both metrics. This result shows that a refined evidence retention strategy can improve the reliability of long-horizon tasks without sacrificing efficiency.

Implications for Long-Horizon Agent Development

CarryOn's approach offers direct reference value for building long-horizon agents. The goal of context management should not stop at compressing length but should focus on the information structure required for decision-making: use a compact state to maintain task coherence, use source-linked evidence to ensure traceable precision, and use a learned selector to replace fixed heuristic rules. For application scenarios that need to coordinate multiple tools and execute dozens of steps, this design is expected to reduce repeated operations or erroneous decisions caused by context loss.

Of course, CarryOn has so far been validated only on three benchmarks and a limited set of LLM Actors. The generalization ability of the learned selector to entirely new task distributions and the retrieval efficiency of source-linked evidence in extremely long interactions still require further observation. But the direction it reveals—shifting context management from "compression" to "what to retain"—deserves continued attention from long-horizon agent researchers and engineering teams.

One-line Summary

CarryOn replaces context compression with execution state plus source-linked evidence, improving long-horizon agent success rates while reducing costs.

Paragraph Summary

This article introduces CarryOn, a context management method for long-horizon LLM agents. Existing summarization, pruning, and history selection methods assume shorter context is better, yet may lose precise details needed for later actions or accidentally delete key evidence. CarryOn splits context into a compact execution state and fine-grained, source-linked evidence, and uses a learned selector and adaptive retention policy to dynamically keep relevant evidence available without extra LLM calls to manage context. The researchers also propose two complementary metrics—S_α and Normalized Cost per Success (NCS)—that balance task success and resource consumption. On three benchmarks (AppWorld, OfficeBench, and Tau2 Telecom) with three LLM Actors, CarryOn achieves a 75.7% macro-average success rate, 6.7 percentage points above the strongest baseline, with the best average scores on both metrics.

#CarryOn#long-horizon LLM agents#context management#source-linked evidence#AppWorld#task success rate
Share:

Related articles