[KongchangAI]
· 1 min read· 988 words

No Fine-Tuning Needed: SkillEarth Teaches General Agents Earth Observation

No Fine-Tuning Needed: SkillEarth Teaches General Agents Earth Observation

SkillEarth externalizes remote sensing tool-invocation knowledge into retrievable, evolvable skill files for parameter-update-free continuous adaptation.

To address general agents' difficulty in continuously adapting to remote sensing, SkillEarth decouples tool-invocation knowledge from model parameters and automatically extracts it into reusable, retrievable, evolvable skill files. Unlike methods relying on holistic outcome-based reflection, it uses dual-judge feedback and a grounded observation routing graph for component-level skill correction and supports cross-model reuse. On OpenEarth-Bench and Earth-Bench, it outperforms existing remote sensing agents and skill extraction methods across tool selection, parameter correctness, execution order consistency, and task completion, and improves performance across three frontier and four open-source LLMs, offering a parameter-update-free paradigm for continuous adaptation of domain agents.

General-purpose agents are entering the remote sensing (RS) domain, but the way their knowledge gets updated is rather clunky: either they rely on predefined expert rules, or they retrain models when new tools or tasks appear. A study titled 《SkillEarth: Teaching General-Purpose Agents to Act on Earth via Skill Evolution》 proposes another path—automatically extracting tool-invocation knowledge from demonstration trajectories into reusable, retrievable, evolvable "skill files," enabling agents to continuously adapt to dynamic Earth environments without updating parameters.

SkillEarth Research Overview

The Continuous Adaptation Dilemma of Remote Sensing Agents

Remote sensing tasks typically involve multi-step tool-invocation chains—for example, image preprocessing, index computation, and post-classification processing—where each step depends on correct tool selection and parameter settings. Existing remote sensing agents either hard-code expert knowledge into workflows or use parameter updates to make models memorize execution patterns. The former fails immediately when new tools appear, while the latter requires retraining for each new task—costly and hard to transfer.

The core problem SkillEarth aims to solve is precisely this "continuous adaptability": when tool sets, task types, or execution patterns change, can a general-purpose agent acquire correct remote sensing tool-invocation knowledge without expert intervention or parameter updates?

Demonstration trajectory refers to the complete operational record left by experts or strong models when completing a task, including the inputs, selected tools, parameter settings, intermediate results, and final outputs at each step. In agent research, it is often used as a supervision signal for imitation learning or knowledge distillation. Unlike reinforcement learning, which relies on reward signals, demonstration trajectories directly show "what action to take under what observation," making them better suited for extracting procedural knowledge. SkillEarth uses such trajectories as raw material for skill extraction, inducing reusable tool-invocation patterns from them rather than making models rote-memorize individual trajectories.

Externalizing Skills as Evolvable Files

SkillEarth's approach is to decouple tool-invocation knowledge from model parameters and turn it into independent skill files. These files are automatically extracted from demonstration trajectories, can be retrieved and reused, and can continue to evolve during subsequent execution.

The key difference from existing automatic skill extraction methods lies in the source of the evolution signal. Most existing methods rely on outcome-based reflection, making holistic modifications to entire skills; SkillEarth instead anchors evolution in real tool-chain execution, achieving component-level fine-grained correction through dual-judge feedback and a grounded observation routing graph. In other words, skills are not broadly rewritten based on whether outcomes are good or bad, but are precisely repaired according to which step in actual execution went wrong.

A direct benefit of this design is skill transferability. Research shows that the extracted skills can be reused across models, transferring to other models without retraining.

Outcome-based reflection is a common method for agent self-improvement: after execution completes, the model reviews the entire process and rewrites strategies or skills based on task success or final score. Its limitation is that the feedback signal is too sparse and coarse-grained—it only indicates whether the overall performance was good, but makes it hard to pinpoint which step selected the wrong tool or which parameter was set incorrectly. In contrast, SkillEarth's dual-judge feedback introduces two independent judging perspectives and anchors feedback to specific tool-chain stages, enabling component-level correction and avoiding the over-modification problem of "one error leads to rewriting the whole document."

A grounded observation routing graph is a routing mechanism that organizes actual observations during execution (such as intermediate image features and tool return states) into a graph structure. Nodes in the graph correspond to observable states or skill components, and edges represent the next action to trigger under specific observation conditions. Unlike retrieving skills solely by semantic similarity, this graph makes skill selection and correction "evidence-based": when the actual observation at a stage deviates from expectations, one can trace along the graph to locate the skill component that needs modification, rather than broadly replacing the entire skill file. This is also an important infrastructure for SkillEarth's fine-grained evolution.

Experimental Performance: Comprehensive Advantage in Tool Selection and Execution Order

The paper evaluates on two benchmarks, OpenEarth-Bench and Earth-Bench, comparing against existing remote sensing agents and skill extraction methods. Results show that SkillEarth performs better on four dimensions: tool selection, parameter correctness, execution order consistency, and task completion.

A detail on generalization: the method improves overall remote sensing tool-invocation performance across three frontier LLMs and four open-source LLMs. This means skill files are not bound to a specific model but exist as relatively independent knowledge artifacts, consistent with the design goal of "no parameter updates."

From a methodological perspective, SkillEarth's value extends beyond remote sensing. It provides a general paradigm: externalizing an agent's procedural knowledge into retrievable, evolvable assets, decoupling model capability from knowledge updates. For domain agents that need to frequently integrate new tools and workflows, this "skill evolution" route may be more sustainable than repeated fine-tuning.

Of course, the method still relies on high-quality demonstration trajectories as the starting point for skill extraction, and the evolution boundaries of skill files on extremely novel tasks remain to be further validated. But given the current results, it offers a noteworthy direction for solving the continuous adaptation problem of general-purpose agents in specialized domains.

One-line Summary

SkillEarth externalizes remote sensing tool-invocation knowledge into retrievable, evolvable skill files, enabling continuous adaptation without parameter updates.

Paragraph Summary

SkillEarth addresses the difficulty general-purpose agents face in continuously adapting to the remote sensing domain by decoupling tool-invocation knowledge from model parameters and automatically extracting it into reusable, retrievable, evolvable skill files. Unlike existing methods that rely on holistic outcome-based reflection, it achieves component-level skill correction through dual-judge feedback and a grounded observation routing graph, and supports cross-model reuse. On OpenEarth-Bench and Earth-Bench, the method outperforms existing remote sensing agents and skill extraction methods on all four dimensions—tool selection, parameter correctness, execution order consistency, and task completion—and improves performance across three frontier LLMs and four open-source LLMs, offering a new parameter-update-free paradigm for continuous adaptation of domain agents.

#SkillEarth#remote sensing agents#skill evolution#tool invocation#demonstration trajectories#OpenEarth-Bench#Earth-Bench#general-purpose agents#continuous adaptation#parameter-free
Share:

Related articles