Deep Personalization: Continuous Context Observation for Local Proactive Agents

The key to proactive agents is deep personalization: observe, remember, adapt, and judge on-device when to intervene.
The article discusses how proactive agents move from "getting things done" to "picking the right moment." The core concept is deep personalization—an observe–remember–adapt loop: continuously observe desktop activity, selectively retain long-term context, and learn when to intervene from implicit feedback. The research implements this in the desktop agent Coco via an independent Observer Agent running on-device. On the OSWorld-Persona benchmark, macro F1 rises from 0.12–0.19 to 0.35–0.48; in a one-week user study, Coco's macro-average F1 increases from 0.199 to 0.515, showing that synthetic-benchmark gains transfer to real scenarios.
AI agents are rapidly becoming more capable, but users still have to decide for themselves when to ask for help and how to ask for it, so many opportunities for support are lost. Proactive agents are seen as key to closing this gap, but true proactivity does not mean frequent interruptions—it requires understanding what the user is doing right now and making personalized judgments about when to intervene, stepping in when help is more likely to be accepted.

From "Getting Things Done" to "Picking the Right Moment": The Core Challenge of Proactivity
The bottleneck for today's agents has shifted from task execution to interaction decisions. Users must proactively identify needs and formulate instructions, and this step itself creates a barrier to use. The vision of proactive agents is for systems to perceive user activity and offer help proactively, but the difficulty is that intervening too early interrupts flow, while intervening too late loses value. The paper summarizes this capability gap as "proactivity requires understanding the user's real-time activity and making personalized judgments about intervention timing."
This also explains why simple rule-based triggers (such as fixed-time reminders or keyword triggers) struggle to work. Effective proactivity depends on understanding an individual's work patterns rather than generic heuristics.
Deep Personalization: An Observe–Remember–Adapt Loop
The research proposes "Deep Personalization" and defines it as a closed-loop process: the agent continuously observes user activity, selectively retains long-term context, and adjusts intervention decisions based on implicit feedback. The three components interlock—
- Continuous observation: it does not rely on explicit user input but obtains signals from desktop activity;
- Selective memory: it does not record everything but maintains a compact long-term user-modeling context;
- Continuous adaptation: it learns "when to help" from implicit signals such as engagement, ignoring, explicit requests, and recurring work patterns.
According to the authors, the value of this design lies in turning personalization from a "one-time configuration" into a continuously evolving process, so that the agent adjusts as the user's daily work changes.
Coco and the Observer Agent: Architecture and Privacy Design
The research implements deep personalization in the proactive desktop agent Coco, with an independent Observer Agent at its core. It monitors desktop activity, maintains a compact long-term user-modeling context, and learns when to offer help based on implicit signals. These signals include user engagement with suggestions, dismissal/ignoring, explicit requests, and recurring work patterns.
One detail on the privacy path: the research team developed efficient on-device policies that allow the Observer Agent to run on a local AI computing platform without sending raw activity data off the device. For a system that continuously monitors the desktop, this is one of the prerequisites for user acceptance.
On-device AI computing platform refers to running models directly on the user's local device (such as a PC's NPU or a local inference framework) rather than uploading data to the cloud for processing. For an Observer Agent that continuously monitors the desktop, on-device operation means raw activity data does not leave the device and only a compact modeling context is retained, thereby reducing privacy risk at the architectural level. The cost of this path is limited local compute and memory, which requires efficient model compression and inference strategies; the paper's emphasis on "efficient on-device policies" shows precisely that the Observer Agent's design must balance effectiveness against local resources.
Evaluation: Synthetic Benchmark and Real User Study
Large-scale evaluation of proactive agents faces practical obstacles—collecting real user activity data involves sensitive privacy. The research therefore proposes a data synthesis pipeline that converts existing computer-use trajectories into the OSWorld-Persona benchmark, which carries persona-conditioned intervention labels and is used to measure whether the agent intervenes "when it should."
The results show consistency at two levels:
- Benchmark testing: on OSWorld-Persona, deep personalization raises the macro F1 of three model backbones from 0.12–0.19 under direct prompting to 0.35–0.48 after adaptation, a relative improvement of 2.4–3.0 times;
- Real-world setting: in a one-week user study, Coco's macro-average F1 increased from 0.199 to 0.515.
The two sets of data point in the same direction, indicating that improvements on the synthetic benchmark can transfer to real usage situations rather than merely overfitting the benchmark.
Macro F1 is the core metric for evaluating proactive intervention decisions. In the binary classification problem of "when to intervene," the model must judge at each moment whether help should be triggered. Macro F1 first computes precision and recall separately for the two classes "should intervene" and "should not intervene," obtains the F1 score for each class, and then takes the arithmetic mean of the two. Compared with ordinary accuracy, macro F1 is more sensitive to class imbalance—a proactive agent should stay silent most of the time, and if the model tends to "never intervene," accuracy may be high but macro F1 will be low. Therefore, this metric genuinely reflects the quality of the agent's judgments on scarce intervention opportunities, and it is also why the numbers 0.12 to 0.48 and 0.199 to 0.515 in the original text are convincing.
Implicit feedback refers to data in which the user does not explicitly state a preference but whose behavior itself carries preference signals. In Coco's setting, accepting or ignoring suggestions, closing pop-ups, and repeating the same operation flow all constitute implicit feedback. Compared with explicit ratings or instructions, implicit feedback is noisier and sparser, but it is cheap to obtain and does not interrupt the user. Deep personalization relies on such signals to continuously adjust intervention strategies, so its long-term stability has become one of the open questions listed in the text.
Significance and Open Questions
The insight of this work is that the competitiveness of proactive agents may no longer depend mainly on model capability itself, but on the precision of context observation and the degree of personalization in intervention decisions. The on-device approach responds to the privacy concerns brought by continuous monitoring and provides an implementable path for local proactive agents.
Questions that still need to be observed include: the stability of implicit feedback signals over long-term use, the expressive capacity of compact context in complex cross-application workflows, and the modeling boundaries in multi-user shared-device scenarios. The authors limit their conclusion to "on-device deep personalization can make proactive agents more timely and more attuned to each user's daily work," and this formulation itself hints at the scope of applicability—personalization effectiveness depends heavily on the regularity of an individual's work patterns.
One-line Summary
The key to proactive agents is deep personalization: observe, remember, adapt, and judge on-device when to intervene.
Paragraph Summary
The article discusses how proactive agents move from "getting things done" to "picking the right moment." The core concept is deep personalization—an observe–remember–adapt loop: continuously observe the user's desktop activity, selectively retain long-term context, and learn when to intervene from implicit feedback such as engagement, ignoring, and explicit requests. The research implements this design in the desktop agent Coco, with an independent Observer Agent monitoring activity and running on-device to avoid raw data leaving the device. For evaluation, the team used a synthesis pipeline to build the OSWorld-Persona benchmark; the macro F1 of three model backbones rose from 0.12–0.19 to 0.35–0.48, and in a one-week real user study Coco's macro-average F1 increased from 0.199 to 0.515, indicating that improvements on the synthetic benchmark can transfer to real scenarios.
Related articles

Farewell to Independent Test Sets: How Evaluability Theory Is Reshaping LLM Evaluation
The arXiv paper *Toward a Theory of Evaluability* argues that the independent test set assumption fails in the foundation model era, proposes a formal definition of evaluability without independent test sets, and explores data contamination challenges across safety, calibration, fairness, and other multi-axis evaluations.

Private Evolution: When Differentially Private Synthetic Data Meets Learning-Augmented Clustering
New research reframes differentially private synthetic data algorithm Private Evolution as learning-augmented clustering, showing tighter bounds when generative models capture distributional structure.

Beyond Reddit: Where to Find Serious AI Discussion Communities
Reddit's AI discussion quality is uneven, and Discord official servers cling to their own narratives. This article examines the real dilemmas of serious ML communities and alternative paths, including academic Slacks, niche subreddits, and open-source project discussion areas, and analyzes the structural tension between community scale and discussion quality.