Anthropic Team Breaks Down Context Engineering: From Claude MD to Filesystem Memory

At AI DevCon, Anthropic team member Lamis systematically reviewed the evolution of context engineering over the past year. Rather than piling on concepts, the talk followed a clear practical thread: from the initially simple and effective Claude MD file, to letting agents manage their own memory, to the concurrency, versioning, and permission challenges in production, ultimately pointing toward an emerging "filesystem-as-memory" architecture.

A Simple Starting Point: Why Claude MD Works and Where It Fails

Anthropic has an internally repeated maxim: "do simple but effective things." The first stop in context engineering is exactly Claude MD—a Markdown file injected into the model's context at the start of a session, containing environment hints, organizational conventions, personal preferences, and more. According to Lamis, this approach is "surprisingly effective" because it feeds the information that truly matters for the agent to the model right at the start of the session.

But problems follow. When this file becomes very important, it also starts to bloat: cramming large amounts of content into context all at once eats into the space the model has for handling actual tasks. The team therefore went back to the drawing board—is there a way to preserve the advantages of the MD file ("human-readable, writable, editable") without having to load everything at once?

The answer is filesystem memory: letting the agent decide for itself when to read, when to write, and what to write. This shift is already reflected in Claude Code, where the agent judges within a session which files are worth reading and which experiences are worth recording—autonomy works quite well here.

Context Window is the key concept for understanding why Claude MD "fails." It refers to the upper limit on the total number of tokens the model can process simultaneously during a single inference, including the system prompt, conversation history, tool return results, and injected memory content. Claude MD is injected in full at the start of a session, meaning it occupies a fixed budget shared with the task itself—the larger the file, the less space remains for reasoning, code reading, and tool output. This is why Anthropic emphasizes "loading on demand": the context window is not an infinite blackboard but a workbench that must be carefully allocated. Filesystem memory works precisely because it shifts from "resident in memory" to "paged in on demand," letting the limited window serve the current task first.

Skills and Progressive Disclosure

The second solution to continuously growing context is "Skills." Lamis used a vivid analogy: like a row of dictionaries on a shelf—when someone speaks to me in a foreign language, I can first glance at the spines to judge which dictionary is relevant, then take it down and flip through it—without having to memorize an entire dictionary in advance.

This is the core of progressive disclosure: the agent first sees only the metadata of a skill, and loads the full content only when the task actually requires it. This keeps context lean while preserving deep capabilities. However, Lamis also pointed out a real constraint: even when using an agent to write skills, humans still need to be heavily involved in judging "what counts as a real skill."

Four Hard Problems in Production

A good prototype does not equal production readiness. When multiple agents concurrently read and write the same memory, sessions span long periods, and memory content grows complex, a series of problems emerge. Lamis listed several typical risks: how to coordinate multiple agents writing to the same file simultaneously; if the global context written by one agent is wrong, it pollutes every agent reading it; how to track changes when humans and agents collaborate; and most dangerously—memory may be maliciously induced to write incorrect content.

The team introduced a set of engineering principles for this:

Versioning, Hashing, and Concurrency Control

Before each memory write, take a hash; take another hash at write time; if the two are inconsistent, reject the write and require re-reading and re-drafting. This optimistic concurrency control approach is borrowed directly from traditional software engineering.

Optimistic Concurrency Control is a concurrency strategy that assumes conflicts are uncommon and detects them only at write time, in contrast to pessimistic locking that acquires locks before writing. The article's "take a hash before writing, take another at write time, reject if inconsistent" is its typical implementation: record a version fingerprint on read, compare on commit, and if modified by someone else in the meantime, fail this write and require re-reading and rewriting. It suits agent scenarios because lock overhead and deadlock risk are avoided, and agents can cheaply re-read and re-draft. The cost is higher retry costs when conflicts are frequent, so in practice it is often combined with permission layering to reduce concurrent writes to frequently shared memory.

Layered Write Permissions

Not all memory should be freely rewritten by agents. Organization-level core context should be subject to stricter constraints, while an agent's personal working notes can be given more autonomy.

Auditable and Integrable

The memory system needs to record "which session, which record, which person" triggered this change, and be designed API-first for easy cross-system integration. Lamis emphasized that if enough is invested in the durability of memory, the payoff shows up in cross-session, cross-system coherence.

In-Session Memory vs. Out-of-Session "Dreaming"

Lamis divided memory work into two complementary paths. The first is in-session memory: the agent reads and writes memory within a single session to help itself perform better on the next similar task. But this path has two natural limitations—the agent must both complete the task and maintain memory, which is a very hard optimization problem; and a single agent can only see the context within its own session, unable to detect repeated errors across sessions.

The second path is out-of-session "Dreaming": an independent agent batch-reviews historical session records, identifies pattern-level problems, and directly modifies the memory store. Lamis used a school analogy—students submit homework, teachers grade it, and there are also superintendents who examine overall teaching quality across classes. Dreaming plays exactly the "superintendent" role: it can see common problems across all students, such as an entire class making the same mistake on a particular math concept, and then suggest adding the correct solution to the course materials.

You may not have noticed that Dreaming reviews not only agent-user conversations but also tool call records and other metadata, which lets it pinpoint finer-grained causes of failure.

Batch Reflection is the background concept for understanding Dreaming's positioning. Unlike online learning, Dreaming does not update memory during task execution; instead, it replays historical sessions offline and periodically, performs pattern induction, and then writes back to the memory store. This separation brings two benefits: first, it decouples the two goals of "completing tasks" and "maintaining memory," preventing a single agent from optimizing both within the same context; second, it gains a cross-session global view, capable of detecting systemic biases that a single session cannot perceive. The cost is delayed feedback, and batch modifications require versioning and auditing mechanisms as a safety net—otherwise a single erroneous induction could pollute the entire store.

Implementation Path and Applicability Boundaries

For teams looking to implement this, Lamis laid out a progressive route: first use simple means like Claude MD and Skills; when the number of agents increases, run cycles lengthen, and collaboration grows complex, then introduce production-grade mechanisms such as versioning, hashing, and permission layering; finally consider adding out-of-session processes like Dreaming.

He also specifically clarified a common misconception: this approach is not limited to programming tasks. Lamis himself uses the memory system to record presentation-writing preferences and slide styles, and these preferences evolve gradually over time.

During the Q&A, an audience member asked whether this amounts to "reinventing the database from first principles." Lamis candidly admitted that the team is indeed returning to practices validated in software engineering—version control, concurrency control, deterministic execution—but with the premise that these mechanisms must be efficiently usable by autonomous agents. In other words, it's not reinventing the wheel but finding the dividing line between "letting agents act autonomously" and "baking necessary constraints into the framework."

Final Thoughts

The value of this talk lies in presenting a real engineering evolution curve: first let agents freely write memory in Markdown, observe which primitives are truly effective, then solidify the validated parts into deterministic mechanisms. Context engineering is not a static collection of tricks but a field that continuously restructures as model capabilities improve and deployment scale expands. For teams building agent applications, this path from simple to complex may be more worth referencing than any single technique.

One-line Summary

Anthropic breaks down context engineering: the evolution path from Claude MD to filesystem memory and Dreaming.

Paragraph Summary

Anthropic team member Lamis at AI DevCon systematically reviewed a year of practical evolution in context engineering. The starting point was the Claude MD file injected at session start—simple and effective but bloating as it grows in importance, eating into the model's task-processing space, which led to filesystem memory that agents read and write autonomously. Skills further relieve context pressure through progressive disclosure: see metadata first, load full content on demand. Production exposes risks like concurrent write conflicts, global context pollution, change tracking, and malicious writes, prompting the team to introduce hash-based optimistic concurrency control, layered write permissions, and auditable API-first design. Memory work splits into two complementary paths—in-session memory and out-of-session Dreaming, the latter having an independent agent batch-review historical sessions and metadata to identify cross-session pattern-level problems. Implementation should follow a progressive route from simple means to production-grade mechanisms.