Stanford's Paper2Agent Published in Nature: Turning Paper Methods into Callable Research Tools

To apply a method from a paper to your own data, you usually have to find the code, set up the environment, and figure out the input formats and execution order. If any step doesn't line up, the method is hard to actually get running. Paper2Agent, proposed by a research team at Stanford University, attempts to hand this entire set of preparatory work over to an AI Agent. The related work, titled Reimagining Research Papers as Interactive and Reliable AI Agents, was published in Nature; the journal page shows a submission date of October 13, 2025, and acceptance in August 2026.

Its core idea is to organize the paper, supplementary materials, code, data, and analysis pipelines into queryable resources and runnable tools, then connect them to a chat Agent. Researchers can ask about methodological details in natural language or have the Agent execute analyses step by step. On a test of 300 tutorial questions, Paper2Agent achieved 91.2% accuracy, higher than the 80.3% achieved when a chat Agent directly accesses the corresponding code repository.

Why public code isn't enough

A public code repository does not mean the method is reusable. Researchers still have to find the right examples, install dependencies, prepare inputs, and determine the execution order among scripts. Every paper has different interfaces and usage patterns, and these hidden costs often exceed reading the paper itself.

If you simply have a large model read the paper, it can explain what the paper says, but it struggles to actually run the method on new data for the researcher. Paper2Agent's difference is that it incorporates code and data into the system, letting the chat model call actual tools: the model understands the task and arranges the steps, computation is executed by the paper's programs, and the results are handed back to the model for organization.

Three types of resources make up the Agent's "server"

Paper2Agent's server holds three types of things. The first is tools that can execute the method; the second is static resources such as the paper, code, and data; the third is prompt templates. The prompt templates tell the Agent what to check first and which tool to call next when performing a complex analysis.

At the protocol level, the system uses the Model Context Protocol (MCP) to specify how the chat Agent discovers resources, understands tool descriptions, and initiates calls. This gives methods from different papers a unified calling convention, so the Agent doesn't need a separately adapted interface for each paper.

The pipeline from paper to tool

Turning a paper into a tool involves several steps: the system first locates the corresponding public repository from the paper, checks dependencies and prepares the runtime environment, then finds the tutorials and examples and actually runs them; subsequently, an extraction agent uses these runnable examples to wrap the core steps into tools with well-defined inputs and outputs; the paper and supplementary materials are organized into resources for the chat Agent to consult.

Model-generated tools may contain incorrect code, so every tool must be automatically validated against the original repository's examples: checking whether the target files are generated, whether the numerical differences from reference results are below a set threshold, and whether charts correspond to the original figures. Failed validation triggers continued diagnosis and repair, and tools that repeatedly fail do not enter the final server. Tool descriptions also indicate the location of the original code, making it easy for researchers to trace the implementation's source.

Model Context Protocol (MCP) is a standardized interface specification proposed and open-sourced by Anthropic in late 2024, designed to let large-model applications discover and invoke external tools, data sources, and prompt templates in a unified way. It uses a client–server architecture: MCP servers expose three types of primitives—tools (executable functions), resources (readable static materials), and prompts (preset invocation patterns)—and clients (such as chat Agents) communicate with them via JSON-RPC. MCP's design goal is precisely to solve the problem of "having to write a separate adapter layer for every data source or tool," which is why it is often compared to the USB-C port of the AI application world. In Paper2Agent, a paper's method is encapsulated as an MCP server, so different papers share the same calling convention.

Automatically validating whether a tool is correct presupposes that the repository comes with reproducible tutorials or examples. Paper2Agent actually runs the original repository's tutorial scripts to obtain reference outputs, then compares the Agent-generated tool outputs against them. Dimensions of comparison include whether target files are generated, whether numerical differences are within tolerance, and whether charts correspond to the original figures. This "using the original code's output as the baseline" approach follows a regression-testing philosophy: it doesn't judge whether an implementation is elegant, only whether its behavior is equivalent. The limitation is that if the original tutorial itself depends on specific hardware, private data, or unfixed random seeds, reference outputs may be unavailable, making such papers difficult to bring into the tooling pipeline.

Concrete cases: AlphaGenome and Scanpy

The team turned AlphaGenome, a model for predicting genomic regulatory effects, into an Agent that generated 22 tools, all of which passed automatic validation, in about 45 minutes at a cost of roughly $14. Researchers can submit a genetic variant and have the tools compute its predicted impact on various biological processes such as gene expression, then generate charts to help inspect the results.

Single-cell analysis often involves running many steps in sequence. The paper uses Scanpy as a demonstration: prompt templates arrange steps such as quality control, normalization, feature selection, dimensionality reduction, and clustering into a reusable pipeline; after the researcher provides a data file, the Agent calls the tools in order and summarizes the results. The pipeline order is likewise extracted from the paper and code, so researchers don't have to rewrite long operating instructions by hand.

Consistency with manual results

The Scanpy Agent was run on four public single-cell datasets, and the post-QC cell and gene counts as well as the marker genes for each cell cluster matched the results researchers obtained using the same set of parameters. The team also used seven kinds of data to check whether it can adjust parameters according to data characteristics. AlphaGenome tests included both tutorial questions and new questions, showing that the tools are not merely copying fixed outputs.

Scanpy is a widely used Python toolkit in the field of single-cell RNA sequencing analysis, covering standard steps such as quality control, normalization, highly variable gene selection, dimensionality reduction (PCA/UMAP), clustering, and marker gene identification. Single-cell analysis is characterized by strongly dependent steps and interacting parameters—for example, QC thresholds change downstream clustering results—so the "pipeline order" is itself part of the method knowledge. The paper uses Scanpy as a demonstration precisely because such multi-step pipelines are hard to reproduce with single-turn Q&A: an Agent must solidify the order, parameters, and intermediate checkpoints to obtain comparable results on new data.

Data from large-scale validation

Among 100 computational biology papers, the system successfully turned 74 into usable Agents, proposing 599 tools in total, of which 593 passed validation. On 42 tasks from 10 non-biology computational papers, accuracy reached 98.1%.

When runnable code is unavailable, Paper2Agent is not idle: it can still organize the paper, supplementary materials, and data descriptions into structured resources for the chat Agent to query. For 26 data- and discovery-oriented papers, the resource layer answered 100 comprehensive questions with 89% accuracy, higher than the 82% web-browsing baseline. But it cannot conjure missing experimental code out of thin air—resource Q&A and method execution are two different capabilities.

Model capability itself also affects results. In the evaluation, the baseline with direct repository access scored 80.3% using Claude Sonnet 4 and 86.3% with the newer Sonnet 4.6; Paper2Agent with Sonnet 4 reached 91.2%. Stronger models help, but tool interfaces, execution order, and result checking also determine task performance. This set of tests only covers tasks defined by the papers and does not represent all research questions.

It is necessary to distinguish two types of evaluation: tutorial questions come from examples included with the paper or repository, have clear reference answers, and mainly test execution fidelity; new questions are questions posed around the same method that do not appear directly in the tutorials, used to test whether tools can be flexibly combined. Accuracy figures should be read with this distinction in mind—the comparison between 91.2% and 80.3% reflects the advantage of "having tool interfaces" over "reading the repository directly," not the method's universal capability across all research tasks. The paper itself also notes that the tests only cover tasks defined by the papers.

Its place in the landscape of research Agents

Placing Paper2Agent in the current lineage of research Agents makes things clearer: some work summarizes research strategies from existing papers to help researchers form directions; some start from research goals to generate and screen testable hypotheses; some attempt to automate the entire machine-learning research pipeline. Paper2Agent does something different—it turns already-published methods into tools that can be called and verified, and higher-level research Agents can use the paper methods and data resources it provides to conduct analyses.

The paper also presents a case of collaboration among Agents for multiple papers: by comparing gene perturbation results from independent experimental datasets side by side, it prioritizes a potentially relevant causal gene. This conclusion comes from computational analysis and still requires subsequent biological research to validate. The more practical change is that when searching for methods, fetching data, and running analyses across papers, researchers can spend less time switching back and forth between repositories and formats.

Judgments that can't be handed to the Agent

Whether the system can produce results close to the tutorials, and whether the scientific explanations it gives hold up, are two different things. Open-ended analyses often have multiple reasonable answers, and a single reference answer can only test whether execution is faithful; it cannot represent the full quality of a scientific explanation.

Researchers still have to decide research directions, judge the strength of evidence, and confirm that new analyses are trustworthy. There are also new engineering issues: running external code requires isolated environments and permission restrictions, distributing tools must comply with the original code licenses, and if new analyses lead to discoveries, method authors, data contributors, and study designers must be appropriately credited. The paper also discusses the idea of having a maintenance Agent periodically re-run tests—making tools public is only the beginning; provenance, licensing, and ongoing maintenance all need to keep pace.

For researchers, Paper2Agent's value lies not in replacing scientific judgment, but in handing the work of finding and executing published methods to an Agent, giving time back for choosing problems, judging evidence, and designing validation.