LongZero
A zero-data reinforcement learning method that trains a large language model via RL on multi-hop reasoning and multi-turn retrieval tasks generated by the model itself, improving long-context reasoning without external training data.
Timeline (last 90 days)
The LongZero method lets a model improve its long-context reasoning ability entirely using data it generates itself, without any external data.
LongZero adopts a two-stage "compress first, then expand" design: first the model synthesizes questions, answers, and supporting evidence in a compact structured form, then the evidence is expanded into multi-document long contexts.
LongZero constructs two types of self-generated tasks: multi-hop reasoning tasks (structured synthesis followed by expansion) and multi-turn retrieval tasks (the model writes questions that require multiple rounds of retrieval to answer).
LongZero uses the GRPO algorithm to train the model, mixing the two task types of multi-hop reasoning and multi-turn retrieval.
The researchers evaluate LongZero on two LLMs across five long-context reasoning benchmarks and compare it against baseline methods that train with RL on synthetic data from external corpora.
LongZero surpasses two baseline methods that use synthetic data from external corpora in performance.
LongZero can generalize to real tasks.
LongZero ensures through structured synthesis that every question comes with explicit supporting evidence and an answer, so that rewards can be computed based on rules or model self-checking, thereby bypassing manual annotation.
All Facts (8)
The LongZero method lets a model improve its long-context reasoning ability entirely using data it generates itself, without any external data.
50%UnverifiedLongZero adopts a two-stage "compress first, then expand" design: first the model synthesizes questions, answers, and supporting evidence in a compact structured form, then the evidence is expanded into multi-document long contexts.
50%UnverifiedLongZero constructs two types of self-generated tasks: multi-hop reasoning tasks (structured synthesis followed by expansion) and multi-turn retrieval tasks (the model writes questions that require multiple rounds of retrieval to answer).
50%UnverifiedLongZero uses the GRPO algorithm to train the model, mixing the two task types of multi-hop reasoning and multi-turn retrieval.
50%UnverifiedThe researchers evaluate LongZero on two LLMs across five long-context reasoning benchmarks and compare it against baseline methods that train with RL on synthetic data from external corpora.
50%UnverifiedLongZero surpasses two baseline methods that use synthetic data from external corpora in performance.
50%UnverifiedLongZero can generalize to real tasks.
50%UnverifiedLongZero ensures through structured synthesis that every question comes with explicit supporting evidence and an answer, so that rewards can be computed based on rules or model self-checking, thereby bypassing manual annotation.
50%