LM2LM
A method that trains a language model via reinforcement learning to generate model improvement proposals. It uses a frozen LLM as a research world model to provide reward signals and periodically calibrates the reward through real execution, enabling recursive self-improvement.
Timeline (last 90 days)
The LM2LM method trains an LLM via reinforcement learning to generate valid model improvement proposals, and uses a frozen LLM as a research world model (RWM) to predict proposal returns as the reward signal.
LM2LM periodically executes proposals that rank high under the current policy and are overestimated by the RWM, and adds the actual measurement results to the RWM's context to calibrate the world model.
The paper claims that across eight autoregressive and diffusion language model development environments, LM2LM enables Qwen3.8-27B to surpass GPT-6-sol in performance, even though the latter was given 300 real experiment records in its context.
LM2LM reduces the cost of data generation by 33%.
LM2LM generalizes to model families not seen during training.
The core contradiction LM2LM faces is how to obtain scalable feedback without sacrificing reliability, because surrogate feedback becomes distorted as policy optimization shifts the proposal distribution.
LM2LM is currently validated only on eight development environments and a limited set of models, whereas the complexity and diversity of real-world scenarios are far greater; how the prediction accuracy and calibration frequency of the world model affect the final results is not elaborated in detail in the paper.
LM2LM designs a human-machine collaborative feedback loop: the model proposes proposals, the world model screens them, and real execution calibrates them.
All Facts (8)
The LM2LM method trains an LLM via reinforcement learning to generate valid model improvement proposals, and uses a frozen LLM as a research world model (RWM) to predict proposal returns as the reward signal.
50%UnverifiedLM2LM periodically executes proposals that rank high under the current policy and are overestimated by the RWM, and adds the actual measurement results to the RWM's context to calibrate the world model.
50%UnverifiedThe paper claims that across eight autoregressive and diffusion language model development environments, LM2LM enables Qwen3.8-27B to surpass GPT-6-sol in performance, even though the latter was given 300 real experiment records in its context.
50%UnverifiedLM2LM reduces the cost of data generation by 33%.
50%UnverifiedLM2LM generalizes to model families not seen during training.
50%UnverifiedThe core contradiction LM2LM faces is how to obtain scalable feedback without sacrificing reliability, because surrogate feedback becomes distorted as policy optimization shifts the proposal distribution.
50%UnverifiedLM2LM is currently validated only on eight development environments and a limited set of models, whereas the complexity and diversity of real-world scenarios are far greater; how the prediction accuracy and calibration frequency of the world model affect the final results is not elaborated in detail in the paper.
50%UnverifiedLM2LM designs a human-machine collaborative feedback loop: the model proposes proposals, the world model screens them, and real execution calibrates them.
50%