SAO
A reinforcement learning algorithm for post-training large language models that combines single-pass sampling with an asynchronous optimization mechanism. It aims to improve long-horizon training stability and efficiency, and has demonstrated performance superior to GRPO on coding and mathematical reasoning benchmarks.
Timeline (last 90 days)
The weight recipes loop includes SAO, OpenClaw-RL, TTT-Discover, and Guidance-TTT, and requires GPUs and a full training stack.
Reef Infra provides a small but complete example named SAO in its repository, including the recipe, processor, objective, and backend loss code.
The single-rollout characteristic of SAO means that a more streamlined sampling and trajectory generation mechanism is used at each policy update step, reducing reliance on multiple rollouts.
The 'Asynchronous' in SAO (Asynchronous Optimization) refers to the decoupled parallel execution between data collection (rollout) and policy updates
SAO (Single-rollout Asynchronous Optimization) can maintain stability throughout a training process of up to one thousand steps.
SAO achieved leading results on both programming and mathematical reasoning, two substantially different task domains, indicating that the method possesses a degree of generality
The information currently disclosed about SAO comes primarily from the research team's preliminary announcements, and a complete technical report or paper is still lacking
The SAO team selected three high-difficulty benchmarks—SWE-Bench Verified, BeyondAIME, and IMOAnswerBench—for validation
SAO consistently outperforms GRPO and its various variants across multiple challenging benchmarks.
All Facts (8)
The weight recipes loop includes SAO, OpenClaw-RL, TTT-Discover, and Guidance-TTT, and requires GPUs and a full training stack.
50%UnverifiedReef Infra provides a small but complete example named SAO in its repository, including the recipe, processor, objective, and backend loss code.
50%UnverifiedThe single-rollout characteristic of SAO means that a more streamlined sampling and trajectory generation mechanism is used at each policy update step, reducing reliance on multiple rollouts.
50%UnverifiedThe 'Asynchronous' in SAO (Asynchronous Optimization) refers to the decoupled parallel execution between data collection (rollout) and policy updates
50%UnverifiedSAO (Single-rollout Asynchronous Optimization) can maintain stability throughout a training process of up to one thousand steps.
50%UnverifiedSAO achieved leading results on both programming and mathematical reasoning, two substantially different task domains, indicating that the method possesses a degree of generality
50%UnverifiedThe SAO team selected three high-difficulty benchmarks—SWE-Bench Verified, BeyondAIME, and IMOAnswerBench—for validation
50%UnverifiedSAO consistently outperforms GRPO and its various variants across multiple challenging benchmarks.
50%