Domain Randomization

Domain randomization was first systematically applied by OpenAI in their 2017 robotic hand research

Domain randomization technology randomly perturbs physical parameters (such as mass, friction coefficient, motor delay, etc.) during training, forcing the policy to learn to operate robustly under various conditions.

The sim-to-real gap refers to the phenomenon where policies trained in simulation often suffer significant performance degradation when transferred to the real world, and domain randomization and system identification are commonly used mitigation techniques.

The four mainstream methods for narrowing the Sim-to-Real Gap are domain randomization, system identification, fine-tuning on real data, and refined actuator modeling.

The key challenge of Sim-to-Real is the Reality Gap, which typically requires domain randomization to enhance the robustness of policies.

Existing methods for mitigating the sim-to-real gap include domain randomization and system identification, but neither has fundamentally eliminated this gap.

Domain Randomization forces a policy to learn robustness to environmental changes by randomizing simulation parameters during training.

OpenAI used automatic domain randomization technology when training the Dota 2 agent.

Domain Randomization technology bridges the sim-to-real gap by randomly perturbing physical parameters in simulation (such as ground friction coefficients, robot mass distribution, and sensor latency) to force policy networks to learn generalizable control strategies that are robust to variable changes.

The hard-to-bridge gap between the simulated world and the real world, known as the Reality Gap, stems from fine-grained discrepancies such as motor response delays, joint friction, sensor noise, and ground material differences

The mainstream solutions for the Sim-to-Real Gap include domain randomization and system identification.

域随机化是指在仿真训练中刻意随机扰动物理参数，迫使策略学习到对参数变化具有泛化能力的行为

增强学习领域用'sim-to-real gap'描述在模拟环境训练的机器人策略因现实物理差异而在真实世界失效的现象

Common approaches to sim-to-real include: randomizing physical parameters (mass, friction coefficients, motor strength) during training, adding observation noise and action latency, using domain randomization to broaden the distribution of environments the policy sees, and performing state estimation and action smoothing at deployment.

The two mainstream approaches to addressing the Reality Gap are Domain Randomization and System Identification, and in practice the most effective solution is often a combination of both.

Domain randomization technology was first systematically proposed by OpenAI in its 2017 robotic grasping research and was later validated in the Rubik's Cube project.

The key to achieving zero-shot transfer lies in narrowing the sim-to-real gap, which typically relies on domain randomization and system identification.

Common improvement approaches include curriculum learning (gradually increasing task difficulty) and domain randomization (randomizing physical parameters to improve policy robustness).

Domain randomization can only cover the types of disturbance the designers anticipated, and long-tail situations in the real environment such as loose handrails, slippery floors, and sudden lighting changes may still fall outside the training distribution.

Methods complementary to domain randomization also include system identification (first measuring key parameters of the real system and then calibrating the simulation) and teacher-student distillation (letting a teacher policy trained with privileged information transfer knowledge to a student policy that relies only on realistically observable inputs).