Behavior Cloning

This experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.

Once incorrect experience learned by an agent from limited interactions becomes entrenched, it can lead to systemic bias and reduced work efficiency.

Behavior cloning transforms policy learning into a supervised learning problem, using environment states as inputs and expert actions as labels to train a mapping function from states to actions.

Behavioral cloning simplifies the problem to supervised learning, but suffers from distribution shift, where differences between the state distributions at training time and execution time lead to error accumulation.

Common technical approaches to making engines "human-like" include supervised learning on human games, behavior cloning, and sampling based on human mistake distributions.

Without a robot arm, the research team collected demonstration data across multiple real tasks and unstructured environments and trained visuomotor policies with behavioral cloning.

Commanded targets reflect the desired actions actually received by the controller and align semantically with the policy's outputs at deployment, whereas measured states mix in execution latency, compliant control and sensor noise, introducing systematic bias when used as labels.

Using recorded robot commands rather than states raises task performance from 11% to 64%.

In offline imitation learning (behavior cloning), when the expert has privileged information, the optimal solution of supervised learning with observation history as input and expert actions as labels is the conditional expectation of the expert action under the posterior distribution of the hidden state; this average often points to a compromise action, loses multimodality, and cannot learn exploratory behavior.

The core idea of DSP is: use a few minutes of human play data to train a play policy via behavior cloning, then let the robot autonomously run this policy to collect hours of interaction data for training a world model.

Imitation Learning is a class of methods that lets robots learn policies directly from human demonstrations, mapping observations to actions through behavior cloning or diffusion policies, and its core advantage is high sample efficiency, but its out-of-distribution generalization ability is weak.

FireFly's current training pipeline is primarily based on imitation learning/behavior cloning, training policies by recording screen observations and player actions.

Behavior cloning can only learn the state distribution covered by demonstration data, and error accumulation tends to occur when encountering rare situations.

A common pathway for evolving from behavior cloning to reinforcement learning is to use the behavior cloning policy as a warm start initialization for RL followed by online fine-tuning.

Action sequences generated by human players completing Nuzlocke challenges can serve as expert demonstration data for imitation learning/behavioral cloning and offline reinforcement learning.

Many GUI agents are trained based on Behavior Cloning, where the model learns the action distribution from human demonstration trajectories rather than the essential goal of the task

Behavioral Cloning (BC) is the most straightforward implementation method in LfD, and is essentially a supervised learning problem

Traditional behavioral cloning typically requires hundreds to thousands of demonstrations to train a usable policy model

该方法属于模仿学习中的行为克隆，其著名缺陷是复合误差问题

该方法属于模仿学习范畴中最简单的形式——行为克隆，直接将专家演示视为有标签数据进行监督学习