Backtrack

Backtrack is a lightweight failure detection and recovery mechanism for frozen VLA models that requires no demonstration data, policy retraining, or additional inference models.

Backtrack uses a small failure prediction head, trained only on outcome-labeled trajectories produced by the frozen policy, that reads the VLA's internal representations to estimate whether the current episode will fail.

When the predicted failure probability exceeds a threshold, Backtrack pauses the policy, backtracks along the robot arm's recent motion trajectory, and returns control to the unmodified policy so that it retries from an earlier configuration.

Backtrack's failure prediction head adds only 0.48%–0.88% of the policy's parameter count.

On the LIBERO-Long long-horizon multi-step task set, Backtrack reduced the failure rate of each policy by 19%–48%, with success rate improvements of up to 7.4 percentage points.

On the three shorter LIBERO task sets, where control success rates already reached 85%–99%, Backtrack's effect was smaller.

On MetaWorld, all twelve controller configurations tested with SmolVLA outperformed the unassisted policy.

Backtrack's design does not depend on a specific action generation paradigm—it only reads the policy's internal representations and applies external backtracking control—so it is compatible with both flow-matching and autoregressive models.

Backtrack's effectiveness depends on the accuracy of the failure prediction head, and since the prediction head is trained only on rollouts of a specific policy, it may be difficult to transfer to new tasks or new environments.

Backtrack's backtracking motions require the robot arm to have a certain range of motion freedom, and may not be applicable in constrained scenarios.