VLA

Google DeepMind's RT-2 was the first to directly use a vision-language model to generate robot action commands.

VLA directly connects pre-trained vision-language models with robot action policies, enabling robots to process natural language instructions, camera images, and low-level motion control in a unified manner.

VLA (Vision-Language-Action) models are a class of multimodal models that unify visual perception, natural language understanding, and action generation in a single end-to-end architecture, typically taking image sequences and language instructions as input and directly outputting robot joint angles, end-effector poses, or discrete action tokens.

Compared with traditional end-to-end imitation learning, VLA models have stronger generalization capabilities, but high inference latency, stringent requirements for training data diversity, and deployment on real-time control and low-cost hardware remain the primary challenges.

VLA models integrate visual perception, language understanding, and robot action planning into a unified model, typically using a pretrained large vision-language model as the backbone, with representative works including Google's RT-2 and OpenVLA.

The model belongs to the category of Vision-Language-Action (VLA) models, unifying a visual encoder, a language understanding module, and an action prediction head within an end-to-end architecture.

The current mainstream technical approach to embodied intelligence is the Vision-Language-Action model (VLA), which takes camera images and language instructions as joint inputs and directly outputs joint control signals for robotic arms.

RT-2 first demonstrated the feasibility of the VLA architecture by discretizing robot actions into tokens and having the vision-language model directly output action sequences.

DeepMind developed the VLA (Vision-Language-Action model) system, which treats the robot's physical actions on equal footing with visual tokens and language tokens.

The key difference between VLA and WAM compared with general MLLMs is that they have undergone action-space alignment, with outputs falling directly within the robot's controllable instruction space.

When the agent is allowed to invoke post-trained VLA or WAM policies, overall task success rate rises to the level of the strongest comparison policy, while token cost drops significantly.

The Harness+policy mode still falls below the per-task performance ceiling of each component running alone, for reasons including suboptimal policy invocation, lack of subgoal decomposition in fixed instruction execution, and interference with grasping when the agent regains control.

RobustRLlib's evaluation platform covers diverse real-world shift scenarios and includes humanoid robot and vision-language-action (VLA) tasks.

Backtrack is a lightweight failure detection and recovery mechanism for frozen VLA models that requires no demonstration data, policy retraining, or additional inference models.

AtomicRL is a reinforcement learning post-training framework that leverages atomic task structure to achieve reliable credit assignment and adaptive exploration, used to improve the performance of vision-language-action (VLA) models in long-horizon robotic manipulation.

On the Find-the-Item task, current state-of-the-art VLA models are only at or below chance level (success rate ≤33.3%).

Jia Xiaosong observed that models such as VLA and WAM that introduce stronger vision-language capabilities are not necessarily better than well-optimized end-to-end models on some autonomous driving benchmarks; what truly limits the model is whether it can make the right decision, not whether it recognizes a certain object.

In a closed-loop trajectory, a single action deviation puts the robot into a state outside the training distribution, and each subsequent step compounds error on top of the mistake, ultimately causing the entire episode to collapse—a failure that may show up as only a few percentage points of decline in average success rate.

Existing VLA accelerators (such as action chunking, visual token pruning, and quantized compilation) generally report only average task success rates, masking the risk of cascading failures that can occur during closed-loop execution.

CARE is a certification framework that provides statistical guarantees for VLA model acceleration configurations, guaranteeing that at least 85.8% of reference solved episodes still succeed at a user-specified failure budget with 95% confidence.