How Much Should You Trust VLA Reasoning? Q-Adaptive Guidance Sets New Waymo SOTA

Vision-Language-Action (VLA) models are becoming a mainstream paradigm for autonomous driving and robot control. These models typically first write out a reasoning trace, then decode specific actions based on that trace. But a tricky problem follows: if the reasoning trace goes wrong, the action is bound to go wrong; even if the reasoning looks correct, it may steer planning off course.

Most existing research asks "is the reasoning correct?"—but a new paper asks a different question: "how strongly should reasoning influence the action?" This work proposes Q-adaptive guidance, which models the choice of guidance weight as a Markov Decision Process and fits it with an exactly measurable action value function, achieving leading results on both the Alpasim simulator and the Waymo end-to-end driving benchmark.

Reframing the Problem: From "Is the Reasoning Correct?" to "How Strong Should the Reasoning Be?"

The traditional approach treats reasoning quality as a binary judgment: adopt it if correct, discard it if wrong. But the paper points out that even when a reasoning trace appears correct on the surface, it may mislead subsequent planning in the details. What really needs to be controlled is the degree to which reasoning intervenes in the final action.

The method the authors adopt is called CoC-CFG: within the same context, they perform classifier-free guidance using a policy with the reasoning segment and a policy without it. A weight is set at each Euler step to attenuate or amplify the influence of reasoning. When w=1, it degenerates to the original policy, so this framework naturally includes the "no guidance at all" baseline.

The key insight is that this weight should not be a fixed hyperparameter, but should be adjusted dynamically per step and per scenario.

Classifier-Free Guidance (CFG) first became popular in diffusion generative models: during training, condition information is randomly dropped so the model learns both "conditional" and "unconditional" distributions; at inference, the two predictions are extrapolated and combined as ε = ε_uncond + w·(ε_cond − ε_uncond), where the weight w controls how much the conditional signal is amplified relative to the unconditional prior. w=1 is equivalent to a pure conditional model, while w>1 strengthens conditional consistency but may sacrifice diversity. CoC-CFG transfers this idea to VLA action decoding: the condition is not a class label but a reasoning trace segment, and the unconditional branch is "look only at the context, not the reasoning," so the guidance weight naturally becomes a knob for "reasoning intervention strength."

Turning Weight Selection into a Measurable MDP

The paper models the weight selection at each step as a Markov Decision Process, but the action value is not estimated via bootstrapping—it is measured directly. Specifically: freeze the expert policy, fix the noise, apply a weight at a certain step, then use the current controller to complete the remaining planning, so the true value of that weight can be obtained exactly.

With exact value labels, Q-adaptive guidance uses a small critic network to fit these values and improves the controller in a conservative manner. This "measure rather than estimate" design avoids the value overestimation and training instability common in reinforcement learning, and makes a small model sufficient.

Bootstrapping is the mainstream reinforcement learning approach to estimating action values: the target value for the current step is constructed from the estimated value of the next state V(s') or Q(s',a'), then propagated backward step by step. It does not require completing an entire episode and has high sample efficiency, but errors accumulate recursively along time steps, and with function approximation it easily leads to overestimation (such as the max operator bias in DQN) and training instability. This paper does the opposite: after applying a candidate weight at each Euler step, it uses the frozen controller to actually complete the remaining planning, directly obtaining the return for that weight as the label. The cost is a full rollout for each candidate weight, but in exchange it obtains unbiased, controllable-variance value labels, so that a small critic is sufficient to fit them, and it bypasses the inherent bias of bootstrapping.

Experimental Results: Leading on Both Open-Loop and Closed-Loop

In the Alpasim driving simulator, Q-adaptive guidance outperforms all fixed-weight schemes as well as online reinforcement learning methods (SAC, GRPO) in the open-loop setting. In the closed-loop test, it passes 262 of 300 scenarios, compared with 252 for the original policy—a clear improvement.

On held-out frames of the Waymo end-to-end driving benchmark, the method's first three stages likewise outperform all fixed weights, four adaptive guidance methods, and SAC and GRPO. Even more convincing is its performance on the hidden test set: frozen Alpamayo 1.5 combined with rule-based camera and vehicle transformations, plus Q-adaptive guidance, achieves a new state of the art among published methods, with a scoring feedback score of 8.135159, compared with the previous best published method's 8.079.

Open-loop and closed-loop evaluation measure different things in autonomous driving. Open-loop evaluation compares the model's predicted trajectory point by point with the human driver's trajectory in the dataset (e.g., displacement error, collision rate); it is cheap to compute and reproducible, but suffers from "causal confusion": the model may merely reproduce the recorded trajectory without truly responding to state changes caused by its own actions. Closed-loop evaluation lets the model actually drive in a simulator, where actions change subsequent observations, exposing cumulative errors and distribution shift, but it depends on the simulator's fidelity. This paper reports both types of results and uses Alpasim's 300-scenario closed-loop pass rate to corroborate that the open-loop lead is not overfitting to recorded data; this dual-track design strengthens the credibility of the conclusions.

What This Work Means

First, reasoning-enhanced VLA systems need an adjustable "trust knob" rather than an all-or-nothing switch. Step-level weights provide a fine-grained control mechanism, letting the model rely more on reasoning when reasoning is reliable and fall back to the context itself when reasoning is suspect.

Second, formalizing guidance weight selection as an MDP and measuring action values is a path that balances theoretical clarity and engineering feasibility. Compared with end-to-end RL training of the entire policy, learning only a small critic is obviously more compute-efficient and easier to converge.

Third, the results show that "how to use reasoning" is as important as "reasoning itself." For safety-sensitive scenarios such as autonomous driving, a controllable reasoning intervention mechanism may bring practical benefits earlier than simply improving reasoning capability.

The paper has not yet disclosed the failure modes of weight adjustment in extreme scenarios, nor the generalization performance of the critic in out-of-distribution scenarios; these remain directions worth watching in follow-up work.

One-line Summary

Models VLA reasoning guidance weights as an MDP and directly measures action values to achieve step-level adaptive guidance.

Paragraph Summary

The article introduces a new study on reasoning guidance for Vision-Language-Action (VLA) models. The traditional approach treats reasoning quality as a binary judgment, whereas this paper instead asks "how strongly should reasoning intervene in the action?" The authors adopt CoC-CFG, performing classifier-free guidance within the same context using a policy with reasoning and one without, setting a weight at each Euler step and modeling the weight selection as a Markov Decision Process. Unlike common reinforcement learning, action values are measured directly by freezing the expert policy, fixing the noise, and completing a full rollout, rather than estimated via bootstrapping, thereby avoiding value overestimation and training instability, with a small critic sufficient to fit them. Experimentally, the method leads in both open-loop and closed-loop Alpasim simulation (passing 262 of 300 scenarios vs. 252 for the original policy) and achieves a SOTA score of 8.135159 on the hidden test set of the Waymo end-to-end benchmark. The article argues that reasoning-enhanced VLA needs an adjustable "trust knob," and that "how to use reasoning" is as important as "reasoning itself."