Observational Scaling
Observational scaling refers to the methodology of predicting target capabilities through observable proxy signals such as public benchmark scores or early interaction feedback, drawing on scaling-law research for large models.
Timeline (last 90 days)
The paper studies two perspectives on observation scaling: predicting a VLM's relative performance as an agent from its public capabilities, and predicting subsequent progress from early physical feedback.
Experiments show that an agent's performance in the first few rounds of interaction can, to some extent, predict the improvement it will ultimately achieve.
Observation scaling presupposes a stable monotonic mapping between the proxy signal and the target capability, but the mapping between public capabilities and physical capabilities may be disturbed by factors such as task distribution, action space, and differences in perceptual modality, so prediction accuracy is limited.
All Facts (3)
The paper studies two perspectives on observation scaling: predicting a VLM's relative performance as an agent from its public capabilities, and predicting subsequent progress from early physical feedback.
50%UnverifiedExperiments show that an agent's performance in the first few rounds of interaction can, to some extent, predict the improvement it will ultimately achieve.
50%UnverifiedObservation scaling presupposes a stable monotonic mapping between the proxy signal and the target capability, but the mapping between public capabilities and physical capabilities may be disturbed by factors such as task distribution, action space, and differences in perceptual modality, so prediction accuracy is limited.
50%