PhysiCalWorld
A test-time adaptation method for world models that extracts physical descriptor tokens through active exploration videos and injects them as in-context prompts into the world model, calibrating scene physical properties without fine-tuning or physical labels.
Timeline (last 90 days)
PhysiCalWorld uses in-context prompting to let the world model quickly calibrate to the physical characteristics of the current scene at test time without fine-tuning model parameters.
PhysiCalWorld does not rely on any physical labels or analytical modeling, but instead extracts a compact physics descriptor token from a segment of interaction video in which it actively explores the test scene.
PhysiCalWorld is trained end-to-end using only a latent prediction loss, without any manually annotated physical parameters.
At inference time, the descriptor token needs to be extracted only once and can be reused across multiple rollouts, with negligible computational overhead.
The research team evaluated PhysiCalWorld in four environments covering a variety of physical changes.
Experimental results show that PhysiCalWorld significantly improves the accuracy of rollout predictions and the success rate of visual planning tasks.
The learned physics descriptor forms a physically structured latent space, enabling the model to generalize to physical conditions not seen during training.
One potential limitation of PhysiCalWorld is that it requires a segment of exploratory interaction as input, which in practical applications may require additional active exploration strategies.
PhysiCalWorld applies the idea of in-context calibration to physical dynamics modeling systematically for the first time.
All Facts (9)
PhysiCalWorld uses in-context prompting to let the world model quickly calibrate to the physical characteristics of the current scene at test time without fine-tuning model parameters.
50%UnverifiedPhysiCalWorld does not rely on any physical labels or analytical modeling, but instead extracts a compact physics descriptor token from a segment of interaction video in which it actively explores the test scene.
50%UnverifiedPhysiCalWorld is trained end-to-end using only a latent prediction loss, without any manually annotated physical parameters.
50%UnverifiedAt inference time, the descriptor token needs to be extracted only once and can be reused across multiple rollouts, with negligible computational overhead.
50%UnverifiedThe research team evaluated PhysiCalWorld in four environments covering a variety of physical changes.
50%UnverifiedExperimental results show that PhysiCalWorld significantly improves the accuracy of rollout predictions and the success rate of visual planning tasks.
50%UnverifiedThe learned physics descriptor forms a physically structured latent space, enabling the model to generalize to physical conditions not seen during training.
50%UnverifiedOne potential limitation of PhysiCalWorld is that it requires a segment of exploratory interaction as input, which in practical applications may require additional active exploration strategies.
50%UnverifiedPhysiCalWorld applies the idea of in-context calibration to physical dynamics modeling systematically for the first time.
50%