Compositional Generalization

ZAP can complete compositional generalization tasks that do not appear in the training data, including placing test tubes into a test tube rack and selecting hole positions, handing over test tubes between two arms in mid-air, drawing X on a tic-tac-toe board, and playing chess.

Compositional generalization refers to a model's ability to understand and generate novel combinations unseen in training, for example understanding "the cat chases the ball" after learning "the cat chases the mouse" and "the dog chases the ball".

Triple-structured representations, by decomposing complex propositions into basic units and combining them explicitly, are expected to improve a model's sensitivity to compositional structure and thus generalize better to novel combinations.

Traditional neural networks perform poorly on compositional generalization, often relying on surface statistical regularities.

Auxiliary tasks are a common design pattern in multi-task learning, where additional prediction objectives are added beyond the primary training objective, with these auxiliary objectives sharing some of the model's parameters.

The academic community typically uses compositional generalization to measure genuine reasoning ability.

Merging the parameters of multiple trained models to generate new trade-off combinations has been shown to be helpful.

The concept of compositional generalization originates from the generative grammar theory of linguist Noam Chomsky, referring to the generation of infinite combinations through finite rules.

Research shows that large-scale multimodal models trained on massive data have to some extent developed emergent compositional generalization capabilities, but still fail on certain extreme combinations.