Representation Collapse
A core challenge in self-supervised learning in which a model maps all inputs to the same point or a low-dimensional manifold, driving the loss toward zero while failing to learn any meaningful representation. It undermines the goal of learning discriminative features from unlabeled data and motivates architectural and loss-function designs that enforce representation diversity.
Timeline (last 90 days)
In the study's experiments, all three JEPA variants trained stably without representation collapse.
JEPA-type methods typically employ asymmetric architectures (such as momentum encoders, stop-gradient) or regularization terms (such as VICReg's variance-covariance constraints) to prevent representation collapse.
All Facts (2)
In the study's experiments, all three JEPA variants trained stably without representation collapse.
50%UnverifiedJEPA-type methods typically employ asymmetric architectures (such as momentum encoders, stop-gradient) or regularization terms (such as VICReg's variance-covariance constraints) to prevent representation collapse.
50%