Competitive Learning
A class of unsupervised neural network learning methods that introduce mutual inhibition among neurons, causing different neurons to spontaneously specialize in processing distinct input patterns. Competitive learning is commonly applied to feature extraction and clustering tasks.
Timeline (last 90 days)
In the early stage of hybrid architecture training, the recurrent layer rapidly masters retrieval capability and suppresses the signal for the attention layer to learn the same function, causing the attention layer to be unable to effectively learn retrieval in most cases and impairing overall performance.
The shared limitation of Hebbian learning and competitive learning is the lack of an explicit optimization objective, making end-to-end fine-tuning for downstream tasks difficult, and they are typically used for feature extraction and representation learning.
All Facts (2)
In the early stage of hybrid architecture training, the recurrent layer rapidly masters retrieval capability and suppresses the signal for the attention layer to learn the same function, causing the attention layer to be unable to effectively learn retrieval in most cases and impairing overall performance.
50%UnverifiedThe shared limitation of Hebbian learning and competitive learning is the lack of an explicit optimization objective, making end-to-end fine-tuning for downstream tasks difficult, and they are typically used for feature extraction and representation learning.
50%