Conceptattention dilution
Attention Diffusion
A phenomenon in attention mechanisms whereby, as input size grows, softmax normalization spreads attention weights thin and raises entropy, making it difficult for the model to focus on key nodes and leading to performance degradation.
Timeline (last 90 days)
Oct 21
Using CVRP as a controlled proxy, the research team systematically identified three factors behind cross-scale generalization degradation: node density shift, attention diffusion, and dilution of graph context by mean pooling.
Unverified50%