1 related articles
Reddit discussion: BatchNorm fell out of favor due to inconvenience, yet the mechanism behind its effectiveness remains unsettled. This article reviews internal covariate shift, optimization landscape smoothing, regularization, and other mainstream explanations along with their cracks, analyzes why BatchNorm has been replaced by LayerNorm and others in engineering, and the practical costs of the theoretical gap.
