Unverified50% confidenceFactExact time
Plain gradient descent (GD) faces the problems of sensitivity to learning rate and slow convergence.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
From GD to Adam: An AI Engineer's Journey Through Optimizer Evolution
redditr/deeplearning10/17/2026
Related Entities
Related Claims
Unverified梯度下降的收敛条件与学习率和损失函数的Lipschitz常数的关系有关75% similarUnverified梯度下降的权重更新规则为 w = w - lr * gradient，其中lr是学习率72% similarVerifiedGradient descent has spawned three main variants — batch gradient descent, stochastic gradient descent, and mini-batch gradient descent — and the Adam optimizer is a combined improvement built on momentum and adaptive learning rates.71% similarUnverifiedConvergence failure is a multi-cause problem; learning rate, optimizer choice, missing batch normalization, unnormalized data, and improper weight initialization can all cause the loss to fail to decrease over a long period or to oscillate violently.71% similarUnverifiedIn the sampled-gradient setting, when the learning rate η decreases, the expected gain under fixed β vanishes faster than the fluctuation itself, causing the benefit of momentum to be swallowed by noise.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053739API
curl https://kongchang.com/api/v1/knowledge/claims/1053739MCP
get_claim(id=1053739)