Neural Tangent Kernel
A kernel function that describes the dynamics of infinitely wide neural networks trained by gradient descent. Determined by the network architecture and initialization, it remains constant in the infinite-width limit, making the training of neural networks equivalent to a kernel method.
Core Facts
Neural Tangent Kernel (NTK) theory states that when the width of a neural network approaches infinity, the network's behavior during gradient descent training is equivalent to kernel regression with the NTK as the kernel, and the network does not perform feature learning.
The NTK framework cannot capture the feature learning phenomena observed in real finite-width networks.
Timeline (last 90 days)
The neural tangent kernel (NTK) framework shows that as network width tends to infinity, training dynamics can be approximated as kernel regression under a fixed kernel function.
Neural Tangent Kernel (NTK) theory states that when the width of a neural network approaches infinity, the network's behavior during gradient descent training is equivalent to kernel regression with the NTK as the kernel, and the network does not perform feature learning.
对于有限宽度网络，NTK在训练中的变化量与1/√m成正比（m为宽度），这一偏差是特征学习发生的机制
神经正切核理论由Arthur Jacot等人于2018年提出，其核心洞见是当网络宽度趋于无穷时训练过程退化为核回归问题
在NTK框架下训练损失以指数速率e^{-2ηλ_min t}衰减，收敛速度由核矩阵最小特征值λ_min决定
All Facts (6)
Neural Tangent Kernel (NTK) theory states that when the width of a neural network approaches infinity, the network's behavior during gradient descent training is equivalent to kernel regression with the NTK as the kernel, and the network does not perform feature learning.
75%VerifiedThe NTK framework cannot capture the feature learning phenomena observed in real finite-width networks.
65%UnverifiedThe neural tangent kernel (NTK) framework shows that as network width tends to infinity, training dynamics can be approximated as kernel regression under a fixed kernel function.
50%Unverified对于有限宽度网络，NTK在训练中的变化量与1/√m成正比（m为宽度），这一偏差是特征学习发生的机制
50%Unverified神经正切核理论由Arthur Jacot等人于2018年提出，其核心洞见是当网络宽度趋于无穷时训练过程退化为核回归问题
50%Unverified在NTK框架下训练损失以指数速率e^{-2ηλ_min t}衰减，收敛速度由核矩阵最小特征值λ_min决定
50%