Sample Complexity
A foundational concept in statistical learning theory that quantifies the number of training samples required by a learning algorithm to achieve a specified level of accuracy and confidence. Sample complexity serves as a key theoretical metric for evaluating the data efficiency of learning algorithms, providing bounds that connect dataset size to generalization performance.
Timeline (last 90 days)
Theoretical analysis shows that the algorithm outputs an ε-close hypothesis with high probability, with both sample complexity and time complexity being poly(d, k, 1/ε).
When the student network's convolution kernel size matches the teacher's, only O(D) samples are needed to recover the teacher parameters; if the kernel size is larger, O(D log D) samples are needed (D being the feature dimension).
The conclusion of overcoming the curse of dimensionality relies on the conditional distribution having a certain smoothness structure (such as Hölder smoothness) and on a refined characterization of the relationship between network depth, width, and the smoothness of the target function.
The explicit complexity bounds in the paper usually hide large constant factors, and the actual implementation is also affected by numerical stability and errors in condition number estimation.
Machine learning measurement of semantic complexity is typically operationalized through sample complexity or convergence speed.
Higher sample complexity means the concept is harder for the learning algorithm and is regarded as a signal of higher semantic complexity.
All Facts (6)
Theoretical analysis shows that the algorithm outputs an ε-close hypothesis with high probability, with both sample complexity and time complexity being poly(d, k, 1/ε).
50%UnverifiedWhen the student network's convolution kernel size matches the teacher's, only O(D) samples are needed to recover the teacher parameters; if the kernel size is larger, O(D log D) samples are needed (D being the feature dimension).
50%UnverifiedThe conclusion of overcoming the curse of dimensionality relies on the conditional distribution having a certain smoothness structure (such as Hölder smoothness) and on a refined characterization of the relationship between network depth, width, and the smoothness of the target function.
50%UnverifiedThe explicit complexity bounds in the paper usually hide large constant factors, and the actual implementation is also affected by numerical stability and errors in condition number estimation.
50%UnverifiedMachine learning measurement of semantic complexity is typically operationalized through sample complexity or convergence speed.
50%UnverifiedHigher sample complexity means the concept is harder for the learning algorithm and is regarded as a signal of higher semantic complexity.
50%