When Learning Signals Are No Longer Continuous: A New Perspective on Instability

Learning algorithms typically assume that informative gradients are continuously available, but this assumption does not hold in settings such as biological neural networks. A recent study takes a deep look at the instability that learning processes may face when gradient signals are sparse, local, and only intermittently available, and derives explicit conditions for stable learning. This theoretical work provides new constraints for understanding biologically plausible learning models.

A Crack in the Mainstream Assumption: Gradients Are Not Always Reliable

Traditionally, our analysis of learning algorithms is built on the premise that accurate gradient information is available at every moment and at every synapse. Whether for backpropagation or various stochastic optimization methods, convergence proofs often rely on the unbiasedness of gradient estimates and controllable variance. However, learning signals in biological neural networks—such as synaptic plasticity signals—are often sparse, local, and only intermittently present at certain times. This means that a synapse may receive no effective feedback for a long period, and when feedback does arrive, the signal may be full of noise.

This study starts from this reality and asks a fundamental question: if gradient information is both noisy and intermittent, can learning still proceed stably?

Variance Explosion: Average Convergence Does Not Equal Stable Learning

The researchers demonstrate a troubling possibility: even if the average learning dynamics converge to the correct optimum, the variance of the learning process may grow asymptotically infinite. In other words, the expected value of the learning trajectory points in the right direction, but the fluctuations of the actual path can grow out of control over time, making the learning process unreliable in practice. This phenomenon is analyzed and verified in single-layer and multi-layer linear networks. The authors derive explicit conditions for stable learning. These conditions are complementary to the stability conditions of stochastic gradient descent (SGD), but conceptually distinct. SGD stability usually concerns the step size and the covariance structure of gradient noise, whereas the new conditions here are directly tied to the frequency and informativeness of feedback.

The core insight of this finding is that unbiased credit assignment alone is not sufficient to guarantee reliable learning; sufficiently frequent and informative feedback is also necessary. This adds an additional constraint to biologically plausible learning models—a model cannot merely assume that “on average the gradient is correct”; it must also ensure that the temporal distribution of feedback signals satisfies stability requirements.

Credit assignment is a core concept in machine learning and computational neuroscience, referring to the process of attributing system output errors to internal parameters or synaptic weights. In backpropagation, credit assignment precisely computes each parameter’s contribution to the loss via the chain rule; in biological neural networks, because there is no global error backpropagation pathway, credit assignment usually relies on local plasticity signals (such as dopamine modulation or correlations in pre- and post-synaptic activity) for approximation. Unbiased credit assignment means that these local signals point in the correct gradient direction in expectation, but as this article reveals, unbiasedness in expectation does not mean the learning process is stable—the temporal distribution properties of the signal are equally critical.

The stability conditions of stochastic gradient descent (SGD) usually refer to an upper bound on the step size to guarantee convergence, for example that the step size is less than twice the reciprocal of the Lipschitz constant of the loss function, or is related to the largest eigenvalue of the gradient noise covariance matrix. Classical analysis assumes that an unbiased gradient estimate can be obtained at every step and that the noise variance is bounded. The new conditions derived in this article are complementary: SGD stability focuses on step size and noise covariance, whereas the new conditions directly constrain the arrival frequency of feedback signals and the amount of information per feedback event. In other words, even if the step size is chosen to satisfy classical SGD conditions, if feedback is too sparse, variance can still accumulate to infinity and cause learning to fail. This distinction is crucial for understanding intermittent feedback settings.

Implications for Biologically Plausible Learning Models

For models that attempt to simulate the brain’s learning mechanisms, this result imposes clear design requirements. Many biologically inspired learning rules (such as Hebbian learning and its variants) attempt to approximate gradients using local signals, but often ignore the temporal statistical properties of the signals. The new study suggests that if feedback signals are too sparse or intermittent, even if they are unbiased in expectation, learning may fail due to variance accumulation. This may explain why biological systems require complex regulatory mechanisms to ensure that plasticity signals appear within appropriate time windows.

In addition, this work provides theoretical guidance for designing more robust artificial learning algorithms. In settings such as federated learning and asynchronous distributed optimization, gradient information may also be sparse or delayed. In such cases, beyond focusing on bias, it is also necessary to rigorously evaluate the impact of feedback frequency on stability.

Hebbian learning is one of the most representative biologically inspired learning rules, whose core idea is “neurons that fire together wire together,” that is, synaptic weights are adjusted according to the correlation between pre- and post-synaptic activity. However, the original Hebbian rule lacks an error signal and cannot directly perform credit assignment. Its variants (such as the Oja rule, the BCM rule, and spike-timing-dependent plasticity, STDP) stabilize learning by introducing normalization or threshold mechanisms, but they usually still assume that plasticity signals are continuously available. The conclusions of this article suggest that if Hebbian-like rules are to work stably under sparse, intermittent neuromodulatory signals, they must additionally account for the temporal statistical properties of signal arrival, rather than focusing only on their expected direction.

Federated learning and asynchronous distributed optimization are settings where the conclusions of this article have direct practical significance. In federated learning, each client computes gradients locally and uploads them intermittently to the server. Upload frequency is limited by factors such as communication bandwidth and device availability, causing the gradient updates received by the server to be sparse and delayed. In asynchronous distributed optimization, worker nodes complete gradients at different times, and the parameter server may not receive updates for certain shards for a long time. Classical analysis usually focuses only on gradient bias and delay compensation, whereas this article points out that feedback frequency itself is an independent dimension of stability: even if each client’s gradient is unbiased, if the aggregation frequency is too low, the variance of the global learning process may still diverge. This provides a new theoretical constraint for designing communication-efficient distributed algorithms.

Theoretical Tools and Open Questions

The authors conduct their analysis in several analytically tractable settings, including single-layer and multi-layer linear networks, and provide concrete forms of the stability conditions. These conditions clarify the trade-off between feedback frequency and noise level. Although linear networks are simplified models, their conclusions provide a benchmark for more complex nonlinear systems. Future research can explore how these conditions behave in nonlinear networks and in real biological data, and how to design local learning rules that satisfy these conditions.

This study reminds us that the stability of learning depends not only on “whether the direction is right,” but also on “whether the feedback is frequent enough.” For any learning system that attempts to operate under real, noisy, intermittent signals—whether artificial or biological—this is a constraint that must be taken seriously.

One-line Summary

Under sparse intermittent gradients, learning variance may diverge; feedback frequency and informativeness must satisfy explicit stability conditions.

Paragraph Summary

Traditional analyses of learning algorithms assume that gradient information is continuously available and unbiased, but learning signals in biological neural networks are often sparse, local, and intermittent. Recent research reveals that even if the average learning dynamics converge to the optimal solution, the variance of the learning process may grow asymptotically infinite, leading to practical instability. The authors derive explicit conditions for stable learning in linear networks. These conditions are complementary to, but conceptually different from, SGD stability conditions, emphasizing feedback frequency and informativeness rather than only step size and noise covariance. The core insight is that unbiased credit assignment is insufficient to guarantee reliable learning; sufficiently frequent and informative feedback is a necessary condition. This conclusion imposes an additional constraint on biologically plausible learning models and may affect settings such as federated learning and asynchronous distributed optimization.