LSTM
Long Short-Term Memory (LSTM) is a specialized recurrent neural network architecture proposed by Hochreiter and Schmidhuber in 1997. Its core design introduces three gating mechanisms—input gate, forget gate, and output gate—along with a cell state that selectively retains or discards information. This design effectively mitigates the vanishing and exploding gradient problems present in standard RNN training, enabling the capture of long-range sequential dependencies. LSTM is widely applied in natural language processing, speech recognition, and time-series forecasting.
Core Facts
The Transformer's core innovation is the self-attention mechanism, which allows models to attend to all positions in the input simultaneously rather than processing step-by-step like RNNs or LSTMs
Through the self-attention mechanism, Transformer allows the model to dynamically attend to all other tokens in the sequence when processing each token, while enabling fully parallelized training.
The self-attention mechanism of the Transformer allows the model to directly compute dependencies between any two positions in a sequence, breaking through the parallelization bottleneck caused by the sequential computation of RNNs/LSTMs.
Timeline (last 90 days)
At the scheduling instant, RT-LSTM's error reduction relative to LSTM reaches 12.5%.
Compared with existing baseline models, RT-LSTM reduces MAE by 4.3% (vs. LSTM), 11.6% (vs. StarNet), and 22.1% (vs. the attention-based spatiotemporal model ASTM).
RT-LSTM adopts a dual-LSTM structure: the history LSTM encodes historical RTT observations and real-time relative features of the serving satellite and gateway; the future LSTM processes future features, derived from TLE propagation, of the identified serving satellite up to the next scheduling instant.
Commonly used models for predictive maintenance include isolation forests, autoencoders, LSTM and gradient boosted trees, among others.
LSTM selectively retains or discards historical information through three gating mechanisms—the forget gate, input gate, and output gate—alleviating the long-range forgetting problem caused by vanishing gradients.
Both LSTM and GRU retain a structure that computes sequentially by time step, and therefore cannot be parallelized.
GRU is a simplified version of LSTM, merging the three gates into two—the update gate and the reset gate—with fewer parameters and faster training, achieving results close to LSTM on many tasks.
In terms of network structure, a CNN processes a 21x21 local observation window, an LSTM carries memory across steps, and three attention modules—SelfAttention, MemoryCrossAttention and FiLM—are stacked on top of the CNN.
The technical foundations of workload prediction typically include time series analysis and machine learning models (such as LSTM, Transformer, etc.), which can predict future load changes from minutes to hours in advance.
Choosing simple models such as Ridge Regression over complex models like LSTM and XGBoost is preferable at the prototype stage, because interpretability and training stability are more important, and complex models are prone to overfitting on small-scale synthetic datasets.
37 more timeline events
All Facts (20)
The Transformer's core innovation is the self-attention mechanism, which allows models to attend to all positions in the input simultaneously rather than processing step-by-step like RNNs or LSTMs
90%VerifiedThrough the self-attention mechanism, Transformer allows the model to dynamically attend to all other tokens in the sequence when processing each token, while enabling fully parallelized training.
85%VerifiedThe self-attention mechanism of the Transformer allows the model to directly compute dependencies between any two positions in a sequence, breaking through the parallelization bottleneck caused by the sequential computation of RNNs/LSTMs.
80%VerifiedThe Transformer architecture completely abandons recurrent structures and relies solely on attention mechanisms to handle sequential relationships, resolving the inherent defects of RNNs and LSTMs including vanishing gradients and the inability to perform parallel computation.
80%VerifiedBefore the advent of Transformer, the field of natural language processing relied primarily on RNNs and LSTMs to process sequential data, architectures that had to process text word by word in sequence and could not perform parallel computation
70%VerifiedRNNs prior to the Transformer could only process words sequentially one at a time, whereas the Transformer can process entire sequences in parallel, greatly improving training efficiency.
65%VerifiedLSTM solves the vanishing gradient problem that standard RNNs encounter when processing long sequences by introducing three control mechanisms: the forget gate, the input gate, and the output gate
65%UnverifiedGNMT系统采用8层LSTM编码器和8层LSTM解码器，并引入了残差连接和注意力机制
90%UnverifiedQlib内置了多种机器学习模型（如LightGBM、LSTM、Transformer等）用于股票预测，并支持自定义因子和策略开发
85%Unverified现代病虫害预警模型常用的建模方法包括时间序列分析、随机森林、LSTM神经网络
85%UnverifiedDatabox的AI能力使用异常检测（如Isolation Forest、LSTM神经网络）和时序预测技术来主动发现数据中的模式和异常
70%Partially VerifiedThe self-attention mechanism allows the model to attend to information at all positions simultaneously, eliminating the sequential dependency limitations of RNNs/LSTMs
65%UnverifiedLSTM addresses the long-range dependency problem by introducing three gating mechanisms: the forget gate, the input gate, and the output gate.
60%UnverifiedCompared to RNNs and LSTMs, which forget earlier content in long texts, all positions in a Transformer can be computed in parallel, making training efficiency far superior to that of RNNs and LSTMs.
60%UnverifiedGoogle's Neural Machine Translation system (GNMT) is based on a multi-layer LSTM plus attention architecture, and RNN-based models dominated the field of machine translation between 2014 and 2016
60%UnverifiedGRU (Gated Recurrent Unit) was proposed by Cho et al. in 2014, controlling information flow through two gating mechanisms—the update gate and the reset gate—with fewer parameters and faster training compared to LSTM.
60%UnverifiedTesseract introduced an LSTM-based recognition engine starting from version 4.0, and version 5.x supports Marathi (the mar language pack).
60%UnverifiedLSTM was proposed by Hochreiter and Schmidhuber in 1997, and its gating mechanism solved the vanishing gradient problem of standard RNNs.
60%UnverifiedTraditional RNNs suffer from the vanishing gradient problem; in practice, LSTMs and GRUs are more commonly used to address this.
60%UnverifiedRT-LSTM adopts a dual-LSTM structure: the history LSTM encodes historical RTT observations and real-time relative features of the serving satellite and gateway; the future LSTM processes future features, derived from TLE propagation, of the identified serving satellite up to the next scheduling instant.
50%