Unverified50% confidenceFactExact time
Residual RMSNorm performs root-mean-square normalization on the residual branch, and the normalization coefficient is determined by global statistics, so any premature rounding is amplified across the entire hidden state.
1
Sources
50%
Confidence
Long-term
Relevance
10/26/2026
First Seen
Sources
Related Entities
Related Claims
Unverified自回归模型将联合概率拆解为逐步条件概率的乘积，通过因果注意力掩码确保每个token只能看到左侧历史信息71% similarUnverifiedDuring MDM generation, the conditional distribution at each position can be computed simultaneously, which naturally supports parallel decoding, in sharp contrast to the way autoregressive models generate tokens one by one.68% similarUnverifiedMasked diffusion randomly masks some tokens in the sequence during training, and during generation starts from the masked state and gradually denoises through multiple iterations to restore the complete sequence, with each step able to predict tokens at multiple positions in parallel.68% similarUnverifiedPPLM dynamically adjusts hidden state directions during the generation process by computing the gradient of a classifier with respect to the model's hidden states, steering outputs to better conform to target attributes.68% similarUnverifiedThe computation flow of a single Decoder layer is: the input vector passes through Self-Attention and Add & Norm, then through FFN and Add & Norm, producing the output vector67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1098065API
curl https://kongchang.com/api/v1/knowledge/claims/1098065MCP
get_claim(id=1098065)