Concept前缀语言模型
PrefixLM
A language model architecture between autoencoding and autoregressive approaches, in which the input sequence is divided into a prefix (bidirectional attention) and a suffix (causal attention). It is commonly used for text generation, translation, and summarization tasks.
Timeline (last 90 days)
Oct 13
PrefixLM (prefix language model) is an architecture design between autoencoding (such as BERT) and autoregressive (such as GPT): the input sequence is divided into a prefix part and a suffix part, tokens within the prefix part can see each other (bidirectional attention), and the suffix part can only see itself and preceding tokens (causal attention).
Unverified50%