ELMo
A deep contextualized word representation model proposed by the Allen Institute for AI. It was the first systematic demonstration of the dynamic word vector approach and laid the foundation for subsequent pretrained language models.
Timeline (last 90 days)
ELMo was the first to systematically demonstrate the idea of context-dependent representations, and BERT and the GPT series pushed it into the mainstream through Transformer attention layers.
In 2018, ELMo introduced context-dependent embeddings, allowing the same word to obtain different vector representations in different contexts, thereby solving the polysemy problem.
ELMo was proposed by AllenNLP in 2018 and uses a bidirectional LSTM language model to produce word representations as weighted combinations of hidden states from different layers, pioneering contextual embeddings.
All Facts (3)
ELMo was the first to systematically demonstrate the idea of context-dependent representations, and BERT and the GPT series pushed it into the mainstream through Transformer attention layers.
50%UnverifiedELMo was proposed by AllenNLP in 2018 and uses a bidirectional LSTM language model to produce word representations as weighted combinations of hidden states from different layers, pioneering contextual embeddings.
50%UnverifiedIn 2018, ELMo introduced context-dependent embeddings, allowing the same word to obtain different vector representations in different contexts, thereby solving the polysemy problem.
50%