GPT
GPT (Generative Pre-trained Transformer) is a series of large-scale pre-trained language models developed by OpenAI, built on the Transformer architecture. The models are trained via unsupervised pre-training on large text corpora and subsequently fine-tuned for specific applications. They support a range of natural language tasks including text completion, question answering, summarization, translation, and code generation, and are among the most representative foundation models in natural language processing.
Core Facts
Large language models such as GPT and BERT are based on the self-supervised learning paradigm
The in-context learning capability of Few-Shot prompting was first systematically described in the GPT-3 paper.
The Transformer architecture was proposed in 2017 by the Google Brain team in the paper "Attention Is All You Need"
Timeline (last 90 days)
ClawsGO 的 long-horizon harness 官方称能调度 Claude 和 GPT 智能体集群连续运行数天。
After logging in to the TokenRouter platform, users can go to the API key page, check multiple model families such as GPT, Claude, Gemini, Kimi, and DeepSeek from All Models, and generate a single API key to call multiple models at once.
After the Transformer appeared in 2018, the industry branched into three technical routes: the encoder-only BERT route, the decoder-only GPT route, and the T5 route that uses encoder and decoder together.
Before 2021 the industry generally chose the BERT route, and after 2021 it rapidly shifted to the GPT route.
All time comparisons in the paper are between NCM and diffusion models, and never compare speed with mainstream autoregressive models, so the 31x is not 31 times faster than GPT.
Mainstream large language models (such as the GPT and Llama series) use an autoregressive approach to predict the next word token by token, and the generation process is serial.
Cursor folded subscriptions into the X system, canceling the original $20 Pro tier; the new tier structure is: the $8 tier supports only Grok, using Claude or GPT models requires at least the $30 tier, and higher usage is billed separately at API prices.
Unlike autoregressive models such as GPT that generate token by token from left to right, dLLMs do not have a fixed revealing order at inference time and have the potential for parallel decoding and generation in arbitrary order, but this introduces the problem of deciding the decoding order.
Using the same model to review code it wrote itself catches fewer bugs, because it fails in the same way during review; therefore different models should be used for code writing and code review (model inversion).
Computer-use capability performs far better in the new generation of GPT and Opus models than in previous generations, and end-to-end QA is approaching being solved for the first time.
40 more timeline events
All Facts (20)
Large language models such as GPT and BERT are based on the self-supervised learning paradigm
90%VerifiedThe in-context learning capability of Few-Shot prompting was first systematically described in the GPT-3 paper.
90%VerifiedThe Transformer architecture was proposed in 2017 by the Google Brain team in the paper "Attention Is All You Need"
90%VerifiedThe core working principle of large language models such as GPT-4, Claude, and Gemini is next-token prediction based on the Transformer architecture.
90%VerifiedOpenAI builds an AI Agent tool ecosystem through ChatGPT's plugin system and the GPT Store.
90%VerifiedGPT-4 and Claude are large language models that can understand context, generate structured outputs, and execute multi-step reasoning
85%VerifiedAzure OpenAI Service provides enterprise customers with the ability to invoke models such as GPT-4, DALL-E, and Whisper in a compliant environment.
85%VerifiedWorkBuddy's interface is very similar to those of Qianwen, Doubao, and GPT.
80%VerifiedGPT is a series of large language models developed by OpenAI based on the Transformer architecture
80%VerifiedThe GPT series follows the decoder route, adhering to the autoregressive language model objective, and has revealed the phenomenon of emergent capabilities through scaling.
80%VerifiedThe GPT series uses tiktoken (based on BPE encoding) for tokenization.
80%VerifiedThe GPT series uses the Decoder-only variant of the Transformer, predicting the next Token autoregressively.
80%VerifiedVLMs (Vision Language Models) typically consist of a visual encoder (such as ViT), a language model, and a projection layer connecting the two, with representative models including GPT-4V/4o, Claude 3.5, the LLaVA series, and InternVL
80%VerifiedThe project connects to large language models via OpenRouter, which serves as an AI model gateway supporting unified API calls to multiple large models such as GPT-4, Claude, and Llama.
80%VerifiedSince 2024, OpenAI has reduced the API prices of the GPT series multiple times.
80%VerifiedThe Function Calling mechanism was first introduced into the GPT series of models by OpenAI in 2023.
80%Verified当前所有主流大模型（GPT、BERT、LLaMA等）都基于Transformer架构
80%VerifiedGLM采用了独特的自回归填空预训练范式，与GPT的纯自回归和BERT的纯掩码语言模型都有所不同
80%VerifiedGPTs is a custom ChatGPT feature launched by OpenAI at the first DevDay in 2023.
80%VerifiedThe context windows of current mainstream large language models (such as GPT-4, Claude, etc.) have been expanded to 128K or even longer tokens.
80%