GPT-3.5
Core Facts
The context window in the GPT-3 era was only 2,048 tokens, whereas Google Gemini already supports millions of tokens, Claude supports 200K tokens, and GPT-4 Turbo supports 128K tokens.
GPT-3.5's context window is 4K tokens (approximately 3,000 English words).
早期GPT-3的上下文窗口仅有4K Token，而现代模型已扩展至128K甚至更长
Timeline (last 90 days)
Legora's initial idea was born during the period when Google's BERT model was openly accessible, but the early use cases were too difficult and it only worked once GPT-3.5 appeared.
Junestrand calls GPT-3.5 'the internet moment of our generation' and says that without OpenAI, Legora might not exist at all.
SkillBot is a skill recommendation chatbot that combines a fine-tuned GPT-3.5 model, retrieval-augmented generation (RAG) technology, and a knowledge graph containing about 2.3 million RDF triples.
Over the past two years the cost of running GDP-3.5-level capability models has fallen by about 280 times; according to Stanford HAI's AI Index data, a task that cost 1 dollar in late 2022 now costs less than one cent.
不同模型（如 GPT-4、GPT-3.5、text-embedding-ada-002）使用的是不同的 tokenizer，同一段文本在不同模型下的 token 数量可能不同。
Experiments based on early GPT-3.5 revealed the existence of "critical pivot tokens"—positions at which the model's answer uncertainty suddenly collapses and locks onto a particular result.
The capability boundary of the local 27B model is roughly equivalent to the level between early GPT-3.5 and GPT-4.
GPT-3.5 exhibits notable volatility in robustness tests, performing comparably to other models in some scenarios while lagging significantly behind in others.
After Meta released the Llama series in 2023, developers for the first time could run a model approaching GPT-3.5 levels entirely on a local machine or even a consumer-grade GPU.
AutoGPT's success relies heavily on the powerful capabilities of GPT-4; when using GPT-3.5 or open-source models, issues such as infinite loops and goal deviation frequently occur.
10 more timeline events
All Facts (20)
The context window in the GPT-3 era was only 2,048 tokens, whereas Google Gemini already supports millions of tokens, Claude supports 200K tokens, and GPT-4 Turbo supports 128K tokens.
90%VerifiedGPT-3.5's context window is 4K tokens (approximately 3,000 English words).
90%Verified早期GPT-3的上下文窗口仅有4K Token，而现代模型已扩展至128K甚至更长
90%VerifiedThe context window of the early GPT-3 was only 4,096 tokens, approximately 3,000 English words.
90%VerifiedInstructGPT and ChatGPT are representative products of the RLHF pipeline.
80%VerifiedThe early GPT-3 had a context window of only 2,048 tokens.
80%VerifiedGPT-3's training data contains hundreds of billions of tokens of internet text.
80%VerifiedFunction Call is a capability introduced by OpenAI in June 2023 with the GPT-3.5/GPT-4 API update, allowing developers to define JSON Schema descriptions of functions in API requests.
80%VerifiedThe Function Calling capability was first formally introduced by OpenAI in June 2023 within the GPT-3.5/4 API.
80%VerifiedGPT-4 ranked in the top 10% on the US bar exam.
75%VerifiedGPT-3.5 has a vector dimension of 12,288.
75%VerifiedChatGPT is based on the GPT-3.5/GPT-4 large language model, trained using the Transformer architecture and RLHF technology.
70%VerifiedOpenAI has used a variant of BPE since GPT-2, and GPT-4's cl100k_base vocabulary contains approximately 100,000 tokens.
65%VerifiedOpenAI's model naming follows a traceable public release pattern, from GPT-3 (2020) and GPT-3.5 (2022) to GPT-4 (2023) and GPT-4o (2024).
65%VerifiedGPT-4采用混合专家（MoE）架构，估计总参数量达1.8万亿，每次推理实际激活参数约为总量的1/8
65%VerifiedGPT-3.5 was originally an intermediate version developed by OpenAI for training RLHF reward models, later released as text-davinci-003, and was only retroactively named GPT-3.5 after ChatGPT launched
65%VerifiedGPT-4 has reached or surpassed the human average on the bar exam and medical licensing examinations.
65%UnverifiedLLaMA是Meta开源的大语言模型系列，以较小的参数量实现了接近GPT-3.5的性能
85%Unverified早期GPT-3.5的上下文窗口仅为4K Token，而2025年的顶级模型已普遍支持100K甚至更长的上下文
85%UnverifiedLLaMA是Meta开源的大语言模型系列，以较小的参数量实现了接近GPT-3.5的性能
80%