Unverified50% confidenceFactExact time
The o200k_base tokenizer is the BPE tokenizer used by the OpenAI series of models; tokenizer count differences across models for JSON and code-like text typically reach tens of percentage points.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOpenAI系列模型采用BPE（字节对编码）算法进行分词，起源于数据压缩领域67% similarVerifiedOpenAI uses the BPE (Byte Pair Encoding) algorithm for token encoding.66% similarUnverifiedModel APIs such as OpenAI's and Anthropic's accept only a subset of JSON Schema keywords, and the remaining constraints are either ignored or downgraded to natural language descriptions.64% similarUnverifiedMainstream large model APIs (such as OpenAI and Anthropic) charge based on the number of input tokens, and every character in a tool definition is a billable unit.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1070897API
curl https://kongchang.com/api/v1/knowledge/claims/1070897MCP
get_claim(id=1070897)