text-embedding-3-small

OpenAI's text-embedding-3-small model converts text into a floating-point array of 1,536 dimensions

Tested through a third-party relay gateway, text-embedding-3-small had an average embedding generation time of 127ms, slower than the database write.

HotKV's caching mechanism first does exact matching (SHA-256 hashing of the normalized question, model identifier, and system prompt version), then semantic matching (using OpenAI text-embedding-3-small to generate vectors, with a hit when cosine similarity exceeds the threshold), and only when neither hits does it call the model and write back to the cache.

Adding an embedding-based semantic cache in front of the LLM, with a cosine similarity threshold set at 0.92 and returning cached results directly when the model is consistent, means about 30% of repeated user queries hit the cache, reducing overall spending by another about 20%.

OpenAI text-embedding-3-small is priced at $0.02 per million tokens; embedding an initial corpus of 500 million tokens costs $10 as a one-time cost, monthly refresh is about $1, and the embedding cost per query is about $0.000001.

Embeddings from different models are not comparable, so store the model name and dimensions alongside each chunk; 1536 dimensions corresponds to the default output length of OpenAI text-embedding-3-small.

Embedding models such as OpenAI's text-embedding-3-small are billed by usage volume, and their usage cost is lower than that of more expensive models, but there is a clear gap in retrieval performance.

In AI FAQ scenarios, RAG is usually divided into two phases, offline and online: the offline phase splits FAQ documents into chunks, converts them into vectors via an embedding model, and stores them in a vector database; the online phase vectorizes the user question, recalls the top-k passages using cosine similarity, and then hands them to the LLM to synthesize an answer

Mainstream embedding models such as OpenAI's text-embedding-3-small and lightweight versions of the BGE series typically have parameter counts in the range of 100M–500M.

Embedding models (such as text-embedding-3-small/large and text-embedding-004) and the LLM responsible for conversation are two independent models.

Embedding models (such as BGE and text-embedding-3-small) are much smaller than LLMs and can typically run on CPUs or low-spec GPUs, with inference latency in the millisecond range.

Locally runnable sentence-transformers models (such as all-MiniLM-L6-v2) can work entirely offline, whereas cloud-based models such as OpenAI's text-embedding-3-small offer higher accuracy but require an internet connection.

In a RAG pipeline, documents are converted into high-dimensional vector representations via an embedding model, typically ranging from 768 to 3072 dimensions.

OpenAI 的 text-embedding-3-small 输出 1536 维向量，嵌入向量维度通常在 384 到 1536 之间

text-embedding-3-small的向量维度为1536维，BGE-small为384维