Chunking Strategy
A set of technical approaches used in RAG pipelines and local indexing to split documents or code into retrievable units. Common strategies include fixed-length chunking, semantic-boundary chunking, and sliding-window chunking. The chosen granularity directly affects the trade-off between retrieval precision and contextual completeness.
Timeline (last 90 days)
The recall quality of semantic retrieval depends on the vectorization scheme and chunking strategy, and poor tuning instead introduces noise.
Two strategies may produce the same number of chunks, but the size variance differs enormously; high variance usually means some chunks are extremely small and some extremely large, which directly affects retrieval stability.
When recall@5 on adversarial questions is low, the correct action is to go back and check the chunking strategy; only after recall@5 meets the bar is it meaningful to invest in optimizing the embedding model or reranker.
分块大小没有万能值，256-512 token是常见起点，但技术文档可能需要按标题层级分块。
Common chunking methods include splitting by fixed character count, splitting by sentence/paragraph boundaries, and sliding splits with overlapping windows
向量检索的效果高度依赖于分块策略，常用方法包括固定长度分块、基于语义的分块以及递归分块
All Facts (6)
The recall quality of semantic retrieval depends on the vectorization scheme and chunking strategy, and poor tuning instead introduces noise.
50%UnverifiedTwo strategies may produce the same number of chunks, but the size variance differs enormously; high variance usually means some chunks are extremely small and some extremely large, which directly affects retrieval stability.
50%UnverifiedWhen recall@5 on adversarial questions is low, the correct action is to go back and check the chunking strategy; only after recall@5 meets the bar is it meaningful to invest in optimizing the embedding model or reranker.
50%Unverified分块大小没有万能值，256-512 token是常见起点，但技术文档可能需要按标题层级分块。
50%UnverifiedCommon chunking methods include splitting by fixed character count, splitting by sentence/paragraph boundaries, and sliding splits with overlapping windows
50%Unverified向量检索的效果高度依赖于分块策略，常用方法包括固定长度分块、基于语义的分块以及递归分块
50%