LEANN
An open-source vector index project from the UC Berkeley team that combines pruned graphs with recomputation of embeddings at query time, substantially reducing the storage and memory footprint of RAG indexes.
Timeline (last 90 days)
A UC Berkeley team released the open-source project LEANN, which compresses vector indexes through a "pruned graph + recomputation at query time" approach.
LEANN compressed a 60-million-chunk (60M-chunk) RAG index from 201 GB to 6 GB.
According to a report by MarkTechPost, LEANN achieves accuracy comparable to HNSW at 90% recall, with memory usage of 4 GB versus 188 GB.
LEANN's query time rises to the order of seconds, so it is not suitable for high-concurrency, low-latency online services.
LEANN achieves order-of-magnitude compression in the storage dimension, making it possible to deploy larger-scale knowledge bases on a single machine, and the burden of cold starts and backup migration also drops significantly.
Recomputing embeddings means that queries require access to the original text or the embedding model, which introduces additional I/O and compute overhead.
LEANN is better suited to internal retrieval where the knowledge base is large but query frequency is low and latency is not sensitive, to resource-constrained edge or single-machine deployments, and to experimental environments that require frequent index rebuilds.
LEANN's limitations include: second-level retrieval is not suited to the stringent latency requirements of interactive question answering; recomputing embeddings depends on the availability of the original data; and the recall performance of the pruned graph needs to be validated for different data distributions.
As the inference cost of embedding models continues to fall, the cost-effectiveness of "recompute at query time" will further improve, and this type of design is expected to secure a place in RAG infrastructure.
LEANN is an open-source project, and developers can directly evaluate its code and paper and validate recall and latency performance on their own datasets.
All Facts (10)
LEANN is an open-source project, and developers can directly evaluate its code and paper and validate recall and latency performance on their own datasets.
50%UnverifiedLEANN compressed a 60-million-chunk (60M-chunk) RAG index from 201 GB to 6 GB.
50%UnverifiedAccording to a report by MarkTechPost, LEANN achieves accuracy comparable to HNSW at 90% recall, with memory usage of 4 GB versus 188 GB.
50%UnverifiedLEANN's query time rises to the order of seconds, so it is not suitable for high-concurrency, low-latency online services.
50%UnverifiedLEANN achieves order-of-magnitude compression in the storage dimension, making it possible to deploy larger-scale knowledge bases on a single machine, and the burden of cold starts and backup migration also drops significantly.
50%UnverifiedRecomputing embeddings means that queries require access to the original text or the embedding model, which introduces additional I/O and compute overhead.
50%UnverifiedLEANN is better suited to internal retrieval where the knowledge base is large but query frequency is low and latency is not sensitive, to resource-constrained edge or single-machine deployments, and to experimental environments that require frequent index rebuilds.
50%UnverifiedLEANN's limitations include: second-level retrieval is not suited to the stringent latency requirements of interactive question answering; recomputing embeddings depends on the availability of the original data; and the recall performance of the pruned graph needs to be validated for different data distributions.
50%UnverifiedA UC Berkeley team released the open-source project LEANN, which compresses vector indexes through a "pruned graph + recomputation at query time" approach.
50%UnverifiedAs the inference cost of embedding models continues to fall, the cost-effectiveness of "recompute at query time" will further improve, and this type of design is expected to secure a place in RAG infrastructure.
50%