BEIR
A standard evaluation benchmark in information retrieval used to assess the zero-shot generalization capability of text retrieval models across diverse retrieval tasks and domains.
Timeline (last 90 days)
The authors conduct experiments on the BEIR benchmark and the TREC Deep Learning tracks, using the open-weight instruction-tuned model Llama-3.1-8B-Instruct as the judge.
The unstructured retrieval field has widely recognized benchmarks such as MS MARCO and BEIR, whereas structured retrieval has long lacked a unified cross-domain evaluation set.
The industry commonly uses public benchmarks such as BEIR and MS MARCO to evaluate retrieval systems.
Recall@k衡量前k个检索结果中是否包含所有相关文档，nDCG@k考虑相关文档的排名位置并归一化使不同查询可比，两者是检索评估的黄金标准，BEIR基准即采用它们。
BEIR is a zero-shot retrieval evaluation suite covering 18 heterogeneous datasets, with test domains including biomedical, financial, legal, and others.
SAFE-QE was evaluated on two benchmark retrieval collections, TREC and BEIR.
The accuracy of retrieval models is typically evaluated on standard benchmarks such as BEIR and MTEB.
MTEB (Massive Text Embedding Benchmark) is currently the most widely cited text embedding evaluation leaderboard, covering tasks such as retrieval, classification, and clustering, and uses datasets including MS MARCO and BEIR.
The TREC series of evaluations focuses primarily on relevance ranking in document retrieval, while BEIR focuses on evaluating zero-shot cross-domain retrieval capability.
All Facts (9)
The authors conduct experiments on the BEIR benchmark and the TREC Deep Learning tracks, using the open-weight instruction-tuned model Llama-3.1-8B-Instruct as the judge.
50%UnverifiedThe unstructured retrieval field has widely recognized benchmarks such as MS MARCO and BEIR, whereas structured retrieval has long lacked a unified cross-domain evaluation set.
50%UnverifiedThe industry commonly uses public benchmarks such as BEIR and MS MARCO to evaluate retrieval systems.
50%UnverifiedRecall@k衡量前k个检索结果中是否包含所有相关文档，nDCG@k考虑相关文档的排名位置并归一化使不同查询可比，两者是检索评估的黄金标准，BEIR基准即采用它们。
50%UnverifiedBEIR is a zero-shot retrieval evaluation suite covering 18 heterogeneous datasets, with test domains including biomedical, financial, legal, and others.
50%UnverifiedSAFE-QE was evaluated on two benchmark retrieval collections, TREC and BEIR.
50%UnverifiedThe accuracy of retrieval models is typically evaluated on standard benchmarks such as BEIR and MTEB.
50%UnverifiedMTEB (Massive Text Embedding Benchmark) is currently the most widely cited text embedding evaluation leaderboard, covering tasks such as retrieval, classification, and clustering, and uses datasets including MS MARCO and BEIR.
50%UnverifiedThe TREC series of evaluations focuses primarily on relevance ranking in document retrieval, while BEIR focuses on evaluating zero-shot cross-domain retrieval capability.
50%