TriviaQA
A large-scale reading comprehension and question-answering benchmark dataset containing a substantial number of knowledge-based question-answer pairs collected from sources including Wikipedia. It is widely used to evaluate the knowledge question-answering capabilities of language models.
Timeline (last 90 days)
On reasoning tasks such as MMLU-Pro and TriviaQA, U-Space's single-forward-pass scorer consistently ranks correct answers ahead of entropy methods or ensemble methods.
By cloning https://github.com/s2labres/U-Space and running ./run_smoke.sh, you can run the end-to-end pipeline for Qwen3.5-4B on 10 TriviaQA questions (single GPU).
On TriviaQA, the error rate of answered questions (under the exact match scoring criterion) decreased from 23.0% to 14.5%.
NQ-Open is derived from real Google search queries; the questions are more natural and more ambiguous, and overall accuracy is generally lower than on TriviaQA.
The findings on TriviaQA were validated through an audit partially involving human review, ruling out artifacts caused by the scoring criterion.
On the TriviaQA dataset, the learned answer selector improved average coverage from 20.2% to 30.9%.
The research methodology selected questions from TriviaQA that models had already answered correctly, appended incorrect answers, and presented them in two ways—a "trusted source version" and a "user expert version"—with the only variable being the identity of the speaker.
All Facts (6)
On reasoning tasks such as MMLU-Pro and TriviaQA, U-Space's single-forward-pass scorer consistently ranks correct answers ahead of entropy methods or ensemble methods.
50%UnverifiedOn the TriviaQA dataset, the learned answer selector improved average coverage from 20.2% to 30.9%.
50%UnverifiedOn TriviaQA, the error rate of answered questions (under the exact match scoring criterion) decreased from 23.0% to 14.5%.
50%UnverifiedThe findings on TriviaQA were validated through an audit partially involving human review, ruling out artifacts caused by the scoring criterion.
50%UnverifiedNQ-Open is derived from real Google search queries; the questions are more natural and more ambiguous, and overall accuracy is generally lower than on TriviaQA.
50%UnverifiedThe research methodology selected questions from TriviaQA that models had already answered correctly, appended incorrect answers, and presented them in two ways—a "trusted source version" and a "user expert version"—with the only variable being the identity of the speaker.
50%