GAIA
Timeline (last 90 days)
Table 1 of the paper shows that ARPO (a pure-text 8B agent without latent-space mode) scores 38.8 on GAIA and 32.0 on WebWalker, both slightly higher than GPT-5.6 Luna.
GAIA, WebWalker, HLE and DeepSearchQA evaluate different capabilities: GAIA targets tool calling for general AI assistants, WebWalker focuses on web navigation and multi-hop retrieval, HLE is a high-difficulty closed-book knowledge reasoning exam, and DeepSearchQA evaluates accuracy on multi-hop deep retrieval question answering.
The incremental gains from latent-space mode are: GAIA rises from 38.8 to 43.7, and WebWalker jumps from 32.0 to 42.5.
According to Table 1 of the paper, HybridAgent 8B scores 43.7 on the GAIA benchmark, surpassing GPT-5.6 Luna's 37.9.
The KnowGPO experiments cover three benchmarks: Xbench, BrowseComp-ZH and GAIA.
GAIA is a general assistant-style benchmark that emphasises combinations of multi-step reasoning and tool use; BrowseComp-ZH is a Chinese browsing benchmark focused on locating hard-to-retrieve facts in the open web; Xbench is a deep-search evaluation for professional domains (including finance and consulting).
Agent capability is usually measured through dimensions such as tool-calling accuracy, multi-step task completion rate, and instruction-following degree, with typical benchmarks including BFCL, τ-bench, and GAIA; Coding capability is measured through code generation and repair benchmarks (such as SWE-bench and LiveCodeBench).
Evaluating browser agents typically uses public benchmarks such as WebArena, WebVoyager, Mind2Web, and GAIA, measuring end-to-end success rate and step efficiency on multi-step tasks on real websites.
SWE-bench tests autonomous code repair capability, GAIA tests real-world multi-step task reasoning, and τ-bench tests tool-calling accuracy.
The research team evaluated DeReAct on two benchmarks: GAIA and SWE-bench Verified.
6 more timeline events
All Facts (20)
GAIA基准测试由Meta等机构联合推出，专门评估AI系统在真实世界任务中的通用助手能力
90%UnverifiedGAIA基准测试由Meta AI和HuggingFace等团队于2023年提出，包含400多个精心设计的问题
85%UnverifiedMagentic-One在多个公开基准测试（如GAIA、AssistantBench）上展现了较强的通用任务处理能力
80%UnverifiedManus在GAIA基准测试中取得了领先成绩
80%Unverified2023年人类在GAIA测试中的平均成功率约为90%，而当时最强的GPT-4在Level 1上仅勉强达到15%
75%UnverifiedGAIA (General AI Assistants benchmark) is designed with real-world problems that require multi-step tool calls to complete.
60%UnverifiedGAIA, WebWalker, HLE and DeepSearchQA evaluate different capabilities: GAIA targets tool calling for general AI assistants, WebWalker focuses on web navigation and multi-hop retrieval, HLE is a high-difficulty closed-book knowledge reasoning exam, and DeepSearchQA evaluates accuracy on multi-hop deep retrieval question answering.
50%UnverifiedAccording to Table 1 of the paper, HybridAgent 8B scores 43.7 on the GAIA benchmark, surpassing GPT-5.6 Luna's 37.9.
50%UnverifiedTable 1 of the paper shows that ARPO (a pure-text 8B agent without latent-space mode) scores 38.8 on GAIA and 32.0 on WebWalker, both slightly higher than GPT-5.6 Luna.
50%UnverifiedThe incremental gains from latent-space mode are: GAIA rises from 38.8 to 43.7, and WebWalker jumps from 32.0 to 42.5.
50%UnverifiedThe KnowGPO experiments cover three benchmarks: Xbench, BrowseComp-ZH and GAIA.
50%UnverifiedGAIA is a general assistant-style benchmark that emphasises combinations of multi-step reasoning and tool use; BrowseComp-ZH is a Chinese browsing benchmark focused on locating hard-to-retrieve facts in the open web; Xbench is a deep-search evaluation for professional domains (including finance and consulting).
50%UnverifiedAgent capability is usually measured through dimensions such as tool-calling accuracy, multi-step task completion rate, and instruction-following degree, with typical benchmarks including BFCL, τ-bench, and GAIA; Coding capability is measured through code generation and repair benchmarks (such as SWE-bench and LiveCodeBench).
50%UnverifiedEvaluating browser agents typically uses public benchmarks such as WebArena, WebVoyager, Mind2Web, and GAIA, measuring end-to-end success rate and step efficiency on multi-step tasks on real websites.
50%UnverifiedSWE-bench tests autonomous code repair capability, GAIA tests real-world multi-step task reasoning, and τ-bench tests tool-calling accuracy.
50%UnverifiedThe research team evaluated DeReAct on two benchmarks: GAIA and SWE-bench Verified.
50%UnverifiedBenchmarks such as SWE-bench and GAIA are specifically designed to measure a model's ability to complete multi-step, cross-tool-call tasks.
50%UnverifiedAREX-2 achieved the following scores on deep research tasks: BrowseComp 84.0, HLE 52.6, GAIA 92.2, and DeepSearchQA 93.8
50%UnverifiedThe mainstream benchmarks for measuring long-horizon task capability include SWE-bench and GAIA.
50%UnverifiedAgent evaluation benchmark suites include GAIA, AgentBench, and SWE-bench.
50%