BFCL
BFCL (Berkeley Function Calling Leaderboard) is a benchmark released by the University of California, Berkeley, designed to evaluate large language models on function calling and tool use. The benchmark covers multiple programming languages and diverse function-calling scenarios—including single-function, parallel, and nested calls—and systematically assesses instruction comprehension, parameter extraction, and formatted output through a standardized test set.
Core Facts
Timeline (last 90 days)
Common benchmarks for evaluating agent capabilities include tool-calling benchmarks such as BFCL and interactive-environment benchmarks such as WebArena and SWE-bench.
BFCL is a commonly used benchmark for evaluating a model's function calling capability, covering scenarios such as simple functions, multi-function selection, and parameter filling.
Under 2-bit quantization, the Simple category of BFCL may still maintain a high pass rate, while Parallel and parameter-filling tasks often degrade first.
BFCL is maintained by the Berkeley team, and its evaluation set is stratified by difficulty into categories including Simple, Multiple, Parallel, Parallel Multiple, and Relevance Detection.
Tīrtha Bench's core test contains 800 tool-calling tasks, taken from Berkeley's publicly released BFCL dataset, of which 400 are single calls, 200 are multi-tool selection, and 200 are tasks requiring parallel calls.
In addition to the 800 BFCL tasks, Tīrtha Bench adds 120 trap prompts specially written by the authors to induce models to make incorrect tool calls.
Mikhail Maslenik reproduced Tīrtha Bench's 800 requests on his own machine using a quantized copy of the same model, and scoring with Berkeley's official checker yielded 90.6%, while Tīrtha's own result was 91.0%, a difference of three out of the 800 tasks.
The reproduction process exposed that two entries in the public test set have expected answers that penalize reasonable replies, and that Tīrtha's scorer is a self-rewritten version of Berkeley's; the authors said they will re-score using the official checker.
Agent capability is usually measured through dimensions such as tool-calling accuracy, multi-step task completion rate, and instruction-following degree, with typical benchmarks including BFCL, τ-bench, and GAIA; Coding capability is measured through code generation and repair benchmarks (such as SWE-bench and LiveCodeBench).
On the BFCL tool-calling test, SmolLM3-3B scores 0.945, slightly above Gemma 4 E2B's 0.927, but the difference is not significant; Qwen3-4B (0.940), Qwen3-1.7B (0.935), Qwen2.5-1.5B (0.910), and Granite 3.3 2B (0.920) are all at the same level as Gemma.
5 more timeline events
All Facts (15)
BFCL (Berkeley Function Calling Leaderboard) is a tool-calling capability evaluation benchmark released by the University of California, Berkeley, with V4 being its multi-turn dialogue extension.
75%UnverifiedCommon benchmarks for evaluating agent capabilities include tool-calling benchmarks such as BFCL and interactive-environment benchmarks such as WebArena and SWE-bench.
50%UnverifiedBFCL is maintained by the Berkeley team, and its evaluation set is stratified by difficulty into categories including Simple, Multiple, Parallel, Parallel Multiple, and Relevance Detection.
50%UnverifiedBFCL is a commonly used benchmark for evaluating a model's function calling capability, covering scenarios such as simple functions, multi-function selection, and parameter filling.
50%UnverifiedThe reproduction process exposed that two entries in the public test set have expected answers that penalize reasonable replies, and that Tīrtha's scorer is a self-rewritten version of Berkeley's; the authors said they will re-score using the official checker.
50%UnverifiedTīrtha Bench's core test contains 800 tool-calling tasks, taken from Berkeley's publicly released BFCL dataset, of which 400 are single calls, 200 are multi-tool selection, and 200 are tasks requiring parallel calls.
50%UnverifiedIn addition to the 800 BFCL tasks, Tīrtha Bench adds 120 trap prompts specially written by the authors to induce models to make incorrect tool calls.
50%UnverifiedAgent capability is usually measured through dimensions such as tool-calling accuracy, multi-step task completion rate, and instruction-following degree, with typical benchmarks including BFCL, τ-bench, and GAIA; Coding capability is measured through code generation and repair benchmarks (such as SWE-bench and LiveCodeBench).
50%UnverifiedOn the BFCL tool-calling test, SmolLM3-3B scores 0.945, slightly above Gemma 4 E2B's 0.927, but the difference is not significant; Qwen3-4B (0.940), Qwen3-1.7B (0.935), Qwen2.5-1.5B (0.910), and Granite 3.3 2B (0.920) are all at the same level as Gemma.
50%UnverifiedBFCL is a specialized benchmark released by the University of California, Berkeley, that evaluates the accuracy of large language models in structured function-calling scenarios.
50%UnverifiedIn the BFCL official 100-item parallel tool-calling test, Saluki scored 42, surpassing the original model's 35, representing approximately a 20% improvement in success rate.
50%UnverifiedIn the tool-calling benchmark BFCL, Qwen3 scored 83%.
50%UnverifiedIn benchmark tests, 2.0 shows notable improvements over the base model on IFBench (37.3→43.7), When2Call (48→58), and BFCL irrelevance (60→78).
50%UnverifiedUnder 2-bit quantization, the Simple category of BFCL may still maintain a high pass rate, while Parallel and parameter-filling tasks often degrade first.
50%UnverifiedMikhail Maslenik reproduced Tīrtha Bench's 800 requests on his own machine using a quantized copy of the same model, and scoring with Berkeley's official checker yielded 90.6%, while Tīrtha's own result was 91.0%, a difference of three out of the 800 tasks.
50%