Unverified50% confidenceBenchmarkExact time
On the 205 BigQuery questions of Spider 2.0-lite, schemagate's accuracy was only 45.9%; the Snowflake portion has not yet been run.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/24/2026
First Seen
Valid until: 11/7/2026
Sources
Related Entities
Related Claims
UnverifiedOn Spider, schemagate and the paper's best configuration both achieved an AC-F1 of 93.0, but schemagate's wrong-answer rate was 5.1%, slightly higher than the paper's 4.6%.65% similarUnverifiedOn the Spider dataset, schemagate tied with the same model (Claude Sonnet 4.5) at 93.0; the paper's best result on Spider came from DeepSeek V3.2-Reasoning at 95.9.64% similarUnverifiedIn agentgateway's testing, nemotron-3.5-lightning (32.9B sparse MoE, 1M context) had a valid answer rate of 45.0% (27/60), an accuracy rate among answered questions of 100% (27/27), and an overall accuracy rate of 45.0% (27/60).53% similarUnverifiedThe Mindscend team's 26.2M-parameter GECToR-style edit annotator v1.5 achieves 0.799 on the F0.5 metric, with a 2.4% false-edit rate and a 0.0% protected-span loss rate.52% similarUnverifiedOn the same pipeline and the same 135 Spider 2.0-lite SQLite questions, Sonnet got 71.9% while Opus got only 67.4%, but Opus performed better on BIRD.51% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1092533API
curl https://kongchang.com/api/v1/knowledge/claims/1092533MCP
get_claim(id=1092533)