GPT-5.4
A language model in OpenAI's GPT series with computer-use and visual understanding capabilities, applicable to Computer-Use Agent tasks.
Timeline (last 90 days)
Against GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5 and GPT-6-Astra, the adversarially trained SEMA-Auto achieves an average of 59.6% ASR@1 on HarmBench.
For GPT-5.4, more frequent correct simulation masked an evidence-use loss of about ten points, making the net change indistinguishable from zero.
The developer's Agent was connected to Slack, Notion, HubSpot and internal APIs; GPT-5.4 frequently called the wrong functions, and after switching to Claude Opus the problem persisted but improved slightly.
By removing half the tools and tightening the schema definitions, a developer noticeably improved the Agent tool-calling accuracy of GPT-5.4 and Claude Opus, while increasing prompt length did not improve the behavior.
The September 18 changelog set October 19 as the removal date for another six models, including GPT-5.5, GPT-5.4, and Grok 4.5; the interval from October 19 to November 16 is only 28 days.
Order effect: swapping the order of two answers, GPT-5.4 changed its verdict 11.4% of the time on one benchmark and 10.5% on another; Gemini 3.1 Pro and Claude Opus 4.6 were between 2.0% and 3.8%.
The benchmark covers 9 models from 6 labs, including Qwen3 235B, Claude Haiku 4.5, Claude Opus 5.5, Gemini series, GPT-5.4 series, GLM-5, and Grok 4.20, of which 4 are reasoning models.
After restoring flushing, the Claude candidate achieved a completion speedup of 2.75× and a time-to-first-byte speedup of 2.80×, while Gemini achieved a completion speedup of 3.44× and a time-to-first-byte speedup of 3.54×, and GPT-5.4 achieved a completion speedup of 2.16× and a time-to-first-byte speedup of 2.19×.
GPT's first candidate used all six edits, precomputed rotary position encodings, and removed OpenMP pragmas, but the build had no OpenMP flags to begin with, rendering the pragmas ineffective, resulting in a final improvement of only 1.0013×.
Across nine revised runs, the median completion speedup was 5.35× for Gemini, 3.69× for Claude, and 1.00× for GPT-5.4, with all three passing automated checks.
14 more timeline events
All Facts (20)
GPT-5.4及更早版本在提取文档中的数字时频繁出错，包括小数点位置偏移、千分位分隔符误读、负号遗漏等问题
80%UnverifiedAgainst GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5 and GPT-6-Astra, the adversarially trained SEMA-Auto achieves an average of 59.6% ASR@1 on HarmBench.
50%UnverifiedFor GPT-5.4, more frequent correct simulation masked an evidence-use loss of about ten points, making the net change indistinguishable from zero.
50%UnverifiedThe developer's Agent was connected to Slack, Notion, HubSpot and internal APIs; GPT-5.4 frequently called the wrong functions, and after switching to Claude Opus the problem persisted but improved slightly.
50%UnverifiedOrder effect: swapping the order of two answers, GPT-5.4 changed its verdict 11.4% of the time on one benchmark and 10.5% on another; Gemini 3.1 Pro and Claude Opus 4.6 were between 2.0% and 3.8%.
50%UnverifiedThe benchmark covers 9 models from 6 labs, including Qwen3 235B, Claude Haiku 4.5, Claude Opus 5.5, Gemini series, GPT-5.4 series, GLM-5, and Grok 4.20, of which 4 are reasoning models.
50%UnverifiedGPT's first candidate used all six edits, precomputed rotary position encodings, and removed OpenMP pragmas, but the build had no OpenMP flags to begin with, rendering the pragmas ineffective, resulting in a final improvement of only 1.0013×.
50%UnverifiedThe revised experiments use Gemini 3.7 Flash, Claude Sonnet 5.5, and GPT-5.4, with each model independently performing three optimization attempts, and prompts using temperature 0 and seed 0.
50%UnverifiedAfter restoring flushing, the Claude candidate achieved a completion speedup of 2.75× and a time-to-first-byte speedup of 2.80×, while Gemini achieved a completion speedup of 3.44× and a time-to-first-byte speedup of 3.54×, and GPT-5.4 achieved a completion speedup of 2.16× and a time-to-first-byte speedup of 2.19×.
50%UnverifiedAcross nine revised runs, the median completion speedup was 5.35× for Gemini, 3.69× for Claude, and 1.00× for GPT-5.4, with all three passing automated checks.
50%UnverifiedThe median confidence difference across models when fabricating totals is enormous: Gemini 3 Flash and Flash-Lite at 95, GPT-5.4 nano at 62, and Claude Sonnet 5 at only 20.
50%UnverifiedGPT-5.4 was reversed by a "trusted source" on 44.7% of questions, Grok-4.20 had a reversal rate as high as 87.5%, while Gemini-3.1-Pro was nearly unaffected at just 0.6%.
50%UnverifiedIn CAD World's 200-task evaluation, the best-performing GPT-5.4 achieved a success rate of only 17.5%.
50%UnverifiedGPT-5.4 achieved a completion rate of 62.5% on CAD World, with an average of 104.6 action steps and an average of 46.9 steps for successfully completed tasks.
50%UnverifiedReaders should independently verify whether "GPT-5.4" and "Claude Opus 4.6" are model versions officially released by their respective vendors, and avoid treating unofficial or speculative names as established facts.
50%UnverifiedThe task success rates of three frontier models are very close, all falling between 84% and 89%.
50%UnverifiedClaude Opus 4.6 produced 30 times more errors than GPT-5.4, averaging 2.5 errors per trajectory vs. 0.08, with the difference being statistically significant (p < 0.0001, Cliff's δ = 0.613).
50%UnverifiedParticipants in the experiment included a locally run 4-bit quantized Gemma 26B (built on MLX), Claude Opus 4.8 accessed via API, and GPT-5.4.
50%UnverifiedGPT-5.4 has an accuracy of 50.5%, while its paired always-predict-up baseline is 51.6%.
50%UnverifiedQwen2.5-72B approaches GPT-4 level performance on benchmarks such as MMLU and HumanEval.
50%