vLLM Achieves Best Result on AgentX: Breaking DeepSeek Inference Record

Introduction

Recently, vLLM achieved the best-ever result for DeepSeek^Ad-V4.1-Flash on @SemiAnalysis_'s AgentX benchmark using real agentic workloads, not only matching DeepInfer's performance but also delivering impressive revenue efficiency numbers. According to news on Twitter, the test used DeepSeek's off-peak pricing, and vLLM achieved $38.96 revenue per GPU hour, with total throughput reaching 954K tokens/s/GPU (including cache, input, and output), and P90 per-user throughput of 196 tokens/s.

These figures come from the vLLM team's real-world testing on AgentX. They state this is the best result for DeepSeek-V4.1-Flash on this benchmark, backed by a series of recent optimizations in vLLM. This article breaks down the test from three angles: performance metrics, the optimization story, and practical impact.

Interpreting the Performance Metrics

Balancing Throughput and Latency

A total throughput of 954K tokens/s/GPU means a single GPU can process nearly one million tokens per second, including cache hits, input processing, and output generation. For agentic workloads, this throughput capacity directly determines how many concurrent users can be served. Meanwhile, the P90 per-user latency of 196 tokens/s shows that good response speed can still be maintained under high concurrency.

Tokens/s/GPU and P90 Latency: In large model inference, throughput typically refers to the total number of tokens processed per GPU per second (including input, output, and cache hits), while per-user latency focuses on the single-request experience. P90 means 90% of user requests are at or above that rate — i.e., a reference point for tail latency. For agentic workloads, because multi-turn interactions and long contexts frequently trigger KV cache reuse, the trade-off between throughput and latency is especially critical — the larger the batch size, the higher the total throughput, but per-request queuing latency may rise. vLLM mitigates this tension through PagedAttention and continuous batching, maintaining acceptable P90 metrics under high concurrency.

Breakthrough in Revenue Efficiency

The $38.96/GPU-hour figure is calculated based on DeepSeek's off-peak pricing, providing a reference for the economics of inference services. In agentic scenarios, tasks typically require multi-turn interactions and consume large amounts of tokens, so high throughput translates directly into higher revenue potential.

Off-peak Pricing and Revenue Efficiency: DeepSeek's off-peak pricing refers to an API billing model that offers discounted prices during non-peak hours, typically used to steer load away from peak times. Revenue efficiency ($/GPU-hour) is not an official metric; rather, it converts the actual number of tokens processed into revenue at that pricing and divides by GPU occupancy time, measuring the monetization capacity per unit of compute. This metric is valuable for inference service providers: it simultaneously reflects throughput, cache hit rate, and pricing strategy. High throughput combined with low-price periods can amplify per-GPU earnings, but note that it depends on specific pricing assumptions, and comparisons should use a consistent pricing basis.

The Story Behind the Optimizations

The vLLM team shared the complete optimization process in their blog. Although details aren't included here, it can be inferred from the results that the optimizations likely involve scheduling, memory management, and computation graphs. AgentX's agentic workloads are characterized by frequent interaction and abundant state, placing high demands on the inference engine's concurrency handling and caching mechanisms. vLLM's ability to achieve such results indicates a clear advantage in handling long contexts and multi-turn conversations.

Comparison with DeepInfer

The tweet mentions "matching DeepInfer's performance," implying that DeepInfer may have previously been the leader on this benchmark. This time, vLLM not only matched it but may have surpassed it on some metrics. However, specific comparison data was not provided in the tweet; refer to the full blog post.

Characteristics of Agentic Workloads: Unlike traditional single-turn Q&A, agentic workloads typically involve tool calls, multi-step reasoning, and state maintenance, manifesting as a mix of many short requests and long contexts, frequent cache reads and writes, and irregular arrival patterns. Such workloads place higher demands on the inference engine's scheduler: it needs to quickly recycle and reuse KV caches, support prefix caching to reduce redundant computation, and maintain session state across multi-turn conversations. vLLM's automatic prefix caching and paged KV management are precisely key optimization directions for such scenarios, and an important background for its high throughput on AgentX.

Industry Impact

These test results offer several insights for the AI inference field. First, open-source inference engines like vLLM are rapidly closing the gap with specialized solutions, even surpassing them in specific scenarios. Second, optimizing agentic workloads requires targeted design, and general-purpose engines can also achieve good results through continuous iteration. Finally, quantified revenue efficiency metrics provide service providers with a basis for cost-benefit evaluation.

Conclusion

vLLM's performance on AgentX demonstrates the potential of open-source inference engines and offers a new option for deploying DeepSeek-V4.1-Flash. As optimization efforts continue, we can expect to see further performance improvements in more real-world scenarios.

One-line Summary

vLLM sets a new best result for DeepSeek-V4.1-Flash on AgentX with real agentic workloads, excelling in both throughput and revenue efficiency.

Paragraph Summary

This article interprets vLLM's results testing DeepSeek-V4.1-Flash on the SemiAnalysis AgentX benchmark using real agentic workloads: total throughput of 954K tokens/s/GPU (including cache, input, and output), P90 per-user throughput of 196 tokens/s, and revenue efficiency of $38.96 per GPU hour based on DeepSeek off-peak pricing. The article covers the balance between throughput and latency, the breakthrough in revenue efficiency, the scheduling and caching mechanisms behind the optimizations, the comparison with DeepInfer, and the industry implications for the competitiveness of open-source inference engines. It notes that vLLM has matched DeepInfer, demonstrating the continuous iteration capability of general-purpose open-source engines in agentic scenarios, and points readers to the vLLM blog for optimization details and reproduction methods.