#推理优化
87 related articles
vLLM Full-Stack Optimization in Practice: Handling the Serving Pressure of Real-World Agent Traffic
vLLM's full-stack optimization for real-world agent traffic covers architecture, framework, and runtime layers, benchmarked on SemiAnalysis's public AgentX workload.
After Day-0 Support: How the Community Achieved 3–4x Inference Speedup for MiniMax M3
AMD and EmbeddedLLM detail how they achieved 3–4x throughput gains for MiniMax M3 on Instinct MI355X using vLLM, with a "follow the bottleneck" optimization approach.
Tech FrontiersDeepSeek Open-Sources Inference Engine DeepInfer: The Systems Engineering Behind 155 Trillion Tokens
DeepSeek open-sources DeepInfer, its vLLM-derived Apache 2.0 inference engine processing 155 trillion tokens daily. A deep dive into MoE parallelism, DSpark speculative decoding, SWA caching, and CPU look-ahead scheduling — plus what's open and what isn't.
TutorialsDeepSeek V4 Flash MTP Speculative Decoding Real-World Test: A Guide to 20% Faster Local Inference
Real-world testing of DeepSeek V4 Flash with MTP speculative decoding: ~20% speedup for code generation, minimal gains for text. Covers memory overhead, accuracy differences, Q4 vs Q3 quantization, and full deployment tutorial.
Industry InsightsNVIDIA Dynamo Snapshot: A Snapshot Recovery Solution for GPU Inference Cold Start Problems
Deep dive into how NVIDIA Dynamo Snapshot reduces LLM inference cold start time from minutes to seconds via GPU state snapshot and recovery, covering Kubernetes integration and elastic inference.
Tech FrontiersGemini 3.5 Flash Released: How Google Balances AI Model Speed and Capability
Google releases Gemini 3.5 Flash, optimizing the balance between speed and capability. Analysis of Flash series evolution, comparisons with GPT-4o mini, and practical value for developers.
