vLLM Singapore Meetup: Focus on High-Throughput Inference and Agent Infrastructure

The offline technical gathering for the large model inference framework vLLM is coming to Singapore. Scheduled for November 13, 1:00 PM to 5:30 PM at SQ Collective, the event brings together engineers from inferact, NVIDIA, Alibaba Cloud, and the vLLM-Omni community, with topics targeting the hardest engineering problems in the inference stack.

Why This Meetup Is Worth Your Attention

vLLM has become one of the de facto standards in open-source inference, and the community discussions around it often reflect real pain points in production environments. The Singapore stop doesn't stop at framework introductions — it breaks the agenda down into high-throughput prefill, reinforcement learning, the Blackwell architecture, the Vera Rubin platform, Agent infrastructure, and interactive world models.

Judging from the speaker lineup, the presence of NVIDIA engineers means hardware-software co-optimization will be a key focus, while Alibaba Cloud's participation brings a large-scale cloud deployment perspective. The involvement of the vLLM-Omni community signals that multimodal inference is becoming an extension of the vLLM ecosystem. The event is free but requires registration approval, hinting that on-site discussions may touch on plenty of unreleased engineering details.

What vLLM is: vLLM is an open-source large model inference and serving engine that originated at UC Berkeley. Its core innovation is PagedAttention — which borrows the idea of virtual memory paging from operating systems to manage KV Cache, dramatically reducing memory fragmentation and boosting concurrent throughput. On top of this, vLLM uses Continuous Batching to dynamically batch requests at the token level, avoiding the resource waste caused by waiting for the longest request. It now supports mainstream Hugging Face models, multiple quantization schemes, tensor parallelism and pipeline parallelism, and provides an OpenAI-compatible API serving interface. It is used by many cloud providers and enterprises for production deployment, making it one of the most widely referenced implementations in the open-source inference ecosystem.

High-Throughput Prefill and RL: Two Main Threads of the Inference Stack

Among the keywords listed in the agenda, high-throughput prefill directly corresponds to the most expensive computation stage in production. As long-context requests proliferate, how to squeeze maximum throughput out of the prefill stage without dragging down decode-stage latency is a problem every inference team must face. Every iteration the vLLM community makes in this area is quickly adopted by downstream frameworks.

Reinforcement learning being listed separately shows that the boundary between RL training and inference is blurring. More and more teams use vLLM to handle the rollout phase of RLHF or online RL, which imposes requirements on scheduling policies and memory management that differ from pure inference serving. Engineers on site will likely share tuning experience under such mixed workloads.

The difference between Prefill and Decode: Large model inference is typically divided into two stages. The Prefill stage processes the user's full prompt in one pass, generating KV Cache through parallel computation — a compute-bound operation. The Decode stage then autoregressively generates output token by token, processing only one token at a time — a memory-bound operation. The two have vastly different demands on hardware resources: prefill pursues high compute utilization and throughput, while decode is limited by memory bandwidth and KV Cache capacity. When long-context requests surge in production, prefill's computational overhead grows significantly, potentially even crowding out decode's scheduling resources. Therefore, "high-throughput prefill" optimization typically involves strategies such as chunked prefill (splitting long prompts into chunks and scheduling them mixed with decode), prefix caching reuse, and PD disaggregation (separate deployment of prefill/decode), with the goal of improving overall throughput while maintaining time-to-first-token (TTFT) latency.

Rollout in RLHF and online RL: In reinforcement learning from human feedback (RLHF) and broader online reinforcement learning (such as PPO, GRPO) training pipelines, the model must continuously generate samples (i.e., rollout), which are then scored by a reward model or rules for policy updates. In this "generate—score—update" loop, the generation phase is essentially a large-scale inference task, often requiring high concurrency, variable-length outputs, and frequent weight synchronization. Unlike traditional online serving, requests in the rollout phase are bursty and batch-oriented, insensitive to per-request latency but extremely sensitive to overall throughput. When using vLLM as a rollout engine, one must handle hot weight updates, KV Cache clearing, scheduling priorities, and coordination with training processes — a hot topic in the open-source community today.

Blackwell, Vera Rubin, and Agent Infrastructure

On the hardware side, both Blackwell and Vera Rubin appearing in the agenda reflects that inference engines need to adapt to next-generation computing platforms in advance. The co-design cycle between frameworks and chips is shortening, and teams that miss early adaptation often pay a price in performance.

Agent infrastructure is another explicitly highlighted direction. Agent workloads are characterized by short requests, high concurrency, and complex state management, differing markedly from the traffic model of traditional chat services. Public information on how vLLM serves such workloads is still relatively scarce, which is exactly where the value of on-site discussion lies.

The appearance of interactive world models carries a forward-looking note. It represents the possibility of inference engines expanding from text generation to real-time interactive scenarios, placing higher demands on both latency and throughput simultaneously.

What Blackwell and Vera Rubin are: Blackwell is NVIDIA's new-generation GPU architecture released in 2024 (B100/B200/GB200, etc.), which significantly improves FP4/FP8 low-precision compute, HBM3e memory bandwidth, and NVLink interconnect bandwidth compared to Hopper, and introduces inference-specific features. Vera Rubin is the codename of NVIDIA's announced successor platform, named after the combination of Vera CPU and Rubin GPU, expected to further strengthen high-bandwidth memory and rack-scale interconnect. For inference engines, new architectures often bring new quantization formats (such as FP4), new attention implementation paths, and different memory hierarchies. Frameworks need to adapt kernels, schedulers, and memory management in advance, otherwise it's hard to reach peak hardware performance. This is why frameworks like vLLM must closely follow hardware roadmaps.

Who Should Attend

If you're responsible for capacity planning, framework selection, or performance tuning for inference services, this event's content density will be quite high. Attendees are encouraged to bring their own serving questions and talk directly with the engineers building this tech stack. For developers following the open-source inference ecosystem, this is also a window into vLLM's roadmap.

The event takes place November 13, 13:00–17:30 SGT at SQ Collective. Attendance is free but requires registration approval.

One-line Summary

vLLM Singapore Meetup on November 13 focuses on hard engineering problems in the inference stack.

Paragraph Summary

The vLLM offline technical gathering will be held on November 13, 13:00–17:30 SGT at SQ Collective in Singapore, free but requiring registration approval. Participants include engineers from inferact, NVIDIA, Alibaba Cloud, and the vLLM-Omni community, with an agenda covering high-throughput prefill, reinforcement learning, the Blackwell architecture, the Vera Rubin platform, Agent infrastructure, and interactive world models. The article notes that high-throughput prefill corresponds to the most expensive computation stage in production, while RL reflects the increasingly blurred boundary between training and inference; the pairing of Blackwell and Vera Rubin shows that frameworks must adapt to new hardware in advance; Agent workloads' short requests, high concurrency, and complex state management differ markedly from traditional chat traffic models, and with limited public information, this is exactly where on-site discussion adds value. The event suits developers responsible for capacity planning, framework selection, and performance tuning who want to bring real serving questions.