Liquid AI Launches RIPL Inference Engine: LFM Latency Drops Below 50 Milliseconds

Liquid AI has introduced RIPL, a real-time inference engine for the LFM series of models. Official data shows that on a single B200, LFM2.5-1.2B on RIPL achieves p95 latency that is 14.4 to 28.4 times lower than Qwen3.5-0.8B on vLLM. This multiple-point gap points to a long-overlooked problem: the efficiency ceiling of general-purpose inference frameworks may be becoming the real bottleneck for deploying small models.

What Problem Does RIPL Solve?

For the past two years, the main thrust of large-model inference optimization has been throughput—how to pack more requests onto a single card and drive per-token cost to a minimum. Technologies like vLLM's PagedAttention and continuous batching evolved around this goal. But real-time interactive scenarios demand something different: the time a user waits for the first token, the response interval in voice conversations, and the round-trip latency of Agent tool calls are all measured by p95 or even p99 tail latency, not average throughput.

Liquid AI's choice to build a dedicated engine in this direction is logically coherent. The LFM series itself consists of small-parameter models designed for edge and on-device use; the 1.2B size means it cannot win by piling on compute—it can only win through extreme compression of the inference path. The name RIPL hints at a "real-time inference pipeline" positioning, contrasting with the general-purpose framework route of "general first, optimize later."

p95 Latency and Tail Latency: In performance testing, p95 latency refers to the latency ceiling reached by 95% of requests—only 5% of requests are slower. Compared with average latency, p95/p99 better reflects the stability of user experience—in real-time interaction, users are extremely sensitive to occasional stutters, and tail latency directly determines whether a product feels "responsive." This is also the fundamental reason RIPL and vLLM have diverged in optimization goals: vLLM pursues maximum throughput, while real-time scenarios must prioritize compressing the long tail.

KV Cache Management: KV Cache is the mechanism that caches historical Key/Value tensors during autoregressive generation, avoiding repeated computation at each step. Its memory layout and scheduling strategy have a huge impact on latency: general-purpose frameworks often adopt paged management (such as PagedAttention) to support dynamic batching, while low-latency engines may choose more compact pre-allocation or in-place update strategies to reduce memory fragmentation and scheduling overhead. This explains why a dedicated engine can achieve multiple-fold advantages on small models.

Where Does the 14x Gap Come From?

First, we need to see the comparison structure clearly: the two models differ in scale (1.2B vs 0.8B), the frameworks differ (RIPL vs vLLM), and the hardware is the same (a single B200). This means the 14.4–28.4x p95 latency advantage cannot be simply attributed to the engine itself—model architecture, quantization strategy, and KV Cache management could all contribute to the difference.

Even so, this magnitude still deserves attention. In the sub-50-millisecond latency range, every 10 milliseconds reduction corresponds to a different product form: whether a voice assistant can achieve natural interruption, whether real-time translation can keep up with the pace of speech, whether an on-device Agent can complete tool calls before the user notices. General-purpose frameworks in such scenarios often need to sacrifice batching efficiency to gain latency, while a dedicated engine can design scheduling strategies from scratch for single-request low latency.

A reasonable speculation is that RIPL has made cuts tailored to the LFM structure in batch size, operator fusion, and memory layout. Small models have low computational density, so the scheduling overhead of general-purpose frameworks gets amplified—this is where dedicated engines easily achieve multiple-fold advantages.

Operator Fusion and Batch Size Trimming: Operator fusion refers to merging multiple fine-grained computations (such as matrix multiplication, activation, and normalization) into a single kernel, reducing kernel launch and memory read/write overhead. Small models have low computational density, individual operators take less time, and scheduling overhead is amplified, making fusion gains more significant. At the same time, real-time engines typically push batch size down to 1 or a minimal value to avoid queuing latency, but must redesign memory layout to maintain utilization—this is exactly the entry point for dedicated engines to tailor to the LFM structure.

Framework Divergence in Small-Model Inference

This launch reveals a broader signal: inference frameworks are moving from "one vLLM rules them all" toward divergence by scenario. Cloud high-throughput, edge low-latency, real-time interaction, and offline batch processing have technical constraints that differ too much, and the abstraction layers of general-purpose frameworks become a burden instead.

Liquid AI's path has similarities with Groq's LPU and Cerebras's wafer-scale approach—both push specific workloads to the limit through software-hardware co-design. The difference is that RIPL is a pure software engine that can run on existing GPUs, with a much lower deployment barrier. For teams wanting to run real-time small models on B200 or similar hardware, this provides an option beyond vLLM.

Of course, the cost of a dedicated engine is ecosystem lock-in. RIPL currently only serves the LFM series, and the range of model choices is far smaller than the open model library supported by vLLM. Whether it is worth accepting this binding for the latency advantage depends on how sensitive a specific product is to response time.

Software-Hardware Co-design and Pure Software Engines: Groq's LPU and Cerebras's wafer-scale engine are both integrated software-hardware solutions that push specific workloads to the limit through custom chips, but they have high deployment barriers and poor flexibility. RIPL is a pure software engine that can run on existing GPUs, trading algorithm and scheduling optimization for low latency, at the cost of only adapting to the LFM series. This stratification means inference optimization is shifting from "hardware-only" to "scenario-defined software," and teams can choose different paths based on latency sensitivity and hardware conditions.

Who Should Pay Attention

Real-time voice interaction, on-device Agents, and online services requiring strict SLAs are the most direct target scenarios for RIPL. If a product requires p95 latency within 100 milliseconds, the tuning space of general-purpose frameworks is often already exhausted, and the multiple-fold advantage of a dedicated engine becomes practically meaningful. Conversely, if the business centers on throughput and cost, the maturity and model coverage of the vLLM ecosystem remain a safer choice.

Liquid AI has pushed the "dedicated inference engine" category back into the spotlight with a set of comparison numbers. What is worth watching next: whether RIPL will open support for other models, and whether general-purpose frameworks like vLLM will follow up with targeted optimizations for tail latency.

One-line Summary

Liquid AI launches the RIPL real-time inference engine, with LFM models achieving over 14x lower p95 latency than vLLM, signaling that inference frameworks are diverging by scenario.

Paragraph Summary

Liquid AI has launched RIPL, a real-time inference engine designed specifically for the LFM series. On a single B200, LFM2.5-1.2B achieves p95 latency 14.4 to 28.4 times lower than Qwen3.5-0.8B on vLLM. The article notes that this comparison involves differences in both model scale and framework, so the multiple-fold advantage cannot be entirely attributed to the engine itself—but the magnitude still deserves attention. General-purpose inference frameworks have long optimized primarily for throughput, while real-time interactive scenarios focus more on p95/p99 tail latency. RIPL gains its advantage through scheduling cuts and memory optimization tailored to the LFM structure. This reflects inference frameworks moving from "one vLLM rules them all" toward divergence by scenario, with dedicated engines trading ecosystem lock-in for latency advantages. Target scenarios are applications sensitive to tail latency, such as real-time voice and on-device Agents.