L40S
A multi-purpose NVIDIA GPU for data centers, based on the Ada Lovelace architecture, suited to AI inference, graphics and video processing.
Core Facts
Timeline (last 90 days)
For individual developers or small-scale services dominated by low concurrency, the 3090 provides performance close to the L40S at about half the cost; for high-throughput online services, the L40S is faster at batch 128 and cheaper with the AWQ model.
At batch 1, the RTX 3090's inference throughput is about 1.03~1.09 times that of the L40S (based on Qwen2.5 1.5B/7B models, measured on vLLM 0.27.1).
At small batch sizes the bottleneck is memory bandwidth, while at large batch sizes the rising compute density makes compute the bottleneck, which explains the performance crossover between the 3090 and the L40S.
With the 7B AWQ model at batch 32 the 3090 already lags behind the L40S (0.71x), whereas the 7B fp16 is not overtaken until batch 128 (0.72x), showing that the quantized model makes the compute bottleneck appear earlier.
At the prices the author actually paid ($0.414 per hour for the 3090, $0.826 per hour for the L40S) and counting only GPU busy time, at low batch sizes the 3090's cost per million output tokens is about half that of the L40S.
At batch 128 with the 7B AWQ model, the L40S's cost per million output tokens is $0.036, lower than the 3090's $0.040.
In the author's tests, the fluctuation across all repeated runs is no more than 0.5% on the 3090 and at worst 2.6% on the L40S, so the data is not noise.
In GPU Index's price data snapshot, the lowest price for H100 is $1.37/hr (Vast.ai, spot instance), for H200 is $2.62/hr (CoreWeave, spot), for A100 is $1.15/hr (Paperspace, three-year contract), for L40S is $0.79/hr (RunPod, community cloud), and for B200 is $4.26/hr (CoreWeave, spot).
Fodder Watch requires a 48 GB PCIe card (L40S class), with a self-purchased node three-year cost of approximately $15,200 (range $14,600 to $15,800).
The L40G generally refers to the L40S or a variant targeting specific scenarios, further enhancing the balance between AI inference and training capabilities.
3 more timeline events
All Facts (12)
The RTX 4060's internal VRAM bandwidth is approximately 288 GB/s.
65%UnverifiedFor individual developers or small-scale services dominated by low concurrency, the 3090 provides performance close to the L40S at about half the cost; for high-throughput online services, the L40S is faster at batch 128 and cheaper with the AWQ model.
50%UnverifiedWith the 7B AWQ model at batch 32 the 3090 already lags behind the L40S (0.71x), whereas the 7B fp16 is not overtaken until batch 128 (0.72x), showing that the quantized model makes the compute bottleneck appear earlier.
50%UnverifiedAt the prices the author actually paid ($0.414 per hour for the 3090, $0.826 per hour for the L40S) and counting only GPU busy time, at low batch sizes the 3090's cost per million output tokens is about half that of the L40S.
50%UnverifiedAt batch 128 with the 7B AWQ model, the L40S's cost per million output tokens is $0.036, lower than the 3090's $0.040.
50%UnverifiedIn the author's tests, the fluctuation across all repeated runs is no more than 0.5% on the 3090 and at worst 2.6% on the L40S, so the data is not noise.
50%UnverifiedAt batch 1, the RTX 3090's inference throughput is about 1.03~1.09 times that of the L40S (based on Qwen2.5 1.5B/7B models, measured on vLLM 0.27.1).
50%UnverifiedAt small batch sizes the bottleneck is memory bandwidth, while at large batch sizes the rising compute density makes compute the bottleneck, which explains the performance crossover between the 3090 and the L40S.
50%UnverifiedFodder Watch requires a 48 GB PCIe card (L40S class), with a self-purchased node three-year cost of approximately $15,200 (range $14,600 to $15,800).
50%UnverifiedThe L40G generally refers to the L40S or a variant targeting specific scenarios, further enhancing the balance between AI inference and training capabilities.
50%UnverifiedIn this int8 workload, both 24 GB cards and the L40S took 3.97 seconds per step, while the H100 PCIe required only 2.99 seconds
50%Unverified英伟达在芯片市场有H100（旗舰）和L40S（主力）的产品区分
50%