vLLM Semantic Router
A semantic routing component based on the vLLM inference engine that places a lightweight routing layer in front of the inference engine. It dispatches incoming requests to the most appropriate downstream model or service based on the semantic content of each request, with the goal of reducing average inference cost.
Timeline (last 90 days)
AMD, in collaboration with the vLLM Semantic Router team, launched the d3-ultra model.
Vela 3.0 was released by the vLLM Semantic Router, KR Labs, and the AMD team in collaboration.
Judging from the naming, d3-ultra is a component under the vLLM Semantic Router project rather than an independent foundation model, and it is used by being embedded in the semantic routing pipeline.
The vLLM Semantic Router team released a 27B open-weight multimodal decision model named d3-ultra.
The vLLM Semantic Router team released a family of decision models called Decision 3.0 (abbreviated as d3), covering multiple parameter scales from 0.6B to 27B, released under the Apache-2.0 open-source license, and supporting three input modalities: text, image, and video.
A semantic routing architecture can route high-frequency simple requests to small models and complex requests to large models, creating a dynamic balance between cost and quality
vLLM Semantic Router released Decision 3.0, a set of multimodal decision models licensed under Apache-2.0
vLLM Semantic Router places a lightweight routing layer in front of the vLLM inference engine, using a decision model to determine the request type and direct it to a specialized model
vLLM Semantic Router released Decision 2.0.
Using an efficient semantic routing layer to make decisions and route traffic before requests reach the large model can both reduce costs and improve service quality.
1 more timeline events
All Facts (11)
AMD, in collaboration with the vLLM Semantic Router team, launched the d3-ultra model.
50%UnverifiedJudging from the naming, d3-ultra is a component under the vLLM Semantic Router project rather than an independent foundation model, and it is used by being embedded in the semantic routing pipeline.
50%UnverifiedA semantic routing architecture can route high-frequency simple requests to small models and complex requests to large models, creating a dynamic balance between cost and quality
50%UnverifiedvLLM Semantic Router places a lightweight routing layer in front of the vLLM inference engine, using a decision model to determine the request type and direct it to a specialized model
50%UnverifiedThis project is led and developed by Xunzhuo Liu and his team.
50%UnverifiedUsing an efficient semantic routing layer to make decisions and route traffic before requests reach the large model can both reduce costs and improve service quality.
50%UnverifiedVela 3.0 was released by the vLLM Semantic Router, KR Labs, and the AMD team in collaboration.
50%UnverifiedThe vLLM Semantic Router team released a 27B open-weight multimodal decision model named d3-ultra.
50%UnverifiedThe vLLM Semantic Router team released a family of decision models called Decision 3.0 (abbreviated as d3), covering multiple parameter scales from 0.6B to 27B, released under the Apache-2.0 open-source license, and supporting three input modalities: text, image, and video.
50%UnverifiedvLLM Semantic Router released Decision 3.0, a set of multimodal decision models licensed under Apache-2.0
50%UnverifiedvLLM Semantic Router released Decision 2.0.
50%