Model Router
A model routing service introduced by Microsoft Azure Foundry that automatically directs requests to the most suitable language model based on task complexity, optimizing cost and latency while maintaining output quality.
Core Facts
Timeline (last 90 days)
The OpenHands Model Router is a mechanism introduced in v1.25.0 that can automatically select among multiple models or providers based on task type, cost or availability and dispatch requests accordingly.
Model routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
The capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
In the author's test of 10 prompts, Model Router delivered a 37.7% cost saving compared to using GPT 5.6 Sol directly.
In the author's tests, average latency was slightly higher with Model Router than with GPT 5.6 Sol used directly, and quality win rate declined.
From a task-type perspective, Model Router is slightly slower on general knowledge and summarization tasks, but performs well on reasoning tasks.
Microsoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
Model Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
When faced with math problems requiring step-by-step reasoning, the router automatically switched to the reasoning model Grok 4.1 Fast Reasoning.
When faced with queries requiring factual verification (such as whether Nairobi is a capital), the router assigned the request to GPT-5.5 for processing.
10 more timeline events
All Facts (13)
There are two paths to getting the model to output an accurate solution: writing detailed requirements all at once, or using follow-up questioning to have the model think on the user's behalf and accumulate context.
65%UnverifiedThe capability ceiling of a model router is always lower than that of the large language model it dispatches, because using a weaker model to direct a stronger one locks the ceiling at the weakest link.
50%UnverifiedModel routing decisions typically occur at the start of a request and cannot dynamically adjust based on the complexity that is revealed as a task progresses.
50%UnverifiedMicrosoft Foundry's Model Router routes prompts to the most suitable language model, rather than invoking the most powerful model for every request.
50%UnverifiedModel Router Auto-Evaluation is an open-source project that can automatically evaluate the performance of Foundry Model Router against any baseline model across three dimensions: quality, cost, and latency.
50%UnverifiedThe Model Router offers three routing modes: quality-biased, cost-biased, and balanced.
50%UnverifiedWhen the routing results for a certain type of request are highly concentrated, the routing layer can be removed and the model bound directly to reduce latency and scheduling costs; if routing results remain consistently dispersed, the router's dynamic decision-making should be retained.
50%UnverifiedThe model router provides a monitoring dashboard where token consumption, input and output of each request, and the actual model relied upon can be viewed.
50%UnverifiedMicrosoft Foundry's Model Router automatically selects the most appropriate model for each request.
50%UnverifiedThe Model Router is deployed in the same way as a regular model and can be found and deployed on the Model tab in the Discover section of Foundry.
50%UnverifiedThe Model Router covers models from multiple providers including OpenAI, Claude, Grok, Llama, and DeepSeek.
50%UnverifiedWhen deploying the Model Router, multiple deployment types are supported, including Global Standard, Standard, and Data Zone Standard.
50%UnverifiedThe OpenHands Model Router is a mechanism introduced in v1.25.0 that can automatically select among multiple models or providers based on task type, cost or availability and dispatch requests accordingly.
50%