Concept模型 Serving / Model Serving
Serving
The full path by which a model is deployed as an online service and handles requests, covering the inference engine, batching and scheduling, KV cache, context management, rate limiting, and elastic scaling.
Timeline (last 90 days)
Oct 24
Serving in this context refers to the entire chain of deploying a model as an online service and handling requests, including the inference engine, batching and scheduling, KV cache and context management, rate limiting, and elastic scaling.
Unverified50%