Calibrate Then Route

An arXiv paper addressing intelligent request routing in disaggregated LLM serving. It proposes a hardware-calibration-based, learned router that estimates the additional completion time per instance to optimize request assignment decisions and reduce overall latency.