Route Requests by Replica Load
Easy · Solved in PyTorch · Machine learning coding practice
Problem
For each arriving request, choose an online replica with the smallest outstanding load per unit of capacity, breaking ties by the smallest replica index. Add that request's cost to the chosen replica's load before routing the next request. This is an educational load-balancing policy, not a reproduction of a particular vLLM router.
loads \ell and capacities c are int64 of shape (N,) with \ell_j \ge 0 and c_j > 0; online o is bool of shape (N,); costs w is int64 of shape (M,) with w_i \ge 0. Routing arrivals in order i = 0, \dots, M-1, request i goes to
Topics: inference, vllm, pytorch
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Route Requests by Replica Load in the browser and run it against the tests.