Replicate to the shortest queues

Abstract

This paper introduces a load-balancing policy that interpolates between two well-known policies, namely join the shortest queue (JSQ) and join the least workload (JLW), and studies it in heavy traffic. This policy, which we call replicate to the shortest queues (RSQ(d)), routes jobs from a stream of arrivals into buffers attached to N servers by replicating each arrival into $1\le d\le N$ tasks and sending the replicas to the d shortest queues. When the first of the tasks reaches a server, its $d-1$ replicas are canceled. Clearly, RSQ(1) is equivalent to JSQ, and it has been shown that RSQ(N) is equivalent to JLW; intermediate values of d provide a trade-off between good performance measures of JSQ and those of JLW. In heavy traffic, a key property underlying asymptotic analysis of load-balancing policies is state space collapse (SSC). Unlike policies such as JSQ, where SSC is well understood, the treatment of SSC under RSQ(d) requires addressing the massive cancellations that highly complicate the queue length dynamics. Our first main result is that SSC holds under RSQ(d) for possibly heterogeneous servers. Based on this result, we obtain diffusion limits for the queue lengths in the form of one-dimensional reflected Brownian motion, asymptotic characterization of the short-time-averaged delay process and a version of Reiman’s snapshot principle. We illustrate using simulations that as d increases the server workloads become more balanced, and the delay distribution’s tail becomes lighter. We also discuss the implementation complexity of the policy as compared to that of the redundancy routing policy, to which it is closely related.