Performance Analysis of Work Stealing Strategies in Large Scale Multi-threaded Computing

Abstract

Distributed systems use randomized work stealing to improve performance and resource utilization. In most prior analytical studies of randomized work stealing, jobs are considered to be sequential and are executed as a whole on a single server. In this paper we consider a homogeneous system of servers where parent jobs spawn child jobs that can feasibly be executed in parallel. When an idle server probes a busy server in an attempt to steal work, it may either steal a parent job or multiple child jobs.

To approximate the performance of this system we introduce a Quasi-Birth-Death Markov chain and express the performance measures of interest via its unique steady state. We perform simulation experiments that suggest that the approximation error tends to zero as the number of servers in the system becomes large. Using numerical experiments we compare the performance of various simple stealing strategies as well as optimized strategies.

A Model Validation

Based on numerical experiments in the Sect. 5, we see that stealing all or half of the children are good stealing policies: stealing all works best for low values of r, while stealing half of the children works well for higher values. Therefore, we validate the mean field model for the policy of stealing all or half of the children. We always start the simulations from an empty system and simulate the behaviour for $T = 10^5$ with a warm up period of 33% of T.

In Fig. 4 we focus on the case where all children are stolen. The 95% confidence intervals were computed based on 5 runs with $N=500$ servers, $m=4$, $\mu _1 = 1, \mu _2 = 2, \rho = 0.75$, $\mathbf {p} = (1,1,1,1,1)/5$ and $r\in \{1,5\}$. We see that there is an excellent match between the simulated waiting and service times and those of the QBD model (calculated using Sect. 4).

Fig. 4.

Waiting and response times from the QBD (blue dots) and simulations (red dashed line) with confidence intervals for 5 runs. (Color figure online)

In Table 2 we compare the relative error of the simulated mean response time, based on 20 runs, to the one obtained from Section 4. We do this for $\mu _1 = 1, \mu _2 = 2$, $\mathbf {p} = (1,1,1,1,1)/5$, $\rho \in \{0.75,0.85\}$, $r \in \{1,10\}$ and $N \in \{250,500,1000,2000,4000\}$.

The relative error in all cases is below 1.5% and tends to increase with the steal rate r. Further, the relative error seems roughly to halve when doubling N, which is in agreement with the results in [7].

Table 2. Relative error of simulation results for E[T(r)], based on 20 runs

Fig. 5.

Waiting and response times from the QBD (blue dots) and simulations (red dashed line) with confidence intervals for 5 runs. (Color figure online)

Next we validate the model for the strategy of stealing half of the children using the same simulation settings. In Fig. 5, we see that there is an excellent match between the simulated waiting and service times and those of the QBD model. Similarly to Table 2, we see in Table 3 that the relative error is below 1.5% in all cases, tends to increase with the steal rate r and seems about halved when doubling N.

Table 3. Relative error of simulation results for E[T(r)], based on 20 runs