K competing queues with customer abandonment: optimality of a generalised \(c \mu \)rule by the Smoothed Rate Truncation method
 78 Downloads
Abstract
We consider a Kcompeting queues system with the additional feature of customer abandonment. Without abandonment, it is optimal to allocate the server to a queue according to the \(c \mu \)rule. To derive a similar rule for the system with abandonment, we model the system as a continuoustime Markov decision process. Due to impatience, the Markov decision process has unbounded jump rates as a function of the state. Hence it is not uniformisable, and so far there has been no systematic direct way to analyse this. The Smoothed Rate Truncation principle is a technique designed to make an unbounded rate process uniformisable, while preserving the properties of interest. Together with theory securing continuity in the limit, this provides a framework to analyse unbounded rate Markov decision processes. With this approach, we have been able to find closefitting conditions guaranteeing optimality of a strict priority rule.
Keywords
Competing queues Abandonments Unbounded transition rates Smoothed Rate Truncation Markov decision processes Generalised \(c \mu \)rule UniformisationMathematics Subject Classification
90C40 60K251 Introduction
In this paper, we consider a server assignment problem. There are K customer classes, and each customer class \(1\le i\le K\) has holding costs \(c_i\) per unit time, per customer. There is a single server that can serve class i at rate \(\mu _i\). Arrivals occur according to independent Poisson streams, independently of the service process. Each class i customers abandons the system at rate \(\beta _i\), \(1\le i\le K\), independently of whether he is being served or waiting in the queue. The question we address in this paper is: what service policy minimises the expected discounted total and average cost?
In the Kcompeting queues model without abandonments, it is wellknown that the \(c \mu \)rule is optimal. The \(c \mu \)rule gives full priority to the queue with the highest index \(c_i\mu _i\), that is, the queue that gives the highest cost reduction per unit time. This result was shown to be optimal in 1985 simultaneously by Baras et al. (1985) and by Buyukkoc et al. (1985).
 1.
Study of a relaxation or approximate version of the original problem (see e.g. Atar et al. 2010; Ayesta et al. 2011; Larrañaga et al. 2013, 2015). The obtained policies may serve as a heuristics.
 2.
Application of specific coupling techniques to obtain an optimal policy. Typically these papers (see e.g. Salch et al. 2013; Down et al. 2011, see also Ertiningsih et al. 2015) are limited to special cases, as the coupling gets more tedious in a more general setting. On the other hand, nonMarkovian service time distributions and/or a nonMarkovian arrival process may be handled.
 3.
Truncation of the process to make it uniformisable. Then use discretetime techniques to derive properties of the optimal policy (see e.g. Down et al. 2011; Bhulai et al. 2014; Blok and Spieksma 2015). This is the solution method that we will follow in this paper.
Ayesta et al. (2011) studied the problem as well. They derive priority rules similar to the \(c \mu /\beta \)rule by analytically solving the case with one or two customers initially present and without arrivals.
Larrañaga et al. (2013) have studied a fluid approximation of the multiserver variant of the competing queues problem. In this fluid approximation optimality of the \(c \mu /\beta \)rule in the overloaded regime is shown and it is shown that for \(K=2\) a switching curve policy is optimal in the underloaded regime. In Larrañaga et al. (2015) the same authors study asymptotic optimality of the multiserver competing queues problem for the average expected cost criterion. The authors consider the problem as a restless multiarmed bandit problem, and compute and show that the Whittle index is asymptotically optimal for convex holding cost. The asymptotics concern large states, and light and heavy traffic regimes. The paper also connects the \(c \mu /\beta \)rule to the Whittle index for fluid approximations.
Other papers do not focus on heuristics, but try to find a subset of the input parameters for which a strict priority rule can be proven to be optimal. Salch et al. (2013) study the competing queues system with a restriction to a maximum of K arrivals. Customers may be impatient, but do not leave the system when they become impatient. Thus, the model is, in fact, a scheduling problem, and the criterion is to minimise the expected weighted number of impatient customers. With the use of a coupling and an interchange argument optimality of a priority policy is proved, provided a set of three conditions on the service, impatience and cost rates holds.
The paper of Down et al. (2011) considers a twocompeting queues reward system, where the two classes have equal service rates. A coupling argument is employed to show that if type 1 customers have the largest abandonment rate and reward per unit time, then prioritising these customers is optimal.
The approach that we will carry out is the following. First, we model the problem as a continuoustime MDP. To make the MDP uniformisable, a truncation is necessary. After uniformisation, the truncated processes can be analysed by value iteration. To justify appropriate convergence of the truncated processes to the original model, a limit theorem is required. To our knowledge, so far such a theorem is available only for the discounted cost criterion, see Blok and Spieksma (2015). Via a vanishing discount approach, the results are transferred to the average cost criterion (see Blok and Spieksma 2017 for the justification). Therefore, we will first show for the discounted cost criterion that prioritising type i customers is optimal, if type i has maximum index with respect to c, \(c \mu \) and \(c\mu /\beta \). These conditions are similar to Salch et al. (2013), however the conditions of Salch et al. (2013) are implied by our conditions. Since the resulting index policy is optimal for all small discount factors, even strong Blackwell optimality of this policy follows.
In the paper of Down et al. (2011) a similar approach is used. The limit argument relies on specific properties of the model and a special truncation that does not affect optimality of the aforementioned priority policy. Due to the involved nature of the truncation, it seems unlikely that Down et al. (2011) can be extended to more dimensions or to heterogeneous service rates. The results of our paper can therefore be viewed as an extension of Down et al. (2011). In this paper, we use a different truncation technique called Smoothed Rate Truncation (SRT). This technique has been introduced by Bhulai et al. (2014) and can be utilised to make a process uniformisable while keeping the structural properties in tact.
The paper is organised as follows. In Sect. 2 we give a complete description of the model, and we present the main results. Section 3 contains the core of our analysis. First, it describes the Smoothed Rate Truncation in more detail, then the structural properties of the value function are derived. In Sect. 4 we prove the main theorem. This can be done by invoking the limit theorems of Blok and Spieksma (2017) and Blok and Spieksma (2015). Section 5 presents some numerical examples that show that none of the used conditions are redundant. In the “Appendix”, we provide the proofs of the propositions in Sect. 3.
2 Modelling and main result
2.1 Problem formulation
We consider K stations that are served by a single server. Customers arrive to the stations according to independent Poisson processes with rates \(\lambda _i> 0\) for \(i=1,\ldots , K\), respectively. The service requirements of class i customers are exponentially distributed with parameter \(\mu _i>0\). Customers have limited patience: they are willing to wait an exponential time with parameter \(\beta _i>0\) for class i. We allow abandonment during service as well, resulting in an abandonment rate in station i of \(\beta _i x_i\) if there are \(x_i\) customers present at station i. In Sect. 2.2 we will also discuss alternative modelling choices.
The service requirements, abandonments and arrivals are all stochastically independent of each other. Class i customers carry holding costs \(c_i\ge 0\) per unit time, \(i=1,\ldots , K\). The service regime is preemptive.
It is not to be expected that the optimal policy has a simple description in general. In this paper, we will restrict to providing sufficient conditions for optimality of an index policy.
2.2 Main result
The two main results of our paper are Theorems 1 and 2, providing sufficient conditions for optimality of the Smallest Index Policy.
Definition 1
The Smallest Index Policy assigns the server to the nonempty station with the smallest index. The policy only idles, if no customers are present.
Theorem 1
Theorem 2
Under the conditions of Theorem 1, the Smallest Index Policy is average cost optimal.
The proofs are postponed until Sect. 4. In Sect. 5 we give examples showing, that if any of the three conditions of (1) is omitted, the Smallest Index Policy can fail to be optimal.
Alternative modelling choices In our model the cost function is a holding cost \(\sum {c_ix_i}\) per unit time, when the system is in state x. In many applications a penalty (say \(P_i\) for class i) is charged, if a customer abandons the system due to impatience. Then the cost per unit time is given by \(\sum _i P_i\beta _i x_i\). Substitution of \(c_i=P_i\beta _i\), \(i=1,\ldots , K\) implies equivalence of these cost structures.
We modelled the system, such that customers can leave the system while being in service. In some models, it may be more realistic that abandonment does not take place, after service has started. However, if the abandonment rates are smaller than the service rates, i.e., \(\beta _i< \mu _i\) for all i, then our analysis is still valid after an appropriate parameter change. That is, we consider the system with service rates \({\hat{\mu }}_i=\mu _i\beta _i>0\). Abandonments during service or service completions in the revised model correspond to a service completion in the original one.
If, for one or more classes, the abandonment rates are greater than or equal to the associated service rates, then this substitution is clearly not possible. However, serving that customer class delays the process of emptying the system. It follows directly that in this case, it can never be optimal to serve these classes of customers. Hence, when there are only customers of that type present then the server should idle in order to minimise the expected average cost. Therefore, the optimal policy never serves class i if \(\mu _i\le \beta _i\). For the remaining customer classes with \(\mu _i>\beta _i\), the Smallest Index Policy is optimal, whenever these classes can be ordered, such that \(c\searrow ,\ c{\hat{\mu }}\searrow ,\ c{\hat{\mu }}/\beta \searrow \).
Finally, it is possible to allow idling at all times. However, it can easily be shown that it is not optimal to have unforced idling. Therefore, we ignore this option for the sake of notational convenience.
2.3 Structural properties
As mentioned in the introduction, Sect. 1, we will first study the \(\alpha \)discounted cost problem. Crucial in establishing optimality of the Smallest Index Policy are certain properties of the value function. If \(V_\alpha \) is nondecreasing (I) and weighted Upstream Increasing (wUI), then optimality of the Smallest Index Policy can be directly deduced from the \(\alpha \)discounted cost optimality equation under certain conditions on the Markov decision problem (cf. Blok and Spieksma 2015, 2017) that we will not discuss explicitly in this paper. We will next define the structural properties (I) and (wUI).
Definition 2
The following lemma makes the connection between the structural properties of the \(\alpha \)discounted cost value function and optimality of the Smallest Index Policy.
Lemma 1
Let the discount factor \(\alpha >0\). Then, the \(\alpha \)discounted cost value function \(V_\alpha \) is welldefined and finite. Suppose \(V_\alpha \in wUI \cap I\), then the Shortest Index Policy is \(\alpha \)discount optimal.
Proof
 P1
 There exist a function Open image in new window , and a constant \(\gamma <\alpha \) with the properties that If F satisfies the first property, then F is called a \(\gamma \)drift function for the MDP.
 P2

There exist a function Open image in new window and a constant \(\xi \), such that the following properties are satisfied.

G is a \(\xi \)drift function for the MDP.
 G is an Fmoment function, i.e. there exists an increasing sequence \(\{K_n\}_n\), Open image in new window , \(K_n<\infty \), Open image in new window , \(n\rightarrow \infty \), such that$$\begin{aligned} \inf _{x\not \in K_n}\frac{G_x}{F_x}\rightarrow \infty ,\quad n\rightarrow \infty . \end{aligned}$$
Property P2 immediately follows by setting \(G_x=e^{\epsilon '(x_1+\cdots +x_K)}\), for any \(\epsilon '>\epsilon \).
3 Discrete time discounted cost analysis
3.1 Smoothed Rate Truncation
The abandonment rates increase linearly in the number of waiting customers. Hence the transition rates are unbounded as a function of the state. Thus, the system is not uniformisable and so there is no discretetime equivalent to the continuoustime problem. To make discretetime theory available, we approximate the MDP with a sequence of (essentially) finite state MDPs. Unfortunately, standard state space truncations generally destroy the structural properties of interest due to boundary effects.
To this end, we have developed the Smoothed Rate Truncation (SRT). This perturbation technique was first introduced in Bhulai et al. (2014). In that paper, SRT is applied to a Markov cost process, and properties of the value function are proven. The distinguishing feature of SRT is that the transition rates are decreased in all states, also close to the origin. This makes the jump rates highly state dependent and complicates the analysis, but it is the key feature of SRT that ensures that the properties are preserved.
The idea of SRT is as follows. Every transition that moves the system into a higher state in one or more dimensions is linearly decreased as a function of these coordinates. This naturally generates a finite subset of the space, that cannot be left with positive probability under any policy. As a consequence, recurrent classes under any policy are always finite. As we get closer to the boundary of the finite set, the rates are smoothly truncated to 0. On the finite state space, the transition rates are bounded. Outside the finite set, the rates can be arbitrarily chosen, since these states are inessential. In particular, they can be chosen such that the jump rates are uniformly bounded.
In our model, a truncation parameter \(N=(N_1,\ldots , N_K)\in {\mathcal {N}}=(\mathbb {N}\cup \infty )^K\) defines the size of the state space. Since the empty state can always be reached, and there is a positive probability of an arrival in any queue within the finite set (not on the boundary clearly), the set of essential states is given by \(S^N=\{x\in S x_i\le N_i, i=1,\ldots ,K\}\).
Furthermore, the perturbed MDP is easily checked to satisfy the conditions of (Blok and Spieksma 2015, Theorems 4.2 and 5.1). The main ingredients of its verification are analogous to the proof of Lemma 1. The results in Blok and Spieksma (2015) guarantee that the value function \(V^{(N)}_\alpha \) of the Nperturbed MDP is welldefined with \(V^{(N)}_\alpha \rightarrow V_\alpha \), \(N\rightarrow \infty ^K\), and any limit point of \(\alpha \)discount optimal policies for the Nperturbation, \(N\rightarrow \infty ^K\), is \(\alpha \)discount optimal for the original MDP.
3.2 Dynamic programming
To employ the induction argument, we need three additional structural properties: convexity, supermodularity and bounded increasingness. We will specify these hereafter. The induction hypothesis \(v_{0,{\bar{\alpha }}}^{(N,d)}\equiv 0\) trivially satisfies all these properties. For the induction step, we will use Event Based Dynamic Programming (EBDP). This method uses event operators—representing arrivals, departures or cost—as building blocks to construct the iteration step of the value iteration algorithm.
Definition 3
 1.
 (a)The total smoothed arrivals operatorusing$$\begin{aligned} {\mathcal {T}}_{SA}^{N}f:={\bar{\lambda }}^{1}\sum _{i=1}^K \lambda _i {\mathcal {T}}_{SA(i)}^Nf, \end{aligned}$$
 (b)the smoothed arrivals operator given by$$\begin{aligned} {\mathcal {T}}_{SA(i)}^Nf(x):=\left\{ \begin{array}{ll} \left( 1\frac{x_i}{N_i}\right) f(x+e_i) +\frac{x_i}{N_i}f(x), &{} \quad x_i\le N_i,\\ f(x), &{} \quad \text {else.} \end{array} \right. \end{aligned}$$
 (a)
 2.
 (a)The total increasing departures operatorusing$$\begin{aligned} {\mathcal {T}}_{ID}^{N}f:=\beta _N^{1}\sum _{i=1}^K \beta _i N_i {\mathcal {T}}_{ID(i)}^Nf, \end{aligned}$$
 (b)the increasing departures operator$$\begin{aligned} {\mathcal {T}}_{ID(i)}^Nf(x):=\left\{ \begin{array}{ll} \frac{x_i}{N_i}f(xe_i)+ \left( 1\frac{x_i}{N_i}\right) f(x), &{} \quad x_i \le N_i,\\ f(xe_i), &{} \quad \text {else.} \end{array} \right. \end{aligned}$$
 (a)
 3.The cost operator$$\begin{aligned} {\mathcal {T}}_Cf(x):= \sum _{i=1}^K c_ix_i +f(x). \end{aligned}$$
 4.The cost + increasing departures operator$$\begin{aligned} {\mathcal {T}}_{CID}^{N}:=\beta _N^{1} {\mathcal {T}}_C(\beta _N {\mathcal {T}}_{ID}^{N}). \end{aligned}$$
 5.The movable server operator$$\begin{aligned} {\mathcal {T}}_{MS}f(x):= \min _{1\le j\le K} \Big \{\frac{\mu _j}{\mu } f((xe_j)^+) + \Big (1\frac{\mu _j}{\mu }\Big )f(x)\Big \}. \end{aligned}$$
 6.For \(f_1,f_2,f_3:{S}\rightarrow {\mathbb {R}}\), the uniformisation operator$$\begin{aligned} {\mathcal {T}}_{UNIF}^N(f_1,f_2,f_3):= {\bar{\lambda }} f_1 +\beta _N f_2 + \mu f_3. \end{aligned}$$
 7.The discount operator$$\begin{aligned} {\mathcal {T}}_{DISC}^{{\bar{\alpha }}}f:= {\bar{\alpha }}f. \end{aligned}$$
Definition 4
 1.Weighted upstream increasing functions on \({S}^N\)$$\begin{aligned}&wUI_N= \{f:{S}\rightarrow {\mathbb {R}}\ \quad \mu _i(f(x+e_i+e_{i+1})f(x+e_{i+1}))\\&\quad \mu _{i+1}(f(x+e_i+e_{i+1})f(x+e_i)) \ge 0,\\&\qquad \text { for all } x,x+e_i+e_{i+1} \in {S}^N,\ 1\le i< K\}. \end{aligned}$$
 2.Increasing functions on \({S}^N\)$$\begin{aligned} I_N=\{f:{S}\rightarrow {\mathbb {R}}\ \ f(x+e_i)f(x)\ge 0,\text { for all } x,x+e_i \in {S}^N,\ 1\le i \le K\}. \end{aligned}$$
 3.Supermodular functions on \({S}^N\)$$\begin{aligned}&\,{Super}_N=\{f:{S}\rightarrow {\mathbb {R}}\ \ f(x+e_i+e_j)f(x+e_i)f(x+e_j)+f(x)\ge 0,\\&\quad \text { for all } x,x+e_i+e_{j} \in {S}^N,\ 1\le i<j\le K\}. \end{aligned}$$
 4.Convex functions on \({S}^N\)$$\begin{aligned}&Cx_N=\{f:{S}\rightarrow {\mathbb {R}}\ \ f(x+2e_i)\\&\quad 2f(x+e_i)+f(x)\ge 0,\text { for all } x,x+2e_i \in {S}^N,\ 1\le i \le K\}. \end{aligned}$$
 5.Bounded increasing functions on \({S}^N\)$$\begin{aligned} Bd_N=\{f:{S}\rightarrow {\mathbb {R}}\ \ f(x+e_i)f(x)\le \frac{c_i}{\beta _i},\text { for all } x,x+e_i \in {S}^N.\ 1\le i \le K\}. \end{aligned}$$
The following propositions are sufficient for the desired structural properties to propagate through the induction step.
Proposition 1
 (i)$$\begin{aligned} {\mathcal {T}}_{SA}^{N}: I_N\rightarrow I_N, Cx_N\rightarrow Cx_N, \,{Super}_N\rightarrow \,{Super}_N, Bd_N\rightarrow Bd_N. \end{aligned}$$
 (ii)If moreover \(N\in {\mathcal {N}}(\lambda )\), then$$\begin{aligned} {\mathcal {T}}_{SA}^{N}: I_N\cap wUI_N\rightarrow wUI_N. \end{aligned}$$
Proposition 2
Proposition 3
Proposition 4
 (i)$$\begin{aligned} {\mathcal {T}}_{CID}^{N}: Bd_N\rightarrow Bd_N. \end{aligned}$$
 (ii)If moreover, for all \(1\le i< K\), it holds that \(c_i\ge c_{i+1},\ c_i\mu _i\ge c_{i+1}\mu _{i+1},\ {c_i\mu _i}/{\beta _i}\ge {c_{i+1}\mu _{i+1}}/{\beta _{i+1}}\), then$$\begin{aligned} {\mathcal {T}}_{CID}^{N}: I_N\cap wUI_N\cap \,{Super}_N\cap Bd_N\rightarrow wUI_N.\end{aligned}$$
Proposition 5
 (i)$$\begin{aligned}&{\mathcal {T}}_{MS}: I_N\cap wUI_N\rightarrow I_N\cap wUI_N,\\&I_N\cap wUI_N\cap Cx_N\cap \,{Super}_N\rightarrow Cx_N\cap \,{Super}_N. \end{aligned}$$
 (ii)If moreover, for all \(1\le i< K,\ {c_i\mu _i}/{\beta _i}\ge {c_{i+1}\mu _{i+1}}/{\beta _{i+1}}\) then$$\begin{aligned} {\mathcal {T}}_{MS}:I_N\cap wUI_N\cap Bd_N\rightarrow Bd_N. \end{aligned}$$
Proposition 6
Proposition 7
The proofs of the propositions are provided in the “Appendix”.
Corollary 1
 (i)Then, for all \(n\ge 0\)$$\begin{aligned} v_{n,{\bar{\alpha }}}^{(N,d)}\in wUI_N\cap I_N\cap Cx_N\cap \,{Super}_N\cap Bd_N; \end{aligned}$$
 (ii)consequently,$$\begin{aligned} V_{{\bar{\alpha }}}^{(N,d)}\in wUI_N\cap I_N\cap Cx_N\cap \,{Super}_N\cap Bd_N. \end{aligned}$$
Proof
Assertion ii) immediately follows from i) due to convergence of value iteration [see Wessels 1977, (Blok and Spieksma 2017, Theorem 5.2)]. \(\square \)
4 Proof of main theorems
Proof of Theorem 1
Suppose that for \(1\le i <K \), \(c_i\ge c_{i+1}\), \( c_i\mu _i\ge c_{i+1}\mu _{i+1}\), \( {c_i\mu _i}/{\beta _i}\ge {c_{i+1}\mu _{i+1}}/{\beta _{i+1}}\). Let the continuoustime discount factor \(\alpha >0\), then the discrete time discount factor \({\bar{\alpha }}= {1}/{\alpha +1}\) satisfies \(0<{\bar{\alpha }}<1\). Take \(N\in {\mathcal {N}}(\lambda )\), then Corollary 1 implies that \(V_{{\bar{\alpha }}}^{(N,d)}=V_{\alpha }^{(N)}\in wUI_N\cap I_N\). This model satisfies the parametrised Markov processes theorem in (Blok and Spieksma 2015, Theorem 5.1), which implies continuity in the truncation parameter. This means that \(V_{\alpha }^{(N)}\rightarrow V_{\alpha }\) as \(N\rightarrow \infty \). Hence, \(V_{\alpha }\in wUI \cap I\) and so by virtue of Lemma 1, the smallest index policy is \(\alpha \)discount optimal. \(\square \)
Proof of Theorem 2
Suppose that for \(1\le i <K \), \(c_i\ge c_{i+1}\), \( c_i\mu _i\ge c_{i+1}\mu _{i+1}\), \( {c_i\mu _i}/{\beta _i}\ge {c_{i+1}\mu _{i+1}}/{\beta _{i+1}}\). By Theorem 1 the Smallest Index Policy, \(\pi ^\alpha \) say, is \(\alpha \)discount optimal, for all \(\alpha >0\).
Notice, that the model satisfies the assumptions of (Blok and Spieksma 2017, Theorem 5.7). This theorem implies the existence of a sequence \((\alpha _m)\) with \(\lim _{m\rightarrow \infty }\alpha _m=0\), such that the limit \(\lim _{m\rightarrow \infty } \pi ^{\alpha _m}\) is average optimal. Since \(\pi ^\alpha \) is the smallest index policy for all \(\alpha >0\), so is the limit policy. Hence the Smallest Index Policy is average optimal. \(\square \)
5 Numerical results
The triple inequality on the input parameters of the process guaranteeing optimality of the Smallest Index Policy induces a lot of parameter configurations that fall outside the scope of the theorems. This naturally gives rise to the question whether all three inequalities are necessary.
 1.Consider the following parameter setting We see that the first condition is violated, \(c_1<c_2\), while the other conditions are satisfied. The optimal policy is a switching curve policy: for ‘small’ states action 2 is optimal and for ‘large’ states action 1 is optimal, see Fig. 1. Note that colour green corresponds to action 1, i.e. serving queue 1, and colour red to action 2 i.e. serving queue 2.
 2.The next parameter setting is given by Observe, that the first and the third condition hold, but the second condition is violated. In Fig. 2 the optimal policy is displayed. We see that the Smallest Index Policy need not be optimal. There is a small region—with only few customers—where it is optimal to take action 2. In the larger states action 1 is optimal, that is, the Smallest Index Policy is optimal.
 3.The final parameter setting is given by Here only the first and second condition are satisfied. Figure 3 shows that it can be optimal to serve the station with the highest index instead of the smallest index.
Notes
References
 Atar, R., Giat, C., & Shimkin, N. (2010). The \(c \mu /\theta \) rule for manyserver queues with abandonment. Operations Research, 58(5), 1427–1439.CrossRefGoogle Scholar
 Ayesta, U., Jacko, P., & Novak, V. (2011). A nearlyoptimal index rule for scheduling of users with abandonment. In 2011 IEEE Proceedings of INFOCOM (pp. 2849–2857). IEEE.Google Scholar
 Baras, J., Ma, D., & Makowski, A. (1985). K competing queues with geometric service requirements and linear costs: The [mu] crule is always optimal. Systems & Control Letters, 6(3), 173–180.CrossRefGoogle Scholar
 Bhulai, S., Brooms, A., & Spieksma, F. (2014). On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue. Queueing Systems, 76(4), 425–446.CrossRefGoogle Scholar
 Blok, H., & Spieksma, F. (2015). Countable state Markov decision processes with unbounded jump rates and discounted cost: Optimality and approximations. Advances in Applied Probability, 47(4), 1088–1107.CrossRefGoogle Scholar
 Blok, H., & Spieksma, F. (2017). Structures of optimal policies in Markov Decision Processes with unbounded jumps: The State of our Art, chap. 5 (pp. 139–196). Berlin: Springer.Google Scholar
 Buyukkoc, C., Varaiya, P., & Walrand, J. (1985). The c\(\mu \) rule revisited. Advances in Applied Probability, 17(1), 237–238.CrossRefGoogle Scholar
 Down, D., Koole, G., & Lewis, M. (2011). Dynamic control of a singleserver system with abandonments. Queueing Systems, 67(1), 63–90.CrossRefGoogle Scholar
 Ertiningsih, D., Bhulai, S., & Spieksma, F. M. (2015). Monotonicity properties of the single server queue with abandonments and retrials by coupling. Tech. rep., Mathematisch Instituut Leiden.Google Scholar
 Larrañaga, M., Ayesta, U., & Verloop, I. (2013). Dynamic fluidbased scheduling in a multiclass abandonment queue. Performance Evaluation, 70(10), 841–858.CrossRefGoogle Scholar
 Larrañaga, M., Ayesta, U., & Verloop, I. (2015). Asymptotically optimal index policies for an abandonment queue with convex holding cost. QUESTA, 81(2), 99–169.Google Scholar
 Salch, A., Gayon, J. P., & Lemaire, P. (2013). Optimal static priority rules for stochastic scheduling with impatience. Operations Research Letters, 41(1), 81–85.CrossRefGoogle Scholar
 Serfozo, R. (1979). An equivalence between continuous and discrete time Markov decision processes. Operational Research, 27(3), 616–620.CrossRefGoogle Scholar
 Strauch, R. (1966). Negative dynamic programming. The Annals of Mathematical Statistics, 37(4), 871–890.CrossRefGoogle Scholar
 Wessels, J. (1977). Markov programming by successive approximations with respect to weighted supremum norms. Journal of Mathematical Analysis and Applications, 58(2), 326–335.CrossRefGoogle Scholar
Copyright information
OpenAccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.