Stability of queueing-inventory systems with customers of different priorities

We study a production-inventory system with two customer classes with different priorities which are admitted to the system following a flexible admission control scheme. The inventory management is according to a base stock policy and arriving demand which finds the inventory depleted is lost (lost sales). We analyse the global balance equations of the associated Markov process and derive structural properties of the steady state distribution which provide insights into the equilibrium behaviour of the system. We derive a sufficient condition for ergodicity using the Foster-Lyapunov stability criterion. For a special case we show that the condition is necessary as well.


Introduction
The research communicated in this article is devoted to queueing-inventory systems where customers of two classes of different priorities are served and for serving a customer a piece of raw material from an associated finished goods inventory is needed.The inventory management is according to a standard base stock policy and replenishment orders are fulfilled by an external supplier and lead time is positive.Customers arriving when the inventory is depleted are rejected (lost sales) and additionally arrivals are regulated by a flexible admission control which respects priorities.
Our main interest are stability conditions for this system and the stationary behaviour of a stable system.We derive structural properties of the steady state distribution which provide insights into the equilibrium behaviour of the system but an explicit expression of the complete stationary distribution is still an open problem.Inspection of the literature shows that up to now there is no result available which provides rigorously proved conditions for stability in case that both customer classes have unbounded queues.Our main result is therefore to derive a sufficient condition for ergodicity by the Foster-Lyapunov stability criterion.For a subclass of the admission policies we show that this condition is necessary as well.We expect that this is in general not the case.Furthermore, we consider the case of instant service, where we determine the stationary distribution explicitly.
The paper is organised as follows.In Section 2, we describe the related literature.In Section 3, we introduce our integrated model for production and inventory management and provide a Markov process Z for the description of the time evolution of the integrated system.In Section 4.1, ergodicity is investigated in detail.In Section 4.2, we assume that the queueing-inventory process Z is ergodic to analyse the properties of the stationary system.In Section 5, we consider the case of zero service time.In Section 6 we summarise our main findings and indicate further research directions.

Related literature and own contributions
Our research is connected to various parts of queueing theory and inventory control and their interplay in queueing-inventory models.Our fundamental production system is a classical M/M/1/∞ queueing system, see e.g.Wolff [Wol89,Chapter 5].The specific features added to the M/M/1/∞ in this article are the following: • Service of a customer needs a piece of raw material available in an associated inventory (for a review of research on these models see e.g.Krishnamoorthy and his coauthors [KSN21]).
• Arriving customers have different priorities which result in a complicated admission control problem (for fundamentals on priority queues see Jaiswal [Jai68]).
Although in our models the customers are of different priority classes, they require the same type of raw material stored in a single inventory and they experience the same service time distributions.The first property means that the items of raw material in the inventory are exchangeable with respect to the requesting customers' services.This property is common in certain applications of e.g.maintenance and repair, see e.g.investigations by Ravid, Boxma, and Perry [RBP13] and Daduna [Dad90] and references therein.The second property is discussed and motivated by Wang and his coauthors [WBSW15, p. 733] who identify three main motivations for prioritisation and argue that two of these lead to identical service times over classes.
Literature on inventory theory related to our problems encompass the following problems and articles.Studies by Gruen, Corsten, and Bharadwaj [GCB02] and Verhoef and Sloot [VS06] analyse customers' behaviour in practice and show that in many retail settings most of the original demand can be considered to be lost in case of a stockout.For an overview of the literature on systems with lost sales we refer to Bijvank and Vis [BV11].They present a classification scheme for the replenishment policies most often applied in literature and practice, and they review the proposed replenishment policies, including the base stock policy.Tempelmeier [Tem05,p. 84] argued that base stock control is economically reasonable if the order quantity is limited because of technical reasons.The base stock policy is "(...) more suitable for item with low demand, including the case of most spare parts" [RM11,p. 661].Morse [Mor58,p. 139] investigated (pure) inventory systems that operate under a base stock policy.He gives a very simple example where the concept "re-order for each item sold" is use-ful: Items in inventory are bulky, and expensive (automobiles or TV sets1 ).He uses queueing theory to model the inventory systems, analogously to e.g.Reed and Zhang [RZ17].Using queueing-theoretical methods to solve inventory models is well established, often under the heading of "production-inventory systems".In this setting the production usually refers to the replenishment systems, early references are e.g.Morse [Mor58], Karush [Kar57] (lost sales), Kaplan [Kap70] (backordering).
More recent examples are Rubio and Wein [RW96], Lee and Zipkin [LZ92], [LZ95], Zazanis [Zaz94], and Li and Arrelo-Risa [LAR21] who investigated classical single item and multi-item inventory systems.Especially they evaluate the performance of base stock control policies in complex situations.
For a review of literature on inventory control systems (= queueing-inventory systems with service time equal to 0, i.e. "instant service" [MPA18b]) with multiple customer classes, we refer to Isotupa ([Iso11, Section 2, pp.3ff.], [Iso15, Section 1, pp. 411ff.])and Arslan and his coauthors [AGR07, pp. 1486ff.].It is differentiated between priority disciplines that regulate customer arrivals and priority disciplines that regulate customer services.We combine both features in our model.
Literature on integrated queueing-inventory models seemingly appeared only from 1992 on, see e.g.Sigman and Simchi-Levi [SSL92], Melikov and Molchanov [MM92].For a recent extensive review we refer to Krishnamoorthy, Shajin and Narayanan [KSN21].Because the research described in the present article is on queueing-inventory systems with priority classes for customers we concentrate here on articles which deal with priority problems in queueing-inventory systems.Importance of research on this topic is emphasized by e.g.Yadavalli and his coauthors [YAJ15] who remarked that patients with serious illnesses are given priority over patients opting for routine checks or else in multi-speciality hospitals.Similarly, Liu and couauthors [LXC13,pp.1544f.]stated that orders with long term contracts have higher priority than unscheduled order since they may bear lower shortage cost than the booked orders.To the best of our knowledge, queueing-inventory systems (under the heading "inventory in counter-stream serving systems") with different classes of customers were first considered by Melikov and Fatalieva [MF98] who formulated a Markov decision model to minimize a cost function which encompasses costs for waiting, inventory holding, loss of demand and for dispatching items.In this problem setting there is no direct prioritisation of classes.Zhao and Lian [ZL11] seemingly were the first to investigate a system with Poisson arrivals of different priority classes, exponentially distributed service and lead times under backordering.They found a priority service rule to minimise the long-run expected waiting cost by a dynamic programming method.They formulate the model as a level-dependent quasi-birth-and-death process such that the steady state probability distribution of their production-inventory systems can be computed by the Bright-Taylor algorithm.Liu and coauthors ([CZL12], [LXC13], [LFW14]) introduce a flexible admission control with a priority parameter 0 ≤ p ≤ 1 "for controlling the application of priority" [LFW14, p. 181].The priority parameter p indicates the probability with which the arrivals of ordinary customers are treated like the arrivals of priority customers.If p = 1, there is no priority in regulating arrivals.If p = 0, there is a strict priority in regulating arrivals.In the last case, their model is the same as the model of Isotupa [Iso07].They derive the stationary distribution of the inventory levels and some performance measures.To obtain the optimal inventory control policy they construct a mixed integer optimization problem in [LXC13] and [LFW14] and develop an efficient searching algorithm in [CZL12].Jeganathan and coauthors investigate in a sequence of papers (e.g.[JKA16], [Jeg15], [YAJ15]) queueing-inventory systems with two classes of customers.They consider models with impatient customers, an optional second service and a mixed priority service (non-preemptive priority and preemptive priority).Li and Zhao [LZ09] investigate a preemptive priority queueing system (without inventory) with two classes of customers and an exponential single server who serves the two classes of customers at potentially different rates.Melikov and coauthors ([MPA18b], [MPA18a]) investigate variants of queueing-inventory systems with demand of two classes with high and low priority.They consider an admission control scheme which allows access for high priority customers in any case (backordering) and rejects low priority customers when the stock level is below a critical value.Using approximation methods they evaluate various performance measures of the systems.
The problem of stability for queueing-inventory systems with priority classes of demand has not found much interest in the literature.Most of the papers with priorities in queueinginventory systems use state space truncation, i.e. consider finite waiting rooms for tackling the stability problem, either both customer queues are assumed to be finite (then stability is not an issue) or one of the queues is truncated (most often the low priority queue, but in some articles high priority customers never queue up).Then the system fits into the realm of QBD processes.Some representative examples are for finite state space: for countable state space: [SDDK20] and [BDK17] (only a queue for low priority customers), [MPA18b] and [MPA18a] (steady state distribution obtained with approximation methods), [ZL11] (heuristic/intuitive criterion for ergodicity for two demand classes, extending the single demand class case of [SSD + 06]).
The conclusion is that (best to our knowledge) there exists up to now no rigoruos result (criterion) for stability of queueing-inventory systems with differentiated priorities for demand classes in case of state spaces which are two-dimensional infinite, i.e. including N 2 0 , counting for high and low priority customers (demands).This observation is not surprising because in a general context the problem can be termed as "ergodicity of random walks in the quarter plane" (N 2 0 ), which is known to be a notoriuosly hard problem, see Guy and his coauthors [FIM99].Even more, in the present problem setting this random walk is influenced by the additional (finite) dimension of the inventory.So our investigation is on ergodicity of random walks in the quarter plane in a random environment.A very recent investigation of such problems is from Dimitriou [Dim22].
Our main contributions are the following: Starting from a Markovian description for a queueing-inventory system with two unbounded queues for customers of different priority classes and flexible admission control we derive (rigorously) sufficient conditions which guarantee stability of the system.For special cases we even show that the condition is sharp (necessary for stability).Although the complete stationary distribution seems to be an open problem we are able to derive several partial balance properties of the system which are of independent interest.Furthermore, we consider the special case of zero service time and determine the stationary distribution explicitly.
• The notation ⊂ between sets means "subset or equal" and means "proper subset".For a set A we denote by |A| the number of elements in A.
• 1 {expression} is the indicator function which is 1 if expression is true and 0 otherwise.
• Empty sums are 0, and empty products are 1.
• All random variables are defined on a common probability space (Ω, F, P ).
• By Markov process we mean time-homogeneous continuous-time strong Markov process with discrete state space (= Markov jump process).Markov processes occurring are assumed to be regular and having paths which are right-continuous with left limits (cadlag).
A Markov process is regular if it is non-explosive (i.e. the sequence of jump times of the process diverges almost surely), its transition intensity matrix is conservative (i.e.row sums are 0) and stable (i.e.all diagonal elements of the transition intensity matrix are finite).

Description of the model
The supply chain of interest is depicted in Figure 1 and consists of two arrival streams of priority and ordinary customers, a production system (a single server with two unlimited waiting rooms), an inventory and a supplier.The production system manufactures units according to customers' demand on a make-toorder basis.There are two types of customers -priority customers (type 1) and ordinary customers (type 2).C = {1, 2} is the set of customer classes.Priority customers arrive according to a Poisson process with rate λ 1 > 0 and ordinary customers arrive according to a Poisson process with rate λ 2 > 0.
Each customer needs exactly one item from the inventory for service.The service time for both types of customers is exponentially distributed with parameter µ > 0. If the server is ready to serve a customer, who is at the head of the line, and the inventory is not depleted, the service begins immediately.Otherwise, the service starts at the instant of time when the next replenishment arrives at the inventory.A served customer departs from the system immediately and the associated item is removed from the inventory at this time instant.
An outside supplier replenishes raw material to the inventory according to a base-stock policy.Hence, each item taken from the inventory results in a direct order sent to the supplier.This means, if a served customer departs from the system, an order for one item of the consumed raw material is placed at the supplier at this instant of time.The base stock level b ≥ 2 is the maximal size of the inventory.The replenishment lead time is exponentially distributed with parameter ν > 0. (Note that there can be more than one outstanding order.)Customers' arrivals are regulated by a flexible admission control with priority parameter p, 0 ≤ p ≤ 1: If the inventory is depleted all arriving customers are rejected ("lost sales").If the on-hand inventory is greater than a prescribed threshold level s, 0 < s < b, customers of both classes are admitted to enter the system.If the on-hand inventory reaches or falls below the threshold Figure 1: The production-inventory system with two customer classes level s, priority customers still enter the system but ordinary customers are allowed to enter only with probability p and are rejected with probability 1 − p.
There is a single server with two separate infinite waiting rooms -one waiting room for priority customers (priority queue) and one waiting room for ordinary customers (ordinary queue) both under a FCFS regime.If both customer queues are not empty, the server needs to decide which one of them should be served.The choice is made according to the preemptive resume discipline.An overview of various priority disciplines can be found in [Jai68,p. 53].
According to the preemptive resume discipline a newly arriving priority customer interrupts immediately an ongoing service of an ordinary customer.The preempted ordinary customer is put at the head of the ordinary customer queue and has to wait until the priority queue is exhausted before he reenters service.The preempted customer resumes service from the point of interruption so that his service time upon reentry has been reduced by the amount of time the customer has already spent in service (cf.[Mil58, p. 1]).Since it is assumed that the service time is exponential, the ordinary customer requires on its reentry stochastically the same amount of service as it required on its earlier entry.Thus, the preemptive resume discipline is equal to the preemptive repeat-identical discipline where the preempted customer requires the same amount of service on its reentry as he required on his earlier entry (cf.[Jai68, p. 53]).
It is assumed that transmission times for orders are negligible and set to zero and that the transportation time between the production system and the inventory is negligible.All service times, inter-arrival times and replenishment lead times constitute an independent family of random variables.
A Markovian process description of the integrated queueing-inventory system is obtained as follows.Denote by X 1 (t) the number of priority customers present in the system at time t ≥ 0, and by X 2 (t) the number of ordinary customers in the system at time t ≥ 0. (We call X j (t) the queue length = number of respective customers either waiting or in service.)Since the customer in service will always be of the priority class when at least one priority customer is present, the value of the vector (X 1 (t), X 2 (t)) determines uniquely the type of the customer in service at time t ≥ 0, if any.By Y (t) we denote the on-hand inventory at time t ≥ 0. We define the joint queueing-inventory process of this system by Then, due to the usual independence and memoryless assumptions Z is a homogeneous Markov process.The state space of Z is where b is the maximal size of the inventory.Z has an infinitesimal generator Q = (q(z; z) : z, z ∈ E) with the following transition rates for (n 1 , n 2 , k) ∈ E: Furthermore, q(z; z) = 0 for any other pair z = z, and Definition 1.If the queueing-inventory process Z on state space E is ergodic we denote its limiting and stationary distribution by

Discussion of the modelling assumptions (1)
The assumption of identical service time distributions for customers of both classes is substantiated by Wang and coauthors [WBSW15] who identify three main motivations for prioritisation: • Different customers have different willingness to pay for the same product.
• Customers may require different products or services, where some of these products are more profitable than others.
• Different service levels may substantially affect long-term profitability.In our model we follow their conclusion: "Modelling the effects of prioritization due to the first and third motivations can be achieved with identical service time distributions for different segments."[WBSW15,p. 733] (2) Randomized differentiated admission control is incorporated in many models and is discussed indepth by Isotupa ([Iso06], [Iso07], [Iso11], [IS13], [Iso15]) in a sequence of papers.Isotupa investigates inventory systems with (r, Q)-and base stock policy for two classes of customers.Especially, the lost sales property for ordinary customers is of interest, e.g. for inventories of spare parts in the airline or shipping industries (cf.[Iso11,pp. 1f.]).Beside of the standard lost sales rule (all arriving customers are rejected independent of class) there is a modification such that during the time the inventory is equal to or less than a threshold level s ≤ Q arriving ordinary customers are lost.If the inventory level is zero, demands due to both types of customers are lost, cf.[Iso07], [Iso11], [Iso15].Our admission control is a modification of this scheme such that the strict rejection of ordinary customers below the control limit s is weakened by a randomized decision.
(3) Preemptive resume regime of priority customers over ordinary customers is a common scheme in queueing models, c.f. [Jai68].Coupling each service with obtaining an item from the inventory implies that an item dedicated at first to an ordinary customer can be attached to an interrupting priority customer as well.Here the exchangeble property is necessary.

Stability and stationary behaviour
In this section, we investigate stability of the queueing-inventory system.This especially means to find conditions on the system which guarantee that the system approaches in the long run a steady state (i.e.stabilises).For queueing systems with priority classes of customers this is a non-trivial problem because in general (e.g. in our setting of a queue without inventory) a solution of the global balance equations for the stationary distribution is not available and the standard ergodicity criterion for QBDs (Quasi-Birth-Death processes) is not applicable, see e.g.[LR99] In Section 4.1 we find natural conditions for Z to be ergodic by constructing a suitable Lyapunov function in the spirit of the classical Foster-Lyapunov stability criterion.Foster [Fos53] introduced a technique for proving stability of Markov chains which was generalised in several directions.Our proof relies on Kelly and Yudovina [KY14, Proposition D.3].
Proposition 1.Let X := (X(t) : t ≥ 0) be an irreducible regular Markov process with countable state space E and transition rate matrix Q := (q(x, y) : x, y ∈ E).Suppose that L : E → [0, ∞) is a function such that for constants ε > 0 and b ∈ R, and some finite exception set F ⊂ E and all x ∈ E holds Then X is ergodic.
Although we do not find a solution of the global balance equations for Z in Section 4.2, we present interesting results of the behaviour of Z in steady state.These results rely on the presence of partial balance properties inherent in the dynamics of the queueing-inventory system in equilibrium.Some of the resulting properties are surprising.

Ergodicity
Irreducibility of the state process Z can be seen directly from the transition rates at the end of Section 3. Our main result is the following theorem in the spirit of the Foster-Lyapunov stability criterion.
Proof.Positive recurrence will be shown by the Foster-Lyapunov stability criterion with L : E → R + 0 as Lyapunov function with where α : {0, 1, . . ., b} → [0, ∞) is strictly decreasing with and the finite exception set is given by Furthermore, we define and Due to the assumption, that µ > λ 1 + λ 2 it holds ε > 0.
First, we will check Second, we will check For k = 0, n 1 ∈ N and n 2 ∈ N 0 it holds
We do not expect that the result of the theorem is sharp for general p ∈ [0, 1], i.e. the condition λ 1 + λ 2 < µ is in general only sufficient for stability.We have only Theorem 3. Consider the system from Theorem 2 for p = 1, i.e. low priority customers are admitted as long as the inventory is not depleted.Then Z is ergodic if and only if λ 1 + λ 2 < µ holds.
The result of Theorem 3 at a first glance seems to be intuitive because no distinction is made between the arriving customers.The problem is that the influence of the interrupts of arrivals and service due to stockout is not accessible to intuition.The Theorem states especially that the interrupts of arrivals and service balance over time.The proof of Theorem 3 is postponed to the next section because we need some preparations which we consider to be of independent interest.

Properties of the stationary system
In this section, we assume that the queueing-inventory process Z is ergodic.So the limiting and stationary distribution π of Z (see Definiton 1) exists but seems to be not accessible directly.Nevertheless we are able to present results on the stationary behaviour of Z.We will exploit the structural information inherent in the global balanec equations of Z for π.
The global balance equations π • Q = 0 of the ergodic queueing-inventory process Z are for Let (X 1 , X 2 , Y ) be a random vector that is distributed according to the stationary distribution π.So, Y is distributed as the stationary distribution of the inventory process in equilibrium and X 1 resp.X 2 are respectively distributed as the queue length process of priority resp.ordinary customers in equilibrium.
Consider the queueing-inventory system with lead time zero for the replenishment orders.Then Y is constant b and the distribution of (X 1 , X 2 ) is the equilibrium of the priority system as described in Section 3 without inventory.A little reflection shows that the distribution of X 1 is geometrical with parameter λ 1 /µ, i.e. the priority customers are served as in a standard M/M/1/∞ with parameters λ 1 and µ. (Indeed, this observation applies to a classical M/M/c queue with two priority classes under a preemptive priority discipline as well.The priority customers behave as customers in a classic M/M/c queue (cf.[WBSW15]).) Our first proposition embeds a similar observation into the setting of the model in Section 3. The problem with our queueing-inventory system is that the two customer classes have to share the same inventory and therefore, the ordinary (= non-priority) customers impose restrictions on the priority customers' service because they consume items from the inventory and generate more stock-outs experienced by the priority customers.
Proposition 4. For the queueing-inventory system from Section 3 the stationary queue length of priority customers conditioned on positive inventory is geometrical with parameter λ 1 /µ, i.e. for n 1 ∈ N 0 holds with normalisation constant The proof of the proposition and some further properties presented below rely on partial balance relations inherent in the global balance equations of Z.
Lemma 5.For the queueing-inventory process Z holds All the equations can be proven by applying the cut-criterion for positive recurrent processes (see [Kel79,Lemma 1.4,p. 8]).This is Lemma 6.For ergodic Z with stationary distribution π it holds for any non-empty proper subset A ⊂ E z∈A z∈A c π(z)q(z, z) = z∈A c z∈A π(z)q(z, z) Proof of Lemma 5.For n 1 ∈ N 0 , equation (8) can be proven by a cut, which divides E into complementary sets according to the queue length of priority customers that is less than or equal to n 1 or greater than n 1 , i.e. into the sets and A c .Then for n 1 ∈ N 0 it holds by direct evaluation Hence, for n 1 ∈ N 0 it holds (8).For n 2 ∈ N 0 , equation (9) can be proven by a cut, which divides E into complementary sets according to the queue length of ordinary customers that is less than or equal to n 2 or greater than n 2 , i.e. into the sets Thus, for n 2 ∈ N 0 it holds (9).For n ∈ N 0 , equation (10) can be proven by a cut, which divides E into complementary sets according to the size of the total queue length that is less than or equal to n or greater than n, i.e. into the sets Thus, for n ∈ N 0 it holds (10).
Proof of Proposition 4. From (8) it follows by iteration for n Conditioning and exploiting λ 1 < µ for summation ends the proof.
A direct consequence of Lemma 5 are the following intuitive rate equations which equalize the mean admitted arrivals per customer classes to class dependent throughputs (= effective departure rates).The proof is in any case by direct summation of the respective formulas in Lemma 5.
Proposition 7.For the queueing-inventory process it holds the following equilibrium of probability flows Similar to the flow equations in Lemma 5 it is possible to derive flow equations with respect to the inventory level.The derivation is more tediuos and can be found in [Ott18, Proposition 11.1.6].The main result is that for k = 0, 1, . . ., b − 1 it holds Note that the statement of (15) exhibits an insensitivity property with respect to variation of the parameters of the system, more specifically it is independent of the threshold level s and the arrival intensities λ 1 and λ 2 .Remark 8. (1) The results in this section can be generalised in a direct way to the case of a system with C customer classes, where C = {1, . . ., C} is the set of customer classes.Customers of type c arrive with rate λ c > 0, a priority parameter p c (0 ≤ p c ≤ 1) and a threshold level s c , c ∈ C.
(2) In the models of Isotupa [Iso15], the replenishment rate depends on the number of pending orders.We can extend our model so that the replenishment lead time depends on the number of orders at the supplier.If there are k > 0 orders present at the supplier, the intensity of the replenishment lead time is ν(k) > 0. For more details see [Ott18,Chapter 11.1].
Proof of Theorem 3. Recall that the global balance equations for Z usually are stated with unknown positive x = (x(z) : z ∈ E) and that, in case of irreducibility (under Q) of E, Z is ergodic if and only if there exist a strictly positive solution x which is summable, i.e. z∈E x(z) < ∞.The first observation is that the partial balance relations of Lemma 5 hold for any solution x, even if the solution is not summable.We shall exploit equation (10) which reads in case of p = 1 for a general solution x(0, 0, k) • λ 1 + λ 2 µ n .

The case of instant service
In this section, we consider the case of instant service (zero production time), which turns the model into the formalism of classical inventory theory.The supply chain of interest is depicted in Figure 2 and consists of priority and ordinary customers, an inventory and a supplier.

Conclusion
We invenstigated ergodicity for queueing-inventory systems with two priority classes in case of unbounded queues for both customer classes.The admission control to the queues is flexible and incorporates two parameters (s, p) which mainly restrict entrance of low priority customers.The main result is a stability condition which is proved by construction of a suitable Lyapunov function.Although we proved that in case of parameter constellation (s, 1) the condition is sufficient and necessary we believe that in general the condition is not sharp.To clarify the situation in general is part of our ongoing research.

Figure 2 :
Figure 2: The pure inventory system with two customer classes