AutoCAT: Automated Product-Form Solution of Stochastic Models

Abstract

We introduce AutoCAT, a class of algorithms to automatically generate exact and approximate product-form solutions for large Markov processes that cannot be solved by direct numerical methods. Focusing on models that can be described as cooperating Markov processes, which include queueing networks and stochastic Petri nets as special cases, it is shown that finding a global optimum for a nonconvex quadratic program is sufficient for determining a product-form solution. Such problems are notoriously hard to solve due to the inherent difficulty of searching over nonconvex sets. Using a potential theory for Markov processes, convexification techniques, and a class of linear constraints that follow from stochastic characterization of product-form solutions, we obtain a family of linear programming relaxations that can be solved efficiently. A sequence of these linear programs is solved under increasingly tighter constraints to determine the exact product-form solution of a model when one exists. This approach is then extended to obtain approximate solutions for non-product-form models. Finally, our new techniques are validated with examples and increasingly complex case studies that show the effectiveness of the method on both conventional and novel performance models.

Appendix

Infinite Processes

Numerical optimization techniques generally require matrices of finite size. In both ENS and its relaxations, we therefore used exact or approximate aggregations to truncate the state spaces of any infinite processes. Let C + 1 be the maximum acceptable matrix order. Then we decompose the generator matrix and its equilibrium probability vector of an infinite process k as

$${\bf Q}_{k} = \left [\begin{array}{*{10}c} {\bf Q}_{k}^{C,C} & {\bf Q}_{k}^{C,\infty } \\ {\bf Q}_{k}^{\infty, C}&{\bf Q}_{k}^{\infty, \infty } \end{array} \right ],\quad {\boldsymbol \pi }_{k} = \left [\begin{array}{*{10}c} {\boldsymbol \pi }_{k}^{C},{\boldsymbol \pi }_{k}^{\infty } \end{array} \right ],$$

where Q _k ^C ₁, C ₂ is a C ₁ ×C ₂ matrix. Similar partitionings are also applied to the transition matrix L _k and to the rate matrices A _a and P _b , $a \in {\mathcal{A}}_{k},b \in {\mathcal{P}}_{k}$. We define the truncation such that the total probability mass in the first C states is 1 relative to the numerical tolerance of the optimizer, i.e., π _k ^∞ 1 < ε _tol . Notice that the latter condition can also be used to determine the ergodicity of the infinite process. Furthermore, from condition RC1 (respectively RC2) we need to account for the cases where passive (respectively active) actions associated with the first C states are only enabled in P _k ^C, ∞ (respectively only incoming from A _k ^∞, C). Such problems are easily handled by adding one fictitious state to the truncated set {1, 2, …, C}. For example, for A _a and P _b we consider the truncated matrices

$${\bf A}_{a} = \left [\begin{array}{*{10}c} {\bf A}_{a}^{C,C} &{\bf A}_{a}^{C,\infty }{\bf 1} \\ {\bf 1}^{T}{\bf A}_{a}^{\infty, C}& 0 \end{array} \right ],\;{\bf P}_{b} = \left [\begin{array}{*{10}c} {\bf P}_{b}^{C,C} &{\bf P}_{b}^{C,\infty }{\bf 1} \\ {\bf 1}^{T}{\bf P}_{b}^{\infty, C}& 0 \end{array} \right ],$$

where 1 is now an infinite column of 1s. Note that the fictitious state is excluded from the validation of conditions RC1 and RC2; thus the value of the diagonal rate on the last row is irrelevant with respect to finding a product form.

Finally, we comment on the choice of the parameter C for a given process k. Since this determines the number of states N _k for the truncated process, an optimal choice of this value can provide substantial computational savings. Let us first note that starting from a small C, it is easy to integrate additional constraints or potential vectors in the linear formulations for a value C ^′ > C. Recall that we propose in the rest of the paper a sequence of linear programs in order to obtain a feasible solution x. Then, if a linear program is infeasible, this can be due either to a lack of a product form or to a truncation where C is too small. The latter case can be readily diagnosed by adding slack variables, as in QCP, to the ergodicity condition and verifying if this is sufficient to restore feasibility. In such a case, the C value is updated to the smallest value such that feasibility is restored in the main linear program.

ZPR Example Model

$$\begin{array}{rcl} {\bf A}_{a}& = \left [\begin{array}{*{10}c} 0 & 0 & 0\\ 0 & 0 &0.2170 \\ 2.9105&2.2575& 0 \end{array} \right ]\qquad {\bf P}_{a} = \left [\begin{array}{*{10}c} 0&1&0&0\\ 0 &1 &0 &0 \\ 1&0&0&0\\ 1 &0 &0 &0 \end{array} \right ]& \\ {\bf A}_{b}& = \left [\begin{array}{*{10}c} 0 & 0 & 0 & 0\\ 5.65 & 0 &0.52 &2.13 \\ 0 &7.00& 0 & 0\\ 0 & 0 & 0 & 0 \end{array} \right ]\qquad {\bf P}_{b} = \left [\begin{array}{*{10}c} 0&1&0\\ 0 &1 &0 \\ 1&0&0 \end{array} \right ]& \\ {\bf L}_{m}& = \left [\begin{array}{*{10}c} 0 & 8 & 0 & 3\\ 6.15 & 0 &8.28 &7.67 \\ 15 &9.70& 0 & 0\\ 16 & 0 & 0 & 0 \end{array} \right ]\qquad {\bf L}_{k} = \left [\begin{array}{*{10}c} 0 & 0 & 0\\ 0 & 0 &3.78 \\ 3.09&2.74& 0 \end{array} \right ]& \\ \end{array}$$

Structural Approximation Pseudocode

273577_1_En_4_Figa_HTML.gif

G-Network Case Study

We report the parameters for the G-network given in [5]. The network consists of M = 10 queues with exponentially distributed service times having rates μ₁ = 4. 5 and ${\mu }_{i} = 4.0 + (0.1)i$ for i ∈ [2, 10]. The external arrival rate defines a Poisson process with rate λ = 5. 0. The routing matrix for (positive) customers has in row i and column j the probability r _i, j ⁺ of a (positive) customer being routed to queue j, as a positive customer, upon leaving queue i. In this case study, this routing matrix is given by

$${\bf R}^{+} = [{r}_{ i,j}^{+}] = \left [\begin{array}{*{10}c} 0 &0.2&0.3&0.4& 0 & 0 & 0 & 0 & 0 & 0\\ 0.1 & 0 & 0 & 0 &0.2 & 0 & 0 &0.2 & 0 & 0 \\ 0 & 0 & 0 & 0 &0.3&0.5&0.2& 0 & 0 & 0\\ 0.3 & 0 & 0 & 0 & 0 & 0 &0.7 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 &0.3 & 0 &0.5 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 &0.2 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right ].$$

Conversely, the probability r _i, j ⁻ of a customer leaving queue i and becoming a negative signal upon arrival at queue j is

$${\bf R}^{-} = [{r}_{ i,j}^{-}] = \left [\begin{array}{*{10}c} 0 &0&0&0&0.1& 0 & 0 & 0 & 0 & 0\\ 0 &0 &0 &0 & 0 &0.4 & 0 & 0 & 0 & 0 \\ 0 &0&0&0& 0 & 0 & 0 & 0 & 0 & 0\\ 0 &0 &0 &0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 &0&0&0& 0 & 0 & 0 & 0 & 0 & 0\\ 0 &0 &0 &0 & 0 & 0 & 0 &0.1 & 0 &0.1 \\ 0 &0&0&0& 0 & 0 & 0 & 0 & 0 & 0\\ 0 &0 &0 &0 &0.1 & 0 & 0 & 0 & 0 & 0 \\ 0.1&0&0&0& 0 & 0 & 0 & 0 & 0 & 0\\ 0 &0 &0 &0 & 0 & 0 &0.2 & 0 &0.05 & 0 \end{array} \right ].$$

Loss and BBS Models

The model is composed of M = 5 queues that cooperate on a set of A = 12 actions, one for each possible job movement from and inside the network. The routing probabilities R[k, j] from queue k to queue j are as follows:

$$R = \left [\begin{array}{*{10}c} 0.16& 0 &0.04&0.50&0.30\\ 0.08 &0.29 &0.02 &0.08 &0.52 \\ 0 & 0 &0.78& 0 & 0\\ 0.29 &0.24 & 0 &0.25 &0.22 \\ 0 &0.49& 0 &0.20& 0 \end{array} \right ].$$

Service times are exponential at all queues with rates mu _k  = k, k = , 1…, 5. For a queue k, the probability of departing from the network is ${r}_{k,0} = 1 -{\sum \nolimits }_{j=1}^{5}R[k,j]$. The Poisson arrival rates from the outside world are given by the vector

$${\bf \lambda} = (0.6600,0.1500,0.0750,0.1650,0.4500).$$