Generalized Gambler's Ruin Problem: explicit formulas via Siegumd duality

We give explicit formulas for ruin probabilities in a multidimensional Generalized Gambler's ruin problem. The generalization is best interpreted as a game of one player against $d$ other players, allowing arbitrary winning and losing probabilities (including ties) depending on the current fortune with particular player. It includes many previous other generalizations as special cases. Instead of usually utilized first-step-like analysis we involve dualities between Markov chains. We give general procedure for solving ruin-like problems utilizing Siegmund duality in Markov chains for partially ordered state spaces studied recently in context of M\"obius monotonicity.

The problem is as relevant today as it was in 17th century. According to [Edw83] Pascal was the first who posed a problem in 1656 in a letter to Fermat. The most common form comes from Huygens who restated the problem as follows (rephrased): "Let two men play with three dice, the first player scoring a point whenever 11 is thrown, whereas the second whenever 14 is thrown. Each player starts with 12 points. Successful roll adds one point to the player and subtracts one from the other player. The loser of the game is the first to reach zero points. What is the probability of victory for each player? " Huygens gave a solution of above problem. James Bernoulli [Ber13] in 1713 generalized and replaced Huygens' numerical results by formulas, i.e., he considered general initial capital i, general total amount of money N and general winning probabilities p ∈ (0, 1). Since then the problem became very popular and different proofs were obtained. Pascal gave his solution to Cercavi without mentioning the method. According to Edwards [Edw83] Fermat probably used the combinatorial argument (as in "Problem of points"), most methods which appeared later used some kind of first-step-like analysis which led to solving some recursions. It is known that the absorption probability of given chain can be related to the stationary distribution of some other ergodic chain. The relation is given via so-called Siegmund duality, the notion introduced in [Sie76]. It was studied in financial context, where the probability that a dual risk process starting at level h is ruined equals the probability that the stationary queue length exceeds level h, see [AA10], [AS09]. Already in [Lin52] such duality between some random walks on integers was shown. For this duality reader is also referred to [CR84], [DF90], [DFPS97] or [Hui10], just to mention few. All above papers have one thing in common: they study Siegmund duality defined for linear ordering of the state space (and most of them birth and death chains only). In this case [Sie76] states that the process has such dual if and only if it is stochastically monotone (w.r.t. total ordering). It is a little bit surprising that it was not exploited in the context of one-dimensional Gambler's ruin problem. The solutions of the classical problem and it's various one-dimensional generalizations are special cases of Theorem 1 and can be relatively easy calculated using usual stochastic monotonicity. On the other hand multidimensional case is quite different. For partial nonlinear ordering the stochastic monotonicity does not imply the existence of Siegmund dual, see [Lig04]. Finding such duals was successful for some specific chains and/or orderings. For example, in financial context, in [BS99] authors considered R d -valued Markov processes (their Siegmund dual was set-valued). Recently, [HM14] considered dualities for Markov chains on partitions and sets. In [Lor15] we show that Siegmund dual exists if and only if chain is Möbius monotone, the connections with Strong Stationary Duality (consult [DF90]) is also given therein. Let us mention at this point that for non-linear ordering Möbius and stochastic monotonicities are, in general, different. In particular, we can have a chain which is not stochastically monotone, but which is Möbius monotone, thus we are able to construct it's Siegmund dual.
In this paper, based on results from [Lor15], we give the solution to the multidimensional Generalized Gambler's ruin problem. The paper is organized as follows. In Section 2 we describe our Generalized Gambler's ruin problem, state it's solution (Theorem 1) and point out other results as some special cases. In Section 3 we recall notion of Siegmund duality and antiduality for chains on partially ordered state spaces and give a general recipe for calculating ruin-like probabilities (summarized in Theorem 2). Section 4 includes some toy example (Cat Eats Mouse Eats Cheese), where the case of negative antidual matrix is presented. Finally, Section 5 contains proof of Theorem 1.

Generalized Gambler's ruin problem and main result
In the one-dimensional Gambler's ruin problem two players start a game with total amount of, say, N dollars and initial values k and N − k. At each step they flip the coin (not necessary unbiased) to decide who wins a dollar. The game is over when one of them goes bankrupt.
We will consider the following generalization. There is one player (referred as "we") playing with d ≥ 1 other players. Our initial assets are (i 1 , . . . , i d ) and assets of consecutive players are (N 1 − i 1 , . . . , N d − i d ) (N j ≥ 1 is a total amount of assets with player j). Then, with probability p j (i j ) we win one dollar with player j and with probability q j (i j ) we lose it. With the remaining probability 1 − d k=1 (p k (i j ) + q j (i k )) we do nothing (i.e., ties are also possible). Once we win completely with player j (i.e., i j = N j ) we do not play with him/her anymore. We lose the whole game if we lose with at least one player, i.e., when i j = 0 for some j = 1, . . . , d.
For d = 1 and p 1 (j) = p, q 1 (j) = q we have the classical Gambler's ruin problem. Many one-dimensional generalizations of this game were considered, e.g., [Lef08] studied the case of some specific sequences of p 1 (j), q 1 (j), later this was extended to any p 1 (j), q 1 (j) in [ES09]. Variations of classical problem with ties allowed, i.e., p 1 (j) = p 1 , q 1 (j) = q 1 , p 1 + q 1 < 1 were considered in, e.g., [Len09a], [Len09b] (the latter one considers so called conditional version of the problem). Some of the articles studied both, the ruin probability and duration of the game, whereas most papers studied only duration of the game. Some generalizations to higher a dimension d ≥ 1 were studied in [RS04], [KP02].
We will describe the game more formally as a Markov chain Z ′ with two absorbing states. The state space is With some abuse of notation, we will sometimes write (i ′ 1 , . . . , i ′ d ) = −∞. The transitions of the described chain are following: The chain, as required, has two absorbing states: (N 1 , . . . , N d ) (we win) and −∞ (we lose). We will give formulas for the probabilities of winning starting at arbitrary state, i.e., for where τ e := inf{n ≥ 0 : Z n = e}. Our main result is following Theorem 1. Consider the generalized Gambler's ruin problem described above. Then, the probability of winning starting at (i 1 , . . . , i d ) is given by Theorem 1 generalizes some previous one-dimensional cases. For example: • (i) Assume we have won with all the players except player j. Then, the probability of winning is .
This way formula (4.1) from [ES09] is recovered (with some slight modification in notation, since authors considered a various versions of reflecting barriers). • (ii) In addition to (i), let p j (r) = p, q j (r) = q, r = 1, . . . , N j . Then we recover winning probability in the the classical Gambler's ruin problem (with possible ties): • (iii) (Homogeneity case). Assume that for all j = 1, . . . , d we have p j (r) = p j , q j (r) = q j , r = 1, . . . , N j . Define Then we have which is a multidimensional generalization of classical Gambler's ruin problem. Of course we obtain the same probabilities if only ratios pj (ij ) are constant, e.g., for the following spatially nonhomogeneous case which is thus a multidimensional generalization of cases considered in [ES09] and in [Lef08]. In the latter article only symmetric case corresponding to p j = q j = 1/2 was considered.

Tools: Siegmund duality and antiduality
We shortly recall notion of Siegmund duality, it's applications to studying absorption probabilities and result concerning existence of Siegmund dual from [Lor15]. Let X be a discrete-time Markov chain with transition matrix P X and finite state space E = {e 1 , . . . , e M } partially ordered by with unique minimal element e 1 and unique maximal element e M . Assume it is ergodic with the stationary distribution π. For A ⊆ E define P X (e, A) := e ′ ∈A P X (e, e ′ ) and similarly π(A) := e∈A π(e). Define also {e} ↑ := {e ′ ∈ E : e e ′ }, {e} ↓ := {e ′ ∈ E : e ′ e} and δ(e, e ′ ) = 1(e, e ′ ). We say that Markov chain Z with transition matrix P Z is the . Note that we can find a matrix fulfilling (3) which is substochastic, since we may have for some e j that ei P Z (e j , e i ) < 1. In a similar way as Siegmund [Sie76] did (he considered only linear ordering), we add then one extra absorbing state, say −∞ (called a coffin state). Denote the resulting matrix by P Z ′ and define P Z ′ (e j , −∞) = 1 − ei P Z ′ (e j , e i ), P Z (−∞, e j ) = δ(−∞, e j ) and P Z ′ (e, e 2 ) = P Z (e, e 2 ) otherwise. Note that (3) implies that e M is an absorbing state, thus Z ′ has two absorbing states. Taking limits as n → ∞ on both sides of (3) we have where τ e = inf{n : Z ′ n = e}. This way the stationary distribution of ergodic chain is related to the absorption of its Siegmund dual.
For partial ordering define C(e, e ′ ) = 1(e e ′ ). Such matrix is always invertible, and its inverse C −1 is often denoted by µ (what we use throughout the paper) and called the Möbius function of ordering . Note that (3) for n = 1 can be written as (5) P X C = CP T Z . The main result of [Lor15] is that for given partial ordering the Siegmund dual chain exists if and only if X is Möbius monotone (see also [LS12] for more details on this monotonicity). In such a case, the Siegmund dual on E ′ = E ∪ {−∞} has transitions outside coffin state given by (the nonnegativity of which is the definition of Möbius monotonicity of X). The natural application is in studying stationary distribution of a chain X (e.g., its asymptotics): calculate Siegmund dual and then its probability of being eventually absorbed in e M . However, we can reverse the process starting with a chain Z ′ with two absorbing states (we win or we lose). The procedure is then the following: 1) remove state −∞ obtaining substochastic matrix P Z ; 2) introduce some partial ordering expressed by matrix C such that e M is a unique maximal element; 3) calculate transitions of Siegmund antidual chain X from (5) calculating P X = CP T Z C −1 ; 4) if the resulting matrix P X has a stationary measure π such that ∀(e ∈ E) lim n→∞ P n X (e, ·) = π(·) then we can calculate absorption probabilities of Z ′ from the relation (4) (if P X is a stochastic matrix, then π is the stationary distribution of the chain related to this matrix).
The details are in the following theorem.
Theorem 2. Let Z ′ be a Markov chain with transition matrix P Z ′ on E ′ = {−∞}∪{e 1 , . . . , e M } =: {−∞}∪E with two absorbing states −∞ and e M . Consider substochastic kernel P Z which is the matrix P Z ′ with row and column corresponding to −∞ removed. Fix some partial ordering on E expressed by matrix C such that e M is a unique maximal state. Calculate The resulting P X has the property that ∀(e ∈ E) e2∈E P X (e, e 2 ) = 1. Assume that there exists invariant measure π fulfilling: ∀(e 2 ∈ E) lim n→∞ P n X (e 2 , e) = π(e), e∈E π(e) = 1.
Remark 3.1. If resulting P X in (7) is a stochastic matrix of ergodic chain, say X, then π is its stationary distribution. Moreover, it time reversal is Möbius monotone with respect to .
Remark 3.2. If the resulting P X has negative entries it does not have a real probabilistic interpretation. However, e.q., in area of quantum mechanics, such "distributions", called negative quasi-probabilities are quite common and natural in this context. This notion was already introduced in [Wig32], where author writes: "[...] cannot be really interpreted as the simultaneous probability for coordinates and momenta, as is clear from the fact, that it may take negative values. But of course this must not hinder the use of it in calculations as an auxiliary function which obeys many relations we would expect from such a probability. " For some recent connections of negative quasi-probability and quantum computations see [VFGE12].
Proof of Theorem 2. The main sketch of the proof was essentially given before the theorem. The only thing which may be not clear is that for all e we have In ( * ) we used the fact, that for any partial order with unique maximal element e M , the Möbius function fulfills ∀(e ∈ E) ej C −1 (e, e j ) = 1(e = e M ). To see this consider column of C −1 corresponding to state e M after applying first elementary column operation of Gauss-Jordan elimination.

Toy example: Cat Eats Mouse Eats Cheese
Before proceeding to the proof of the main result on generalized Gambler's ruin problem (i.e., Theorem 1) we give a 5-state example. The reason for this is that we wanted to present an example having the resulting matrix P X with negative entries. The example is taken from [Bre99] (Example 3.2 Cat Eats Mouse Eats Cheese, where the answer is easily calculated using first-step analysis): "A merry mouse moves in a maze. If it is at time n in a room with k adjacent rooms, it will be at time n + 1 in one of the k adjacent rooms, choosing one at random, each with probability 1 k . A fat lazy cat remains all the time in a given room, and a piece of cheese waits for the mouse in another room (see Fig. 1). The cat is not completely lazy: If the mouse enters the room inhabited by the cat, the cat will eat it. What is the probability that the mouse ever gets to eat the cheese when starting from room 1, the cat and the cheese being in rooms 3 and 5, respectively? " where P Z ′ is the original Cat Eats Mouse Eats Cheese matrix, P Z is the matrix with state 3 removed (E = {1, 2, 4, 5}) and C represents the ordering we introduced ordering with 5 being a maximal state. We are to calculate ρ(j) = P (τ 5 < τ 3 |Z ′ 0 = j), j = 1, 2, 4, 5.
These were the only nonzero entries of P X . Summarizing we have P X ((i 1 , . . . , i d ), (i ′ 1 , . . . , i ′ d )) = Assume that for all (i 1 , . . . , i d ) we have P X ((i 1 , . . . , i d ), (i 1 , . . . , i d )) ≥ 0. Then, these are the transitions of a closed network with d independent servers: being at state (i 1 , . . . , i d ) the arrival to server j is q j (i j ) and the departure is p j (i j − 1). Its stationary distribution is following However, if for some entry of P X is negative, then of course the matrix does not represent any Markov chain, but with π given in (8) we still have that πP X = π and lim n→∞ P n X ((i 1 , . . . , i d ), ·) = π(·) for every (i 1 , . . . , i d ) ∈ E. Thus, because of Theorem 2 equality (4) holds in any case and we obtain (2) what finishes the proof.