Asymptotic Expansions for Stationary Distributions of Nonlinearly Perturbed Semi-Markov Processes. II

Asymptotic expansions with explicit upper bounds for remainders are given for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces. The corresponding algorithms are based on a special technique of sequential phase space reduction, which can be applied to processes with an arbitrary asymptotic communicative structure of phase spaces.


Introduction
In this paper, we present new algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with a finite phase space. This is Part II of the paper, Silvestrov, D. and Silvestrov, S. (2016), where algorithms for constructing of asymptotic expansions with remainders of a standard form o(·) have been given. In Part II, we present algorithms for construction asymptotic expansions of a more advanced form, with explicit upper bounds for remainders.
We consider models, where the phase space is one class of communicative states, for embedded Markov chains of pre-limiting perturbed semi-Markov processes, while it can possess an arbitrary communicative structure, i.e., can consist of one or several closed classes of communicative states and, possibly, a class of transient states, for the limiting embedded Markov chain.
The initial perturbation conditions are formulated in the forms of Taylor and Laurent asymptotic expansions with explicit upper bounds for remainders, respectively, for transition probabilities (of embedded Markov chains) and expectations of sojourn times, for perturbed semi-Markov processes.
The algorithms are based on special time-space screening procedures for sequential phase space reduction and algorithms for re-calculation of asymptotic expansions with explicit upper bounds for remainders, which constitute perturbation conditions for the semi-Markov processes with reduced phase spaces.
The final asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are given in the form of Taylor asymptotic expansions, with explicit upper bounds for remainders.
The algorithms presented in the paper have an universal character. They can be applied to perturbed semi-Markov processes with an arbitrary asymptotic communicative structure and are computationally effective due to recurrent character of computational procedures.
The survey of works in the area and detailed comments concerned with the proposed method are given in the Introduction to Part I of this paper.
A comprehensive bibliography of works in the area can be found in these books and, also, in the research report by Silvestrov, D. and Silvestrov, S. (2015), which is an extended preliminary version of the present paper.
In conclusion, we would like to mention that, by our opinion, the results presented in the paper have a good potential for continuation of studies. We comment some prospective directions for future studies in the last section of the paper.
Part II includes four sections and two appendices. In Section 2, we present so-called operational rules for Laurent asymptotic expansions with explicit upper bounds for remainders. In Section 3, we present basic perturbation conditions and algorithms for construction of asymptotic expansions with explicit upper bounds for remainders, for transition characteristics of nonlinearly perturbed semi-Markov processes with reduced phase spaces. In Section 4, we present algorithms for construction of asymptotic expansions with explicit upper bounds for remainders, for expected hitting times for nonlinearly perturbed semi-Markov processes. In Section 5, we present an algorithm for construction of asymptotic expansions with explicit upper bounds for remainders, for stationary distributions of nonlinearly perturbed semi-Markov processes. In Appendix A, we give proofs of lemmas representing operational rules for Laurent asymptotic expansions without and with explicit upper bounds for remainders. In Appendix B, we discuss and present examples illustrating algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes developed in the present paper.

Laurent asymptotic expansions with explicit upper bounds for remainders
In this section, we present so-called operational rules for Laurent asymptotic expansions with explicit upper bounds for remainders. The corresponding proofs and comments are given in Appendix A.
Let A(ε) be a real-valued function defined on an interval (0, ε 0 ], for some 0 < ε 0 ≤ 1, and given on this interval by a Laurent asymptotic expansion, where (a) −∞ < h A ≤ k A < ∞ are integers, (b) coefficients a h A , . . . , a k A are real numbers, We refer to such Laurent asymptotic expansion as a (h A , k A , δ A , G A , ε A )expansion.
The (h A , k A , δ A , G A , ε A )-expansion is also a (h A , k A )-expansion, according the definition given in Part I of the paper, since, o A (ε k A )/ε k A → 0 as ε → 0.
We say that (h A , k A , δ A , G A , ε A )-expansion A(ε) is pivotal if it is known that a h A = 0.
It is useful to note that there is no sense to consider, it seems, a more general case of upper bounds for the remainder o A (ε k A ), with parameter δ A >

Indeed, let us define k ′
The above remarks imply that the asymptotic expansion A(ε) can be represented in different forms. In such cases, we consider forms with larger parameters h A and k A as more informative. As far as parameters δ A , G A and ε A are concerned, we consider as more informative forms, first, with larger values of parameter δ A , second, with smaller values of parameter G A and, third, with larger values of parameter ε A .
In what follows, lemmas, theorems and relations from Part I of the paper are indexed by symbol * .
The following proposition supplements Lemma 1 * .
Let us consider four Laurent asymptotic expansions, The following Lemma presents operational rules for computing parameters of upper bounds for remainders of Laurent asymptotic expansions.
Lemma 2. The above asymptotic expansions have the following operational rules for computing remainders: with parameters h C , k C and coefficients c r , r = h C , . . . , k C given in proposition (i) of Lemma 2 * , and parameters δ C , G C and ε C given by the formulas: with parameters h C , k C and coefficients c r , r = h C , . . . , k C given in proposition (ii) of Lemma 2 * , and parameters δ C , G C and ε C given by formulas: is a (h C , k C , δ C , G C , ε C )-expansion with parameters h C , k C and coefficients c r , r = h C , . . . , k C given in proposition (iii) of Lemma 2 * , and parameters δ C , G C and ε C given by formulas: coefficients c r , r = h C , . . . , k C given in proposition (iv) of Lemma 2 * , and parameters δ C , G C and ε C given formulas: expansion with parameters h D , k D and coefficients d r , r = h D , . . . , k D given in proposition (v) of Lemma 2 * , and parameters δ D , G D , ε D given by formulas: where coefficients c r , r = h C , . . . , k C and parameters h C , k C , δ C , G C , ε C are given for the (h C , k C , δ C , G C , ε C )-expansion of function C(ε) = 1 B(ε) in the above proposition (iv), or by formulas: Remark 1. Coefficients ε B , ε C , ε D ∈ (0, 1] are taken to nonnegative powers in all terms penetrating the sums, which define parameters G C , G D andε B , in Lemma 2.
The following operational rules for computing remainders for multiple summation and multiplication of Laurent asymptotic expansions, used in what follows, are analogues of the corresponding summation and multiplication rules given in Lemma 2.
The summation and multiplication rules for computing of upper bounds for remainders given in propositions (ii) and (iii) of Lemma 2 possess the communicative property, but do not possess the associative and distributional properties.
Lemma 2 let us get an effective low bound for parameter δ A for any (h A , k A , δ A , G A , ε A )-expansion A(ε) obtained as the result of a finite sequence of operations (described in Lemma 2) performed over expansions from some finite set of such expansions.
The following lemma summarize these properties of Laurent asymptotic expansions with explicit upper bounds for remainders.
Lemma 4. The summation and multiplication operations for Laurent asymptotic expansions defined in Lemma 2 possess the following algebraic properties, which should be understood as equalities for the corresponding parameters of upper bounds for their remainders: -expansion obtained as the result of a finite sequence of operations (multiplication by a constant, summation, multiplication, and division) performed over . . , N, according the rules presented in Lemmas 2 * and 2, then

Asymptotic expansions for transition characteristics of nonlinearly perturbed semi-Markov processes with reduced phase spaces
Let us recall the perturbed semi-Markov processes η (ε) (t), t ≥ 0, with phase space X = {1, . . . , N} and transition probabilities Q ij (t), t ≥ 0, i, j ∈ X, introduced in Part I of the paper. These processes depend on a perturbation parameter ε ∈ (0, ε 0 ], for some 0 < ε 0 ≤ 1. We also recall transition probabilities of the corresponding embedded Markov chains, p ij (ε) = Q (ε) ij (∞), i, j ∈ X, and expectations of sojourn times e ij (ε) = ∞ 0 tQ We assume that condition A, introduced in Part I, holds for semi-Markov processes η (ε) (t). In particular, we recall the transition sets Y i , i ∈ X (which include states j ∈ X with non-zero probabilities p ij (ε) and guarantee ergodicity of the processes η (ε) (t)) introduced in this condition.
However, we replace the perturbation condition D by the following stronger condition, in which the corresponding Taylor asymptotic expansions are given in the form with explicit upper bounds for remainders: Also, we replace the perturbation condition E by the following stronger condition, in which the corresponding Laurent asymptotic expansions are given in the form with explicit upper bounds for remainders: As was pointed out in Part I, condition A implies that sets Y + rr = Y rr \ {r} = ∅, r ∈ X and the non-absorption probabilityp rr (ε) = 1−p rr (ε) ∈ (0, 1], for r ∈ X, ε ∈ (0, ε 0 ]. This probability satisfy the following relation, for every The above relation let us construct an algorithm for getting asymptotic expansions with explicit upper bounds for remainders, for non-absorption probabilitiesp rr (ε).
Lemma 5. Let conditions A and D ′ hold. Then, for every r ∈ X, the pivotal (l − rr ,l + rr )-expansion for the non-absorption probabilityp rr (ε) given in Lemma 8 * is, also, a (l − rr ,l + rr ,δ rr ,Ḡ rr ,ε rr )-expansion, with parametersδ rr ,Ḡ rr andε rr , which can be computed according the algorithm described below, in the proof of the lemma.
Proof. Let r ∈ Y r . First, propositions (i) of Lemmas 3 * and 3 (the multiple summation rule) should be applied to the sum j∈Y + rr p rj (ε). Second, propositions (i) (the multiplication by constant −1) and (ii) (the summation with constant 1) of Lemmas 2 * and 2 should be applied to the asymptotic expansion for probability p rr (ε) given in condition D ′ , in order to get the asymptotic expansion for function 1 − p rr (ε). Third, Lemmas 1 * and 1 should be applied to the asymptotic expansion for functionp rr (ε) given in two alternative forms by relation (2). This yields the corresponding pivotal the (l − rr ,l + rr )-expansion for probabilitiesp rr (ε), given in Lemma 8 * , and proves that this expansion is a (l − rr ,l + rr ,δ rr ,Ḡ rr ,ε rr )-expansion, with parameters computed in the process of realization of the above algorithm. The case r / ∈ Y r is trivial, since, in this case, probabilityp rr (ε) ≡ 1. Let us recall formula (19) * for the transition probabilities r p ij (ε), i, j ∈ r X = X \ {r} of the reduced embedded Markov chain r η (ε) n , introduced in Part I, Let us introduce parameter, Obviously, inequalities δ ij ≥ δ • , j ∈ Y i , i ∈ X hold for parameters δ ij penetrating upper bounds for the remainders of asymptotic expansions in condition D ′ . Theorem 1. Conditions A and D ′ , assumed to hold for the Markov chains η (ε) n , also hold for the reduced Markov chains r η (ε) n , for every r ∈ X. Also, for every j ∈ r Y i , i ∈ r X, r ∈ X, the pivotal ( r l − ij , r l + ij )-expansion for the transition probability r p ij (ε) given in Theorem 2 * is a ( r l − ij , r l + ij , r δ ij , r G ij , r ε ij )-expansion penetrating condition D ′ for the Markov chains r η (ε) n . Parameters r δ ij , r G ij and r ε ij can be computed using the algorithm described below, in the proof of the theorem. The inequalities Proof. Condition A holds for the Markov chains r η (ε) n by Lemma 6 * , with the same parameter ε 0 as for the Markov chains η (ε) n and with the transition sets r Y i , i ∈ r X given by relation (20) * .
Let us prove that condition D ′ holds for the Markov chains r η (ε) n , with the same parameter ε 0 as for the Markov chains η (ε) n and the transition sets (v) (the division rule) of Lemmas 2 * and 2 should be applied to the quotient p rj (ε) 1−prr(ε) . Second, propositions (iii) (the multiplication rule) of Lemmas 2 * and 2 should be applied to the product p ir (ε) · p rj (ε) 1−prr(ε) . Third, propositions (ii) (the summation rule) of Lemmas 2 * and 2 should be applied to sum r p ij (ε) = p ij (ε) + p ir (ε) · p rj (ε) 1−prr(ε) . The asymptotic expansions for probabilities p ir (ε), p rj (ε), and p ij (ε), given in condition D ′ , and probability 1 − p rr (ε), given in Lemmas 8 * and 5, should be used. This yields the corresponding pivotal ( r l − ij , r l + ij )-expansions for transition probabilities r p ij (ε), j ∈ r Y i , i ∈ r X, r ∈ X, given in Theorem 2 * , and proves that these expansions are ( r l − ij , r l + ij , r δ ij , r G ij , r ε ij )-expansions, with parameters computed in the process of realization of the above algorithm.
In these cases, the above algorithm is readily simplified. Thus, condition D ′ holds for the reduced Markov chains r η Let us recall formula (22) * for the expectations of sojourn times r e ij (ε), for i, j ∈ r X = X \ {r} for the reduced semi-Markov process r η (ε) (t), introduced in Part I, Let us introduce parameter, Obviously, inequalities δ ij ,δ ij ≥ δ * , j ∈ Y i , i ∈ X hold for parameters δ ij andδ ij penetrating upper bounds for the remainders of asymptotic expansions in conditions D ′ and E ′ .
Theorem 2. Conditions A -C, D ′ and E ′ , assumed to hold for the semi-Markov processes η (ε) (t), also hold for the reduced semi-Markov processes r η (ε) (t), for every r ∈ X. Also, for every j ∈ r Y i , i ∈ r X, r ∈ X, the pivotal ( r m − ij , r m + ij )-expansion for expectation r e ij (ε) given in Theorem 3 * is a ( r m − ij , r m + ij , rδij , rĠij , rεij )-expansion penetrating condition E ′ for the semi-Markov processes r η (ε) (t). Parameters rδij , rĠij and rεij can be computed using the algorithm described below, in the proof of the theorem. The Proof. Conditions A and D ′ hold for the semi-Markov processes r η (ε) (t), respectively, by Lemma 6 * and Theorem 1, with the same parameter ε 0 as for the semi-Markov processes η (ε) (t), and the transition sets r Y i , i ∈ r X given by relation (20) * . Also conditions B and C hold for processes r η (ε) (t), by Lemma 7 * .
It worth to note that, despite bulky forms, formulas for parameters of upper bounds for remainders, in the asymptotic expansions given in Lemma 5 and Theorems 1 and 2, are computationally effective.

Asymptotic expansions for expected hitting times with explicit upper bounds for remainders
As in Part I, letr i,N = r i,1 , . . . , r i,N = r i,1 , . . . , r i,N −1 , i be a permutation of the sequence 1, . . . , N such that r i,N = i, and letr i,n = r i,1 , . . . , r i,n , n = 1, . . . , N be the corresponding chain of growing sequences of states from space X.
Theorem 3. Let conditions A -C, D ′ and E ′ hold for the semi-Markov processes η (ε) (t). Then, for every i ∈ X, the pivotal (M − ii , M + ii )-expansion for the expectation of hitting time E ii (ε), given in Theorem 4 * and obtained as the result of sequential exclusion of states r i,1 , . . . , ii can be computed using the algorithm described below, in the proof of the theorem. Also, inequalitȳ Proof. Let us assume that p the initial semi-Markov process. Let us exclude state r i,1 from the phase space of semi-Markov processr i,0 η (ε) (t) using the time-space screening procedure described in Section 5 * . Letr i,1 η (ε) (t) be the corresponding reduced semi-Markov process. The above procedure can be repeated. The state r i,2 can be excluded from the phase space of the semi-Markov processr i,1 η (ε) (t). Let r i,2 η (ε) (t) be the corresponding reduced semi-Markov process. By continuing the above procedure for states r i,3 , . . . , r i,n , we construct the reduced semi-Markov processr i,n η (ε) (t).
The algorithm described above can be repeated, for every i ∈ X.
It is worth to note that the algorithms based on sequential exclusion of states from the phase space of perturbed semi-Markov processes make it possible to get Laurent asymptotic expansions (without and with explicit upper bounds for remainders) for expected hitting times, for nonlinearly perturbed semi-Markov processes. Such asymptotic results have their own important value.
By applying the algorithm of sequential phase space reduction described in Theorem 4 to the above sequence of statesr i,j,N −2 , we construct the reduced semi-Markov processr i,j,N−2 η (ε) (t). This process has the phase spacē r i,j,N−2 X = X ij = {i, j}, which is a two-states set. The transition probabili-ties of the embedded Markov chainr i,j,N−2 p i ′ j ′ (ε) = p ij,i ′ j ′ (ε), i ′ , j ′ ∈ X ij , the expectations of sojourn timesr i,j,N−2 e i ′ j ′ (ε) = e ij,i ′ j ′ (ε), i ′ , j ′ ∈ X ij , and the transition setsr i,j,N−2 Y i ′ = Y ij,i ′ , i ′ ∈ X ij can be found using the recurrent algorithm described in Theorem 4 * . These probabilities, expectations and transition sets are invariant to any permutationr ′ i,j,N −2 of sequencer i,j,N −2 . This legitimates the above alternative simplified notations.
Theorem 4 * let us construct the pivotal upper bounds for the corresponding remainders.
By Theorem 1 * , the expectation of hitting time E i ′ ,j ′ (ε) coincides for the initial semi-Markov processes η (ε) (t) and the reduced semi-Markov process r i,j,N−2 η (ε) (t), for every i ′ , j ′ ∈ X ij . This obviously implies that these expectations are also invariant to any permutationr ′ i,j,N −2 of sequencer i,j,N −2 . It is easy to write down the formulas for the above expectations, for the two-states semi-Markov processr i,j,N−2 η (ε) (t). These formulas are, Under the assumption that conditions of Theorem 4 * hold, the operational rules given in Lemma 2 * can be applied to functions E i ′ j ′ (ε), i ′ , j ′ ∈ X ij , in order to get the corresponding (M − i ′ j ′ , M + i ′ j ′ )-expansions. These expansions are invariant to any permutationr ′ i,j,N −2 of sequencer i,j,N −2 , used in the corresponding recurrent algorithm based on sequential exclusion states r i,j,1 , . . . , r i,j,N −2 from the phase space X.
Finally, under the assumption that conditions of Theorem 3 hold, the operational rules given in Lemma 2 can be applied, in order to prove that the above ( -expansions, and to compute parametersr i,jN−2δ i ′ j ′ ,r i,j,N−2Ġ i ′ j ′ and r i,j,N−2ε i ′ j ′ of upper bounds for the corresponding remainders. Also, by Lemma 4, the inequalityr i,jN−2δ i ′ j ′ ≥ δ * holds, for every i ′ , j ′ ∈ X ij , sequencer i,jN −2 , and i, j ∈ X.

Asymptotic expansions for stationary distributions with explicit upper bounds for remainders
Let us recall the pivotal (n − i , n + i )-expansion for stationary probability π i (ε) of nonlinearly perturbed semi-Markov process η (ε) (t) given, under conditions A -E, in Theorem 5 * . This asymptotic expansion has the following form, for i ∈ X, According Theorem 5 * , the above asymptotic expansion is invariant with respect to the choice of sequence statesr i,N −1 = (r i,1 , . . . , r i,N −1 ) used in the corresponding algorithm, for every i ∈ X.
The following theorem is the main new result in Part II of this paper. Theorem 4. Let conditions A -C, D ′ and E ′ hold for the semi-Markov processes η (ε) (t). Then, for every i ∈ X, the pivotal (n − i , n + i )expansion (7) for the stationary probability π i (ε), given in Theorem 5 * and obtained as the result of sequential exclusion of states r i,1 , . . . , r i,N −1 from the phase space X of the processes η (ε) (t), is a (n − i , n + i ,r i,N−1 δ i ,r i,N−1 G i ,r i,N−1 ε i )expansion. Parametersr i,N−1 δ i ,r i,N−1 G i andr i,N−1 ε i can be computed using the algorithm described below, in the proof of the theorem. Also, inequalityr i,N−1 δ * i ≥ δ * holds making it possible to rewrite function π i (ε) as the Proof. Let us choose an arbitrary state i ∈ X. First, proposition (i) (the multiple summation rule) of Lemmas 3 * and 3 should be applied to the pivotal (m − i , m + i )-expansion for the expectation e i (ε) = j∈Y i e ij (ε) given by relation (29) * , in the proof of Theorem 5 * . This yields a (m − i , m + i ,δ i ,Ġ i ,ε i )expansion for the expectation e i (ε), with the corresponding parametersδ i ,Ġ i andε * i . Second, the propositions (v) (the division rule) of Lemmas 2 * and 2 should be applied to the quotient π i (ε) = e i (ε) E ii (ε) . The (m − i , m + i ,δ i ,Ġ i ,ε i )expansion for the expectation e i (ε) and the (M − ii , M + ii ,r i,N−1δ ii ,r i,N−1Ġ ii , r i,N−1ε ii )-expansion for the expectation of hitting time E ii (ε), given in Theorems 4 * and 3, should be used. This yields the corresponding pivotal (n − i , n + i )expansion for stationary probability π i (ε), given in Theorem 5 * , and proves that this expansion is a (n − i , n + i ,r i,N−1 δ i ,r i,N−1 G i ,r i,N−1 ε i )-expansion, with parameters computed in the process of realization of the above algorithm. Inequalityr i,N−1 δ i ≥ δ * holds, for every sequencer i,N −1 , by proposition (iii) of Lemma 4.
The explicit upper bounds for remainders in the asymptotic expansions given in Theorem 4 have a clear and informative power-type form. An useful property of these upper bounds is that they are uniform with respect to the perturbation parameter. The recurrent algorithm for finding these upper bounds is computationally effective.
Unfortunately, the summation and multiplication operational rules for computing power-type upper bounds for remainders possess commutative but do not possess associative and distributive properties. This causes dependence of the resulting upper bounds for remainders in the asymptotic expansions for stationary probabilities π i (ε), i ∈ X on a choice of the corresponding sequences of statesr i,N −1 = r i,1 , . . . , r i,N −1 , i ∈ X used in the above algorithm. This rises two open questions, the first one, about possible alternative forms for remainders possessing the desirable algebraic properties mentioned above, and, the second one, about an optimal choice of sequences of statesr i,N −1 , i ∈ X.
In conclusion, we would like to mention some prospective directions for future research studies.
The method of sequential reduction of phase space presented in the paper can be applied for getting asymptotic expansions for high order power and exponential moments of hitting times, for nonlinearly perturbed semi-Markov processes. This is an interesting problem, which has its own important theoretical and applied values.
We are quite sure that a combination of results in the above direction with the methods of asymptotic analysis for nonlinearly perturbed regenerative processes developed and throughly presented in Gyllenberg and Silvestrov (2008) will make it possible to expand results from this book, related to asymptotic expansions for stationary and more general quasi-stationary distributions as well as other characteristics for nonlinearly perturbed semi-Markov processes with absorption, to nonlinearly perturbed semi-Markov processes with an arbitrary asymptotic communicative structure of phase spaces.
The problems of aggregation of steps in the time-space screening procedures for semi-Markov processes, tracing pivotal orders for different groups of states as well as getting explicit formulas, for coefficients and parameters of upper bounds for remainders in the corresponding asymptotic expansions for stationary distributions and moments of hitting times, do require additional studies. It can be expected that such formulas can be obtained, for example, for nonlinearly perturbed birth-death-type semi-Markov processes, for which the proposed algorithms of phase space reduction preserve the birth-death structure for reduced semi-Markov processes.
Applications to control and queuing systems, information networks, epidemic models and models of mathematical genetics and population dynamics, analogous to those presented in the books cited in the introduction, also create a prospective area for future research based on the asymptotic results obtained in the present paper.

Appendix A: Operational rules for Laurent asymptotic expansions
Let us give short proofs of Lemmas 1 * -4 * and 1 -4 omitting some known or obvious details.
A.1. The formulas given in Lemmas 1 * and 1 are quite obvious.
A.2. The same relates to formulas in propositions (i) (the multiplication by a constant rule) of Lemmas 2 * and 2.
Proposition (ii) (the summation rules) of Lemmas 2 * and 2 can be obtained by simple accumulation of coefficients for different powers of ε and terms accumulated in the corresponding remainders, and, then, by using obvious upper bounds for absolute values of sums of terms accumulated in the corresponding remainders.
Proposition (iii) (the multiplication rule) of Lemma 2 * can be proved by multiplication of the corresponding asymptotic expansions A(ε) and B(ε) and accumulation of coefficients for powers ε l for l = h C , . . . , k C in their product, where Obviously, o C (ε k C ) ε k C → 0 as ε → 0. It should be noted that the accumulation of coefficients for powers ε l can be made in (8) only up to the maximal value Also, relation (9) readily implies relations (a) -(c), which determines parameters δ C , G C , ε C in proposition (iii) of Lemma 2.
The assumptions of proposition (iv) in Lemma 2 * imply that ε −h B B(ε) → b h B = 0 as ε → 0. This relation implies that there exists 0 < ε ′ 0 ≤ ε 0 such that B(ε) = 0 for ε ∈ (0, ε ′ 0 ], and, thus, function h B , or, equivalently, that the following representation takes place, The latter two relations prove proposition (iv) of Lemma 2 * , for the case h B = k B . Indeed, these relations mean that function C(ε) = 1 B(ε) can be represented in the form of (h C , k C )-expansion with parameters Moreover, since B(ε) · C(ε) ≡ 1, 0 < ε ≤ ε ′ 0 , remainder c 1 (ε) can be found from the following . This is formula (c) from proposition (iv) of Lemma 2 * , for the case h B = k B . Note that, in the case h B = k B , the above asymptotic expansion for function C(ε) can not be extended. Indeed, ε on the right hand side in the latter relation has an uncertain asymptotic behavior as ε → 0.
Let us now assume that h B + 1 ≤ k B . In this case, the assumptions of proposition (iv) of Lemma 2 * and the above asymptotic relations imply or, equivalently, that the following representation takes place, → 0 as ε → 0. The latter two relations prove proposition (iv) of Lemmas 2 * , for the case h B + 1 = k B . Indeed, these relations mean that function C(ε) can be represented in the form of (h C , k C )-expansion with parameters h C = −h B , Moreover, since B(ε) · C(ε) ≡ 1, the remainder o 2 (ε −h B +1 ) can be found from the following relation, . This is formula (c) from proposition (iv) of Lemma 2 * , for the case h B + 1 = k B . Note that, in the case h B + 1 = k B , the above asymptotic expansion for function C(ε) can not be extended.
on the right hand side in the latter relation has an uncertain asymptotic behavior as ε → 0.
We can repeat the above arguments for the general case h B + n = k B , for any n = 0, 1, . . . and to prove that, in the case h B + n = k B , function C(ε) can be represented in the form of (h C , k C )-expansion with parameters h C = −h B , k C = k B − 2h B = −h B + n = h C + n and coefficients c h C , . . . , c k C given in proposition (iv) of Lemma 2 * . Moreover, identity B(ε) · C(ε) ≡ 1, 0 < ε ≤ ε ′ 0 , let us find the corresponding remainder o C (ε k C ) from the following relation, Proposition (iii) of Lemma 2 * , applied to the product on the left hand side in relation (10), permits to represent this product in the form of (h, k)- By canceling coefficient for ε l on the left and right hand sides in relation (10), for l = 0, . . . , k B − h B , and then, by solving equation (10) with respect to the remainder o C (ε k C ), we get the formula for this remainder given in proposition (iv) of Lemma 2 * , The assumptions made in proposition (iv) of Lemma 2, imply that B(ε) = 0 and the following inequality holds for 0 < ε ≤ ε C , where ε C is given in proposition (iv) of Lemma 2, The existence of ε ′ 0 declared in proposition (iv) of Lemma 2 is obvious. For example, one can choose ε ′ 0 = ε C . It is also useful to note that formulas given in proposition (iv) of Lemma 2 imply that ε C = ε B ∧ε B ∈ (0, ε 0 ], since ε B ∈ (0, ε 0 ] andε B ∈ (0, ∞).
The assumptions made in proposition (iv) of Lemma 2 and inequality (12) imply that the following inequality holds, for 0 < ε ≤ ε C , Inequality (13) proofs proposition (iv) of Lemma 2. Propositions (v) of Lemmas 2 * and 2 and relations (a) -(c) given in these propositions can be obtained by direct application, respectively, of propositions (iii) and (iv) of Lemmas 2 * and 2, to the product D(ε) = A(ε) · 1 B(ε) . Now, when it is already known that By equating coefficients for powers ε l for l = h D , . . . , k D on the left and right hand sides of the third equality in relation (14), we get alternative formulas (e) for coefficients d h d , . . . , d k D given in proposition (v) of Lemma 2 * .
Proposition (iii) of Lemma 2, applied to the product on the right hand side in (14), permits to represent this product in the form of (h, k)-expansion By canceling coefficient for ε l on the left and right hand sides in relation (14), for l = h A , . . . , k A ∧ (k B + h A − h B ), and then, by solving equation (14) with respect to the remainder o D (ε k D ), we get the formula (f) for this remainder given in proposition (v) of Lemma 2 * , Inequality (12) and the assumptions made in proposition (v) of Lemma 2 finally imply that the following inequality holds, for 0 < ε ≤ ε D given in relation (f) of this proposition, Inequality (16)  A.4. The first two identities for Laurent asymptotic expansions given in proposition (i) of Lemma 4 * are obvious. The third identity given in this proposition follows in an obvious way from proposition (i) of Lemma 2 * . By applying propositions (iii) and (iv) of Lemma 2 * to the product and coefficients c n = I(n = 0), n = 0, . . . , k C . Also, relations (10) and (11) imply that the elimination identity A(ε) · A(ε) −1 ≡ 1 holds, since the remainder of Laurent asymptotic expansion for function A(ε) −1 is given by formula (c) from proposition (iv) of Lemma 2 * .
Propositions (ii) and (iii) of Lemma 4 * in the parts concerned commutative property of summation and multiplication operations follow from, respectively, propositions (ii) and (iii) of Lemma 2 * .

Appendix B: Examples
Let us, first, comment some general questions connected with construction of examples illustrating the asymptotic results presented in the paper.
Let Y i = ∅, i ∈ X be some subsets of space X such that condition A (c) holds for these sets, i.e., for every pair of states i, j ∈ X, there exists an integer n ij ≥ 1 and a chain of states i = l ij,0 , l ij,1 , . . . , l ij,n ij = j such that l ij,1 ∈ Y l ij,0 , . . . , l ij,n ij ∈ Y l ij,n ij −1 .
Let p ij (ε), ε ∈ (0, ε 0 ], j ∈ Y i , i ∈ X be some real-valued functions which satisfy condition D, i.e., can be represented in the form of Taylor asymptotic Condition D does not guarantee that matrix p ij (ε) is stochastic, for every ε ∈ (0, ε 0 ]. This can be achieved by imposing some additional conditions on coefficients and remainders in the above asymptotic expansions.
First, condition F requires holding of the following relation, Note that relation (19) implies that parameters l − i,Y i = 0, i ∈ X. It is not difficult to choose coefficients a ij [l], l = l − ij ≤ l ≤ l + ij , j ∈ Y i , i ∈ X in such way that relation (19) would hold. Any such coefficients, with the first coefficients a ij [l − ij ] > 0, j ∈ Y i , i ∈ X, can serve as coefficients in the asymptotic expansions penetrating condition D.
Second, condition F requires holding of the following identity, for every i ∈ X, Remainders o ij (ε l + ij ), j ∈ Y i , i ∈ X satisfying the above identities can be chosen in different ways.
The simplest one is to choose In this case, the above identities would reduce to equalities, These equalities supplement equalities given in relation (19). Such choice of remainders corresponds to models with polynomial perturbations.
We, however, would like to impose on remainders conditions mainly required of them by conditions D or D ′ .
There always exist j i ∈ Y i , i ∈ X such that l + ij i = l + i,Y i , i ∈ X. Identity (20) can be rewritten in the following form, for every i ∈ X, Relation (21) can be used as the formula defining remainders penetrating the corresponding asymptotic expansions in condition D.
Since l + ij i = l + i,Y i , i ∈ X, the following relation holds, for remainder o ij (ε l + ij ) defined by relation (21), for every i ∈ X, Thus, remainders o ij i (ε l + ij i ), i ∈ X defined by relation (21) can also serve in the corresponding asymptotic expansions in condition D.
Let us define ε ij i = min j∈Y i ,j =j i ε ij , i ∈ X and δ ij i = min j∈Y i ,j =j i δ ij , i ∈ X.
In this case, the following inequality holds, for every ε ∈ (0, ε ij i ], i ∈ X, Thus, the inequalities, i ∈ X, penetrating condition D ′ hold for remainders o ij i (ε l + ij i ), i ∈ X, with parameters ε ij i , δ ij i and G ij i defined above.
As follows from the above remarks, identity (21) holds for remainders o ij (ε l + ij ), j ∈ Y i , , i ∈ X , for ε ∈ (0, ε ′ 0 ], i ∈ X, where ε ′ 0 = min j∈Y i ε ij = min j∈Y i ,j =j i ε ij . Thus, condition F holds, if parameter ε 0 is replaced by the new value ε ′ 0 . In this case, functions p ij (ε), i, j ∈ X can, for every ε ∈ (0, ε ′ 0 ], serve as transition probabilities of a Markov chain. Note that remainders o ij (ε l + ij ), j ∈ Y i , i ∈ X constructed above can be very irregular functions. Let us, for example, consider the case, where all asymptotic expansions in condition D have the same order, i.e., parameters l + ij = l + , j ∈ Y i , i ∈ X. In this case, identities (20) take the form, can be continuous functions of ε taking zero value in at most finite numebrs of points. However, let us multiply them, for example, by the Dirichlet function D(ε). The new remainders o ′ ij (ε l + ) = D(ε)o ij (ε l + ), j ∈ Y i , i ∈ X also satisfy identities (20) and o ′ ij (ε l + )/ε l + → 0 as ε → 0, for j ∈ Y i , i ∈ X. At the same time, they are very irregular functions. This example is, of course, an artificial one. But, it well illustrates the above statement about possible irregularity of remainders and, in sequel, transition probabilities, as functions of the perturbation parameter.
Let us also make some remarks concerned the expected sojourn times. First, let us define e ij (ε) = 0, ε ∈ (0, ε 0 ] j ∈ Y i , i ∈ X that is consistent with condition A (b).
Functions p ij (ε), i, j ∈ X and e ij (ε), i, j ∈ X constructed above can serve, respectively, as transition probabilities of the embedded Markov chain η (ε) n and expectations of sojourn times for some semi-Markov process η (ε) (t), for every ε ∈ (0,ε 0 ]. A variant of transition probabilities for such semi-Markov processes is given in Section 3 * .

(27)
In the asymptotic expansions penetrating relations (26) and (27), the coefficients a ij [l − ij ], b ij [m − ij ] > 0, j ∈ Y i , i ∈ X, and coefficients a ij [l], l = l − ij , . . . , l + ij , j ∈ Y i , i ∈ X satisfy relation (19). We also assume that parameter ε 0 =ε 0 and remainders o(ε l + ij ),ȯ(ε m + ij ), j ∈ Y i , i ∈ X, in the asymptotic expansions representing elements of matrices given in relations (26) and (27), are chosen according the procedures described above, in particular, the identities (20) hold. In this case, matrices, given in the above relations, can, for every ε ∈ (0, ε 0 ], serve as, respectively, the matrix of transition probabilities for the corresponding embedded Markov chain and the matrix of expectations of sojourn times, for the semi-Markov process η (ε) (t), and conditions A -E hold.
We would like also to explain an unexpected, in some sense, asymptotic behavior of stationary probabilities π i (ε), in the above example. As a matter of fact, states 1 and 2 are asymptotically absorbing states with non-absorption probabilities of different order, respectively, O(ε 2 ) and O(ε). While, state 3 is a transient asymptotically non-absorbing state. This, seems, should cause convergence of the stationary probability π 1 (ε) to 1 and the stationary probabilities π 2 (ε) and π 3 (ε) to 0 as ε → 0, with different rates of convergence. This, however, does not take place, and all three probabilities converge to non-zero limits. This is because of the expected sojourn times e 1 (ε), e 2 (ε) and e 3 (ε) have orders, respectively, O(ε), O(1) and O(ε −1 ). These expectations compensate absorption effects for states 1, 2 and 3.
In the above example, computations of explicit upper bounds for remainders in the asymptotic expansions for stationary probabilities π 1 (ε), π 2 (ε) and π 3 (ε) can also be realized in the case, where conditions D ′ and E ′ hold instead of conditions D and E. We, however, omit this presentation, in order to escape overloading the paper by technical numerical computations.