Skip to main content
Log in

Linear Quadratic Gaussian Homing for Markov Processes with Regime Switching and Applications to Controlled Population Growth/Decay

  • Published:
Methodology and Computing in Applied Probability Aims and scope Submit manuscript

Abstract

The problem of optimally controlling one-dimensional diffusion processes until they enter a given stopping set is extended to include Markov regime switching. The optimal control problem is presented by making use of dynamic programming. In the case where the Markov chain has two states, the optimal homotopy analysis method (OHAM) is used to obtain an analytical approximation of the value function, which is compared to the finite difference approximation with successive updates of the nonlinear and coupling terms. As an example, the method is applied to controlled population growth with regime switching.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Asmussen S (1989) Risk theory in a Markovian environment. Scand Actuar J 1989(2):69–100

    Article  MathSciNet  Google Scholar 

  • Bassukas I D (1994) Comparative gompertzian analysis of alterations of tumor growth patterns. Cancer Res 54(16):4385–4392

    Google Scholar 

  • Dawson NJ, Kounta M (2019) Homotopy analysis method applied to second-order frequency mixing in nonlinear optical dielectric media. J Nonlin Opt Phys Mater 28(4):1950033

    Article  Google Scholar 

  • Ferrante L, Bompadre S, Possati L, Leone L (2000) Parameter estimation in a gompertzian stochastic model for tumor growth. Biometrics 56(4):1076–1081

    Article  MathSciNet  Google Scholar 

  • Fleming W H, Soner H M (2006) Controlled Markov processes and viscosity solutions, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Hamilton J D (1989) A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57(2):357–384

    Article  MathSciNet  Google Scholar 

  • He J H (2004) Comparison of homotopy perturbation method and homotopy analysis method. Appl Math Comput 156(2):527–539

    MathSciNet  MATH  Google Scholar 

  • Hieber P (2014) First-passage times of regime switching models. Stat Prob Lett 92:148–157

    Article  MathSciNet  Google Scholar 

  • Lefebvre M (2014) LQG homing for jump-diffusion processes. In: ROMAI J., vol 10, pp 147–152

  • Li X, Jiang D, Mao X (2009) Population dynamical behavior of Lotka-Volterra system under regime switching. J Comput Appl Math 232(2):427–448

    Article  MathSciNet  Google Scholar 

  • Liao S (2003) Beyond perturbation: Introduction to the homotopy analysis method. Chapman & Hall / CRC, Boca Raton

    Book  Google Scholar 

  • Liao S, Zhao Y (2016) On the method of directly defining inverse mapping for nonlinear differential equations. Num Algorithms 72(4):989–1020

    Article  MathSciNet  Google Scholar 

  • Liu M, Wang K (2010) Persistence and extinction of a stochastic single-specie model under regime switching in a polluted environment II. J Theor Biol 267(3):283–291

    Article  MathSciNet  Google Scholar 

  • Lu Y, Li S (2005) On the probability of ruin in a Markov-modulated risk model. Insur Math Econ 37(3):522–532

    Article  MathSciNet  Google Scholar 

  • Luo Q, Mao X (2007) Stochastic population dynamics under regime switching. J Math Anal Appl 334(1):69–84

    Article  MathSciNet  Google Scholar 

  • Luo Q, Mao X (2009) Stochastic population dynamics under regime switching II. J Math Anal Appl 355(2):577–593

    Article  MathSciNet  Google Scholar 

  • Sj L (1995) An approximate solution technique which does not depend upon small parameters: A special example. Int J Nonlin Mech 30(3):371–380

    Article  Google Scholar 

  • Wang J, Forsyth P (2008) Maximal use of central differencing for Hamilton-Jacobi-Bellman PDEs in finance. SIAM J Numer Anal 46 (3):1580–1601

    Article  MathSciNet  Google Scholar 

  • Whittle P (1983) Optimization over time: Dynamic programming and stochastic control Wiley series in probability and mathematical statistics: Applied probability and statistics. Wiley, Chichester

    Google Scholar 

  • Whittle P, Gait P A (1970) Reduction of a class of stochastic control problems. IMA J Appl Math 6(2):131–140

    Article  Google Scholar 

Download references

Acknowledgements

MK would like to thank the University of The Bahamas Internal Grants programme for Research, Creative and Artistic Proposals for supporting this research project. NJD would like to thank the Hawaii Pacific University, College of Natural and Computational Sciences, Scholarly Endeavors Program for supporting this research project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Moussa Kounta.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Proof of Proposition 4.2

Appendix A: Proof of Proposition 4.2

Proof

$$ \begin{array}{@{}rcl@{}} \mathcal{L}[v_{i,m}(x)]&=&\mathcal{L}\left\{\chi_{i,m} v_{i,m-1}(x)+\hbar_{i} \mathcal{L}_{i}^{-1}\left[e^{-2rx}\Re_{i,m-1}(x)\right]+c^{m}_{i0}+c^{m}_{i1}e^{-rx}\right\} \\ &=&\chi_{i,m} \mathcal{L}_{i}[v_{i,m-1}(x)]+\hbar_{i} e^{-2rx}\Re_{i,m-1}(x) \end{array} $$
(1)

Because \({\mathcal{L}}\circ {\mathcal{L}}^{-1}(x)=x,\quad \forall x\in \hat {V} \) and \({\mathcal{L}}(x)=0,\quad \forall x\in V^{*}\), (1) implies

$$ \begin{array}{@{}rcl@{}} \sum\limits_{j=1}^{m} (\mathcal{L}[v_{i,j}(x)]&-&\mathcal{L}\left[ v_{i,j-1}(x)\right])= \sum\limits_{j=1}^{m} \left( \hbar_{i} e^{-2rx}\Re_{i,j-1}(x)\right)\\ &\Rightarrow& \mathcal{L}[v_{i,m}(x)]=\mathcal{L}[v_{i,0}(x)]+\sum\limits_{j=1}^{m} \left( \hbar_{i} e^{-2rx}\Re_{i,j-1}(x) \right) \end{array} $$

Because \({\mathcal{L}}[v_{i,0}(x)]=0\), we obtain

$$ \lim_{m\to \infty}\mathcal{L}_{i}(v_{i,m}(x))=\hbar_{i} e^{-2rx} \sum\limits_{j=0}^{\infty} \Re_{i,j}(\boldsymbol{v}_{i,j}(x)) $$

Also, Eq. 4.8 is absolutely convergent, and therefore

$$ \lim_{m\to \infty}v_{i,m}(x)=0 . $$

Then

$$ \begin{array}{@{}rcl@{}} \hbar_{i} e^{-2rx}\sum\limits_{j=0}^{\infty}\Re_{i,j}(x))=\lim_{m\to \infty}\mathcal{L}_{i}(v_{i,m}(x)) =\mathcal{L}_{i}(\lim_{m\to \infty}v_{i,m}(x))=\mathcal{L}_{i}(0)=0 \end{array} $$

Because \(\hbar _i \neq 0\) and e− 2rx ≠ 0, it follows that

$$ \sum\limits_{j=0}^{\infty} \Re_{i,j}(x)=0 . $$

Also note that the Taylor series of

$$ \begin{array}{@{}rcl@{}} \mathcal{N}_{i} \left[\sum\limits_{j=0}^{\infty} v_{ij}(x)p^{j}\right]= \sum\limits_{j=0}^{\infty} \Re_{i,j}(x)p^{j}, \end{array} $$

at p = 1

$$ \begin{array}{@{}rcl@{}} \mathcal{N}_{i} \left[\sum\limits_{j=0}^{\infty} v_{ij}(x)\right]= \sum\limits_{j=0}^{\infty} \Re_{i,j}(x)=0 . \end{array} $$

1.1 Appendix B: Recursive Calculation of the Value Function Via OHAM

By using Eq. 4.5, we obtain

$$ v_{i,m}(x)=\chi_{i,m} v_{i,m-1}(x)+\hbar_{i} \mathcal{L}_{i}^{-1}\left[e^{-2rx}\Re_{i,m-1}(x)\right]+c^{m}_{i0}+c^{m}_{i1}e^{-rx} , $$

where \({\mathcal{L}}_i(c^m_{i0}+c^m_{i1}e^{-rx})=0\) and \(c^m_{i0}, \ c^m_{i1}\) are constants of integration, which will be determined from the boundary conditions. By making use of Eq. 4.6, we obtain

$$ \begin{array}{@{}rcl@{}} \Re_{i,0}(x)&=&R_{i} r^{2}a_{i1}^{2}e^{-2rx}+[a_{i1}r^{2}-S_{i}a_{i1}r+T_{i}q_{i\bar{i}}(a_{\bar{i}i}-a_{i1})]e^{-rx}+T_{i}q_{i\bar{i}}(a_{\bar{i}0}-a_{i0}) \end{array} $$

and

$$ \begin{array}{@{}rcl@{}} v_{i,1}(x)&=&\hbar_{i} R_{i}a_{i1}^{2}\frac{e^{-4rx}}{12}+\hbar_{i} [a_{i1}r^{2}-S_{i}a_{i1}r+T_{i}q_{i\bar{i}}(a_{\bar{i}i}-a_{i1})]\frac{e^{-3rx}}{6r^{2}}\\ &+&\hbar_{i} T_{i}q_{i\bar{i}}(a_{\bar{i}0}-a_{i0})\frac{e^{-2rx}}{2r^{2}}+c^{1}_{i0}+c^{1}_{i1}e^{-rx} \end{array} $$

By letting,

$$ \alpha_{i4}=\frac{\hbar_{i} R_{i}a_{i1}^{2}}{12}, \ \alpha_{i3}=\frac{\hbar_{i} [a_{i1}-S_{i}a_{i1}+T_{i}q_{i\bar{i}}(a_{\bar{i}i}-a_{i1})]}{6r^{2}}, \ \alpha_{i2}= \frac{\hbar_{i} T_{i}q_{i\bar{i}}(a_{\bar{i}0}-a_{i0})}{2r^{2}}, $$

we obtain

$$ v_{i1}(x)=\sum\limits_{k=2}^{4}\alpha_{ik}e^{-krx}+c^{1}_{i0}+c^{1}_{i1}e^{-rx} $$

with

$$ \begin{array}{@{}rcl@{}} c_{i1}^{1}&=&\frac{{\sum}_{k=2}^{4}\alpha_{ik}e^{-kra}-{\sum}_{k=2}^{4}\alpha_{ik}e^{-krb} }{e^{-rb}-e^{-ra}},\quad c_{i0}^{1}=-\sum\limits_{k=2}^{4}\alpha_{ik}e^{-kra}-c_{i1}^{1}e^{-ra} \end{array} $$

The second iteration follows as

$$ \begin{array}{@{}rcl@{}} \Re_{i,1}(x)\!&=&\!8r^{2}R_{i}a_{i1}\alpha_{i4}e^{-5rx} + \left[16r^{2}\alpha_{i4} + 6R_{i}r^{2}a_{i1}\alpha_{i3} - 4S_{i}r\alpha_{i4} + T_{i}q_{i\bar{i}}(\alpha_{\bar{i}4}-\alpha_{i4})\right]e^{-4rx}\\ &&+ \left[9\alpha_{i3}r^{2}+4R_{i}r^{2}a_{i1}\alpha_{i2}-3S_{i}r\alpha_{i3}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}3}-\alpha_{i3})\right]e^{-3rx}\\ &&+\left[4r^{2}\alpha_{i2}+2R_{i}r^{2}a_{i1}c^{1}_{i1}-2S_{i}r\alpha_{i2}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}2}-\alpha_{i2})\right]e^{-2rx}\\ &&+\left[c^{1}_{i1}r^{2}-S_{i}rc^{1}_{i1}+T_{i}q_{i\bar{i}}(c^{1}_{\bar{i}1}-c^{1}_{i1}) \right]e^{-rx}+T_{i}q_{i\bar{i}}(c^{1}_{\bar{i}0}-c^{1}_{i0}) , \end{array} $$

which, in turn, gives

$$ \begin{array}{@{}rcl@{}} v_{i,2}(x)&=&v_{i,1}(x)+8\hbar_{i} R_{i}a_{i1}\alpha_{i4}\frac{e^{-7rx}}{42}\\&&+\hbar_{i} \left[16r^{2}\alpha_{i4}+6R_{i}r^{2}a_{i1}\alpha_{i3} -4S_{i}r\alpha_{i4}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}4}-\alpha_{i4})\right]\frac{e^{-6rx}}{30r^{2}}\\ &&+ \hbar_{i} \left[9\alpha_{i3}r^{2}+4R_{i}r^{2}a_{i1}\alpha_{i2}-3S_{i}r\alpha_{i3}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}3}-\alpha_{i3})\right]\frac{e^{-5rx}}{20r^{2}}\\ &&+\hbar_{i} \left[4r^{2}\alpha_{i2}+2R_{i}r^{2}a_{i1}c^{1}_{i1}-2S_{i}r\alpha_{i2}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}2}-\alpha_{i2})\right]\frac{e^{-4rx}}{12r^{2}}\\ &&+\hbar_{i} \left[c^{1}_{i1}r^{2}-S_{i}rc^{1}_{i1}+T_{i}q_{i\bar{i}}\left( c^{1}_{\bar{i}1}-c^{1}_{i1}\right) \right]\frac{e^{-3rx}}{6r^{2}}+\hbar_{i}T_{i}q_{i\bar{i}}(c^{1}_{\bar{i}0}-c^{1}_{i0})\frac{e^{-2rx}}{2r^{2}}\\&&+ c^{2}_{i0}+c^{2}_{i1}e^{-rx}. \end{array} $$

By letting,

$$ \begin{array}{@{}rcl@{}} \beta_{i7}&=&\frac{8\hbar_{i} R_{i}a_{i1}\alpha_{i4}}{42}, \ \beta_{i6}=\frac{\hbar_{i} \left[16r^{2}\alpha_{i4}+6R_{i}r^{2}a_{i1}\alpha_{i3} -4S_{i}r\alpha_{i4}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}4}-\alpha_{i4})\right] }{30r^{2}},\\ \beta_{i5}&=&\frac{\hbar_{i} \left[9\alpha_{i3}r^{2}+4R_{i}r^{2}a_{i1}\alpha_{i2}-3S_{i}r\alpha_{i3}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}3}-\alpha_{i3})\right]}{20r^{2}}\\ \beta_{i4}&=&\frac{\hbar_{i} \left[4r^{2}\alpha_{i2}+2R_{i}r^{2}a_{i1}c^{1}_{i1}-2S_{i}r\alpha_{i2}+T_{i}q_{i\bar{i}}(\alpha_{\bar{i}2}-\alpha_{i2})\right]}{12r^{2}}+\alpha_{i4}\\ \beta_{i3}&=&\frac{\hbar_{i} \left[c^{1}_{i1}r^{2}-S_{i}rc^{1}_{i1}+T_{i}q_{i\bar{i}}(c^{1}_{\bar{i}1}-c^{1}_{i1}) \right]}{6r^{2}}+\alpha_{i3},\quad \beta_{i2}=\frac{\hbar_{i}T_{i}q_{i\bar{i}}(c^{1}_{\bar{i}0}-c^{1}_{i0})}{2r^{2}}+\alpha_{i2}, \end{array} $$

we obtain

$$ v_{i,2}(x)={\sum}_{k=2}^{7}\beta_{ik}e^{-krx}+c^{1}_{i0}+c^{1}_{i1}e^{-rx}+c^{2}_{i0}+c^{2}_{i1}e^{-rx}, $$

with

$$ \begin{array}{@{}rcl@{}} c_{i1}^{2}&=&\frac{{\sum}_{k=2}^{7}\beta_{ik}e^{-kra}-{\sum}_{k=2}^{7}\beta_{ik}e^{-krb} }{e^{-rb}-e^{-ra}}-c_{i1}^{1},\\ c_{i0}^{2}&=&-{\sum}_{k=2}^{7}\beta_{ik}e^{-kra}-c_{i1}^{2}e^{-ra}-c_{i0}^{1}-c_{i1}^{1}e^{-ra}. \end{array} $$

The third iteration follows as

$$ \begin{array}{@{}rcl@{}} \Re_{i,2}(x)\!&=&\!49r^{2}R_{i}\beta_{i7}^{2}e^{-14rx}+84r^{2}\beta_{i6}\beta_{i7} R_{i}e^{-13rx}+ (70\beta_{i5}\beta_{i7}+36\beta_{i6}^{2})R_{i}r^{2}e^{-12rx} \\ &&\!+ (56\beta_{i4}\beta_{i7} + 60\beta_{i6}\beta_{i5})r^{2}R_{i}e^{-11rx} + (42\beta_{i3}\beta_{i7} + 48\beta_{i4}\beta_{i6}+25\beta_{i5}^{2})r^{2}R_{i}e^{-10rx}\\ &&\!+ (28\beta_{i2}\beta_{i7} + 36\beta_{i3}\beta_{i6} + 40\beta_{i4}\beta_{i5}) )r^{2}R_{i}e^{-9rx} + [(14\beta_{i7}(c_{i1}^{2}+c_{i1}^{1})+24\beta_{i2}\beta_{i6}\\ &&\!+30\beta_{i5}\beta_{i3}+16\beta_{i4}^{2})+14a_{i1}\beta_{i7}]r^{2}R_{i}e^{-8rx}\\&&\!+[ (12(c_{i1}^{1}+c_{i1}^{2})\beta_{i6}+20\beta_{i5}\beta_{i2}+24\beta_{i3}\beta_{i4}+12a_{i1}\beta_{i6})r^{2}R_{i}+49r^{2}\beta_{i7}\\&&\!-7S_{i}r\beta_{i7}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}7}-\beta_{i7}) ]e^{-7rx}\\ &&\!+[ (10(c_{i1}^{1}+c_{i1}^{2})\beta_{i5}+16\beta_{i4}\beta_{i2}+9\beta_{i3}^{2}+10a_{i1}\beta_{i5})r^{2}R_{i}+36r^{2}\beta_{i6} -6S_{i}r\beta_{i6}\\&&\!+T_{i}q_{i\bar{i}}(\beta_{\bar{i}6}-\beta_{i6}) ]e^{-6rx}\\ &&\!+[ (8(c_{i1}^{1}+c_{i1}^{2})\beta_{i4}+12\beta_{i3}\beta_{i2}+8a_{i1}\beta_{i4})r^{2}R_{i}+25r^{2}\beta_{i5} -5S_{i}r\beta_{i5}\\&&\!+T_{i}q_{i\bar{i}}(\beta_{\bar{i}5}-\beta_{i5}) ]e^{-5rx}\\&&\!+[ (6(c_{i1}^{1}+c_{i1}^{2})\beta_{i3}+4\beta_{i2}^{2}+6a_{i1}\beta_{i3})r^{2}R_{i}+16r^{2}\beta_{i4} -4S_{i}r\beta_{i4}\\&&\!+T_{i}q_{i\bar{i}}(\beta_{\bar{i}4}-\beta_{i4}) ]e^{-4rx}\\ &&\!+[ (4(c_{i1}^{1}+c_{i1}^{2})\beta_{i2}+8a_{i1}\beta_{i2}+4a_{i1}\beta_{i2})r^{2}R_{i}+9\beta_{i3} r^{2} -3S_{i}r\beta_{i3}\\&&\!+T_{i}q_{i\bar{i}}(\beta_{\bar{i}3}-\beta_{i3`}) ]e^{-3rx}\\ &&\!+[ ((c_{i1}^{1}+c_{i1}^{2})+2a_{i1}(c_{i1}^{1}+c_{i1}^{2})r^{2}R_{i}+4\beta_{i2} r^{2} -2S_{i}r\beta_{i2}\\&&\!+T_{i}q_{i\bar{i}}(\beta_{\bar{i}2}-\beta_{i2`}) ]e^{-2rx}\\ &&\!+[(c_{i1}^{1}+c_{i1}^{2})(r^{2} -S_{i}r)+T_{i}q_{i\bar{i}}[(c_{\bar{i}1}^{1}+c_{\bar{i}1}^{2})-(c_{i1}^{1}+c_{i1}^{2}))]e^{-rx} \\&&\!+T_{i}q_{i\bar{i}}[(c_{\bar{i}0}^{1}+c_{\bar{i}0}^{2})-(c_{i0}^{1}+c_{i0}^{2}))], \end{array} $$

which, in turn, gives

$$ \begin{array}{@{}rcl@{}} v_{i,3}(x)&=&v_{i,2}(x)+\hbar_{i} 49R_{i}\beta_{i7}^{2}\frac{e^{-16rx}}{240}+84\hbar_{i}\beta_{i6}\beta_{i7} R_{i}\frac{e^{-15rx}}{210}\\&&+ \hbar_{i}(70\beta_{i5}\beta_{i7}+36\beta_{i6}^{2})R_{i}\frac{e^{-14rx}}{182}\\ &&+ \hbar_{i}(56\beta_{i4}\beta_{i7}+60\beta_{i6}\beta_{i5})R_{i}\frac{e^{-13rx}}{156}\\&&+\hbar_{i}\left( 42\beta_{i3}\beta_{i7}+48\beta_{i4}\beta_{i6}+25\beta_{i5}^{2}\right)R_{i}\frac{e^{-12rx}}{132}\\ &&+ (28\beta_{i2}\beta_{i7}+36\beta_{i3}\beta_{i6}+40\beta_{i4}\beta_{i5}) )R_{i}\frac{e^{-11rx}}{110}\\ &&+ \hbar_{i}[(14\beta_{i7}(c_{i1}^{2}+c_{i1}^{1})+24\beta_{i2}\beta_{i6}+30\beta_{i5}\beta_{i3}+16\beta_{i4}^{2})+14a_{i1}\beta_{i7}]R_{i}\frac{e^{-10rx}}{90}\\ &&+\hbar_{i}[ (12(c_{i1}^{1}+c_{i1}^{2})\beta_{i6}+20\beta_{i5}\beta_{i2}+24\beta_{i3}\beta_{i4}+12a_{i1}\beta_{i6})r^{2}R_{i}+49r^{2}\beta_{i7}\\&&-7S_{i}r\beta_{i7}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}7}-\beta_{i7}) ]\frac{e^{-9rx}}{72r^{2}}\\ &&+\hbar_{i}[ (10(c_{i1}^{1}+c_{i1}^{2})\beta_{i5}+16\beta_{i4}\beta_{i2}+9\beta_{i3}^{2}+10a_{i1}\beta_{i5})r^{2}R_{i}+36r^{2}\beta_{i6} \\&&-6S_{i}r\beta_{i6}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}6}-\beta_{i6}) ]\frac{e^{-8rx}}{56r^{2}}\\ &&+\hbar_{i}[ (8(c_{i1}^{1}+c_{i1}^{2})\beta_{i4}+12\beta_{i3}\beta_{i2}+8a_{i1}\beta_{i4})r^{2}R_{i}+25r^{2}\beta_{i5} -5S_{i}r\beta_{i5}\\&&+T_{i}q_{i\bar{i}}(\beta_{\bar{i}5}-\beta_{i5}) ]\frac{e^{-7rx}}{42r^{2}}\\ &&+\hbar_{i}[ (6(c_{i1}^{1}+c_{i1}^{2})\beta_{i3}+4\beta_{i2}^{2}+6a_{i1}\beta_{i3})r^{2}R_{i}+16r^{2}\beta_{i4} -4S_{i}r\beta_{i4}\\&&+T_{i}q_{i\bar{i}}(\beta_{\bar{i}4}-\beta_{i4}) ]\frac{e^{-6rx}}{30r^{2}}\\ &&+\hbar_{i}[ (4(c_{i1}^{1}+c_{i1}^{2})\beta_{i2}+8a_{i1}\beta_{i2}+4a_{i1}\beta_{i2})r^{2}R_{i}+9\beta_{i3} r^{2} -3S_{i}r\beta_{i3}\\&&+T_{i}q_{i\bar{i}}(\beta_{\bar{i}3}-\beta_{i3`}) ]\frac{e^{-5rx}}{10r^{2}}\\ &&+\hbar_{i} [ ((c_{i1}^{1}+c_{i1}^{2})+2a_{i1}(c_{i1}^{1}+c_{i1}^{2})r^{2}R_{i}+4\beta_{i2} r^{2} -2S_{i}r\beta_{i2}\\&&+T_{i}q_{i\bar{i}}(\beta_{\bar{i}2}-\beta_{i2`}) ]\frac{e^{-4rx}}{12r^{2}}\\ &&+\hbar_{i}[(c_{i1}^{1}+c_{i1}^{2})(r^{2} -S_{i}r)+T_{i}q_{i\bar{i}}[(c_{\bar{i}1}^{1}+c_{\bar{i}1}^{2})-(c_{i1}^{1}+c_{i1}^{2}))]\frac{e^{-3rx}}{6r^{2}}\\ &&+\hbar_{i}T_{i}q_{i\bar{i}}[(c_{\bar{i}0}^{1}+c_{\bar{i}0}^{2})-(c_{i0}^{1}+c_{i0}^{2}))]\frac{e^{-2rx}}{2r^{2}}+c_{i0}^{3}+c_{i1}^{3}e^{-rx} \end{array} $$

By letting,

$$ \begin{array}{@{}rcl@{}} \gamma_{i16}&=&\frac{\hbar_{i} 49R_{i}\beta_{i7}^{2}}{240},\quad \gamma_{i15}=\frac{84\hbar_{i}\beta_{i6}\beta_{i7} R_{i}}{210}, \quad \gamma_{i14}=\frac{\hbar_{i}(70\beta_{i5}\beta_{i7}+36\beta_{i6}^{2})R_{i}}{182}\\ \gamma_{i13}&=&\frac{ \hbar_{i}(56\beta_{i4}\beta_{i7}+60\beta_{i6}\beta_{i5})R_{i}}{156},\quad \gamma_{i12}=\frac{\hbar_{i}(42\beta_{i3}\beta_{i7}+48\beta_{i4}\beta_{i6}+25\beta_{i5}^{2})R_{i}}{132} \end{array} $$
$$ \begin{array}{@{}rcl@{}} \gamma_{i11}\!&=&\!\frac{(28\beta_{i2}\beta_{i7}+36\beta_{i3}\beta_{i6}+40\beta_{i4}\beta_{i5}) )R_{i}}{110},\\ \gamma_{i10}\!&=&\!\frac{\hbar_{i}[(14\beta_{i7}(c_{i1}^{2}+c_{i1}^{1})+24\beta_{i2}\beta_{i6}+30\beta_{i5}\beta_{i3}+16\beta_{i4}^{2})+14a_{i1}\beta_{i7}]R_{i}}{90}\\ \gamma_{i9}\!&=&\!\frac{\hbar_{i}[ (12(c_{i1}^{1} + c_{i1}^{2})\beta_{i6} + 20\beta_{i5}\beta_{i2} + 24\beta_{i3}\beta_{i4} + 12a_{i1}\beta_{i6})r^{2}R_{i} + 49r^{2}\beta_{i7} - 7S_{i}r\beta_{i7} + T_{i}q_{i\bar{i}}(\beta_{\bar{i}7} - \beta_{i7}) ]}{72r^{2}}\\ \gamma_{i8}\!&=&\!\frac{\hbar_{i}[ (10(c_{i1}^{1} + c_{i1}^{2})\beta_{i5} + 16\beta_{i4}\beta_{i2} + 9\beta_{i3}^{2} + 10a_{i1}\beta_{i5})r^{2}R_{i} + 36r^{2}\beta_{i6} -6S_{i}r\beta_{i6}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}6}-\beta_{i6}) ]}{56r^{2}}\\ \gamma_{i7}\!&=&\!\frac{\hbar_{i}[ (8(c_{i1}^{1}+c_{i1}^{2})\beta_{i4}+12\beta_{i3}\beta_{i2}+8a_{i1}\beta_{i4})r^{2}R_{i}+25r^{2}\beta_{i5} -5S_{i}r\beta_{i5}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}5}-\beta_{i5}) ]}{42r^{2}}+\beta_{i7}\\ \gamma_{i6}\!&=&\!\frac{\hbar_{i}[ (6(c_{i1}^{1}+c_{i1}^{2})\beta_{i3}+4\beta_{i2}^{2}+6a_{i1}\beta_{i3})r^{2}R_{i}+16r^{2}\beta_{i4} -4S_{i}r\beta_{i4}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}4}-\beta_{i4}) ]}{30r^{2}}+\beta_{i6}\\ \gamma_{i5}\!&=&\!\frac{\hbar_{i}[ (4(c_{i1}^{1}+c_{i1}^{2})\beta_{i2}+8a_{i1}\beta_{i2}+4a_{i1}\beta_{i2})r^{2}R_{i}+9\beta_{i3} r^{2} -3S_{i}r\beta_{i3}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}3}-\beta_{i3`}) ]}{10r^{2}}+\beta_{i5}\\ \gamma_{i4}\!&=&\!\frac{\hbar_{i} [ ((c_{i1}^{1}+c_{i1}^{2})+2a_{i1}(c_{i1}^{1}+c_{i1}^{2})r^{2}R_{i}+4\beta_{i2} r^{2} -2S_{i}r\beta_{i2}+T_{i}q_{i\bar{i}}(\beta_{\bar{i}2}-\beta_{i2`}) ]}{12r^{2}}+\beta_{i4}\\ \gamma_{i3}\!&=&\!\frac{\hbar_{i}[(c_{i1}^{1}+c_{i1}^{2})(r^{2} -S_{i}r)+T_{i}q_{i\bar{i}}[(c_{\bar{i}1}^{1}+c_{\bar{i}1}^{2})-(c_{i1}^{1}+c_{i1}^{2}))]}{6r^{2}}+\beta_{i3},\\ \gamma_{i2}\!&=&\!\frac{\hbar_{i}T_{i}q_{i\bar{i}}[(c_{\bar{i}0}^{1}+c_{\bar{i}0}^{2})-(c_{i0}^{1}+c_{i0}^{2}))]}{2r^{2}} +\beta_{i2}, \end{array} $$

we obtain

$$ v_{i,3}(x)=\sum\limits_{k=2}^{16}\gamma_{ik}e^{-krx}+c^{1}_{i0}+c^{1}_{i1}e^{-rx}+c^{2}_{i0}+c^{2}_{i1}e^{-rx}+c^{3}_{i0}+c^{3}_{i1}e^{-rx}, $$

with

$$ \begin{array}{@{}rcl@{}} c_{i1}^{3}\!&=&\!\frac{\left( {\sum}_{k=2}^{16}\gamma_{ik}e^{-krb} - c^{1}_{i0} - c^{1}_{i1}e^{-rb} - c_{i0}^{2} - c_{i1}^{2}e^{-rb}\right) - \left( {\sum}_{k=2}^{16}\gamma_{ik}e^{-kra}-c^{1}_{i0} - c^{1}_{i1}e^{-ra} - c_{i0}^{2} - c_{i1}^{2}e^{-ra}\right)}{e^{-ra}-e^{-rb}} \end{array} $$
$$ \begin{array}{@{}rcl@{}} c_{i0}^{3}=-\sum\limits_{k=2}^{16}\gamma_{ik}e^{-kra}-c^{1}_{i0}-c^{1}_{i1}e^{-ra}-c_{i0}^{2}-c_{i1}^{2}e^{-ra}-c_{i1}^{3}e^{-ra}. \end{array} $$

An approximate analytical solution truncated to second order would follow as

$$ \bar{V}_{i}(x)\approx v_{i,0}(x)+v_{i,1}(x)+v_{i,2}(x)+v_{i,3}(x)\ldots \ i=1,2. $$

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kounta, M., Dawson, N.J. Linear Quadratic Gaussian Homing for Markov Processes with Regime Switching and Applications to Controlled Population Growth/Decay. Methodol Comput Appl Probab 23, 1155–1172 (2021). https://doi.org/10.1007/s11009-020-09800-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11009-020-09800-2

Keywords

Mathematics Subject Classification (2010)

Navigation