Skip to main content
SpringerLink
Log in

Numerical solutions for optimal control of stochastic Kolmogorov systems with regime-switching and random jumps

  • Published:
Statistical Inference for Stochastic Processes Aims and scope Submit manuscript

Abstract

This work is devoted to numerical solutions of controlled stochastic Kolmogorov systems with regime switching and random jumps. Markov chain approximation methods are used to design numerical algorithms to approximate the controlled switching jump diffusions, the cost functions, and the value functions. Under suitable conditions, the convergence of the algorithms is proved. Numerical examples are provided to demonstrate the performance of the algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The computational data are available from the authors upon request. Code availability The computation are done in Python language using Numpy and Numba packages with computer environment Intel Core i7-8700 CPU 3.20GHz. The computational code is available from the authors upon request.

References

  • Bao J, Mao X, Yin G, Yuan C (2011) Competitive Lotka-Volterra population dynamics with jumps. Nonlinear Anal 17:6601–6616

    Article  MathSciNet  Google Scholar 

  • Chen X, Chen Z-Q, Tran K, Yin G (2019) Properties of switching jump diffusions: maximum principles and Harnack inequalities. Bernoulli 25:1045–1075

    Article  MathSciNet  Google Scholar 

  • Dieu NT, Nguyen DH, Du NH, Yin G (2016) Classification of asymptotic behavior in a stochastic SIR model. SIAM J Appl Dyn Sys 15(2):1062–1084

    Article  MathSciNet  Google Scholar 

  • Du N, Nguyen D, Yin G (2016) Conditions for permanence and ergodicity of certain stochastic predator-prey models. J Appl Probab 53:187–202

    Article  MathSciNet  Google Scholar 

  • Fleming WH, Nisio M (1966) On the existence of optimal stochastic controls. J Math Mech 15:777–794

    MathSciNet  MATH  Google Scholar 

  • Hening A, Nguyen DH (2018) Coexistence and extinction for stochastic-systems. Ann Appl Probab 28:1893–1942

    Article  MathSciNet  Google Scholar 

  • Jin Z, Yang HL, Yin G (2013) Numerical methods for optimal dividend payment and investment strategies of regime-switching jump diffusion models with capital injections. Automatica 49:2317–2329

    Article  MathSciNet  Google Scholar 

  • Khasminskii RZ (2012) Stochastic stability of differential equations, 2nd edn. Springer, New York

    Book  Google Scholar 

  • Khasminskii RZ, Klebaner FC (2001) Long term behavior of solutions of the Lotka-Volterra system under small random perturbations. Ann Appl Probab 11:952–963

    Article  MathSciNet  Google Scholar 

  • Khaminskii RZ, Klebaner FC, Liptser R (2003) Some results on the Lotka-Volterra model and its small random perturbations. Acta Appl Math 78:201–206

    Article  MathSciNet  Google Scholar 

  • Khasminskii R, Potsepun N (2006) On the replicator dynamics behavior under Stratonovich type random perturbations. Stoch Dyn 6:197–211

    Article  MathSciNet  Google Scholar 

  • Krause A, Kurowski L, Yawar K, Van Gorder RA (2018) Stochastic epidemic metapopulation models on networks: SIS dynamics and control strategies. J Theoret Biol 449:35–52

    Article  MathSciNet  Google Scholar 

  • Krylov NV (2000) On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab Theory Related Fields 117:1–16

    Article  MathSciNet  Google Scholar 

  • Kushner HJ (1984) Approximation and weak convergence methods for random processes with applications to stochastic systems theory. MIT press, Cambridge

    MATH  Google Scholar 

  • Kushner H, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time, Springer, New York, 2nd Ed

  • Menaldi J-L (1989) Some estimates for finite difference approximations. SIAM J Control Optim 27:579–607

    Article  MathSciNet  Google Scholar 

  • Nguyen D, Nguyen N, Yin G (2020) General nonlinear stochastic systems motivated by chemostat models: complete characterization of long-time behavior, optimal controls, and applications to wastewater treatment. Stochastic Process Appl 130:4068–4642

    MathSciNet  MATH  Google Scholar 

  • Nguyen D, Nguyen N, Yin G (2020) General nonlinear stochastic systems motivated by chemostat models: Complete characterization of long-time behavior, optimal controls, and applications to wastewater treatment. Stochastic Proc Appl 130:4068–4642

    MathSciNet  MATH  Google Scholar 

  • Nguyen DH, Nguyen NN, Yin G (2021) Stochastic functional kolmogorov equations II: extinction. J Differential Eqs 294:1–39

    Article  MathSciNet  Google Scholar 

  • Song QS, Yin G (2009) Rates of convergence of numerical methods for controlled regime-switching diffusions with stopping times in the costs. SIAM J Control Optim 48:1831–1857

    Article  MathSciNet  Google Scholar 

  • Song QS, Yin G, Zhang Z (2006) Numerical method for controlled regime-switching diffusions and regime-switching jump diffusions. Automatica 42:1147–1157

    Article  MathSciNet  Google Scholar 

  • Warga J (1962) Relaxed variational problems. J Math Anal Appl 4:111–128

    Article  MathSciNet  Google Scholar 

  • Yin G, Zhang Q, Badowski G (2003) Discrete-time singularly perturbed Markov Chains: aggregation, occupation measures, and switching diffusion limit. Adv Appl Probab 35:449–476

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Connecticut, Storrs, CT, 06269, USA

    Hongjiang Qian & George Yin

  2. School of Mathematics and Statistics, Central South University, Changsha, China

    Zhexin Wen

Authors
  1. Hongjiang Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhexin Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Yin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The research of H. Qian and G. Yin was supported in part by the Air Force Office of Scientific Research, and the research of Z. Wen was supported in part by Postdoctoral Foundation of Central South University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, H., Wen, Z. & Yin, G. Numerical solutions for optimal control of stochastic Kolmogorov systems with regime-switching and random jumps. Stat Inference Stoch Process 25, 105–125 (2022). https://doi.org/10.1007/s11203-021-09267-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11203-021-09267-z

Keywords

Mathematics Subject Classification

Search

Navigation