On the Optimal Control of a Random Walk with Jumps and Barriers



The modeling and optimal control of a class of random walks (RWs) is investigated in the framework of the Chapman-Kolmogorov (CK) and Fokker-Planck (FP) equations. This class of RWs includes jumps driven by a compound Poisson process and are subject to different barriers. A control mechanism is investigated that is included in the CK stochastic transition matrix and the purpose of the control is to track a desired discrete probability density function and attain a desired terminal density configuration. Existence and characterization of optimal controls are discussed. The proposed approach allows the derivation of a new FP model that accommodates the presence of the jumps and guarantees conservation of total probability in the case of reflecting barriers, which are modelled by appropriate operators. Results of numerical experiments are presented that successfully validate the proposed control framework.


Random walk with jumps Optimal control Modeling of random walk Fokker-Planck equation 

Mathematics Subject Classification (2010)

93E20 60G50 34K99 60J27 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amann H, Escher J (2008) Analysis II. BirkhäuserGoogle Scholar
  2. Annunziato M, Gottschalk H (2015) Calibration of Lévy processes using optimal control of Kolmogorov equations with periodic boundary conditions. arXiv:1506.08439
  3. Arnold L (1974) Stochastic differential equations. Wiley, New YorkGoogle Scholar
  4. Bensoussan A (2011) Stochastic control by functional analysis methods, vol 11. ElsevierGoogle Scholar
  5. Beraldi R (2009) Random walk with long jumps for wireless ad hoc networks. Ad Hoc Netw 7(2):294–306CrossRefGoogle Scholar
  6. Beutler FJ, Ross KW (1985) Optimal policies for controlled Markov chains with a constraint. J Math Anal Appl 112(1):236–252MathSciNetCrossRefMATHGoogle Scholar
  7. Chang JS, Cooper G (1970) A practical difference scheme for Fokker-Planck equations. J Comput Phys 6(1):1–16CrossRefMATHGoogle Scholar
  8. Codling EA, Plank MJ, Benhamou S (2008) Random walk models in biology. J R Soc Interface 5(25):813–834CrossRefGoogle Scholar
  9. Cox DR, Miller HD (1965) The theory of stochastic processes. Wiley, New YorkMATHGoogle Scholar
  10. Feller W (1954) Diffusion processes in one dimension. Trans Am Math Soc 77 (1):1–31MathSciNetCrossRefMATHGoogle Scholar
  11. Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions, vol 25. Springer Science & Business MediaGoogle Scholar
  12. Friedman A (2006) Stochastic differential equations and applications. Dover Publications, Inc., New YorkGoogle Scholar
  13. Gaviraghi B, Schindele A, Annunziato M, Borzì A (2016) On optimal sparse-control problems governed by jump-diffusion processes. Appl Math 7(16):1978CrossRefGoogle Scholar
  14. Hager WW, Zhang H (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J Optim 16(1):170–192MathSciNetCrossRefMATHGoogle Scholar
  15. Luenberger DG (1969) Optimization by vector space methods. WileyGoogle Scholar
  16. Milstein GN, Tretyakov MV (2013) Stochastic numerics for mathematical physics. Springer Science & Business MediaGoogle Scholar
  17. Pugh CC (2015) Real mathematical analysis. Undergraduate texts in mathematics, 2nd edn. Springer, ChamCrossRefGoogle Scholar
  18. Scalas E (2006) The application of continuous-time random walks in finance and economics. Physica A: Statistical Mechanics and its Applications 362(2):225–239MathSciNetCrossRefGoogle Scholar
  19. Yuste SB, Abad E, Lindenberg K (2013) Exploration and trapping of mortal random walkers. Phys Rev Lett 110:220603CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Tim Breitenbach
    • 1
  • Mario Annunziato
    • 2
  • Alfio Borzì
    • 1
  1. 1.Institut für MathematikUniversität WürzburgWürzburgGermany
  2. 2.Dipartimento di MatematicaUniversità degli Studi di SalernoFiscianoItaly

Personalised recommendations