Applied Mathematics & Optimization

, Volume 66, Issue 1, pp 55–80 | Cite as

Controlled Nonlinear Stochastic Delay Equations: Part II: Approximations and Pipe-Flow Representations

  • Harold J. KushnerEmail author


This is the second part of a work dealing with key issues that have not been addressed in the modeling and numerical optimization of nonlinear stochastic delay systems. We consider new classes of models, such as those with nonlinear functions of several controls (such as products), each with is own delay, controlled random Poisson measure driving terms, admissions control with delayed retrials, and others. Part I was concerned with issues concerning the class of admissible controls and their approximations, since the classical definitions are inadequate for our models. This part is concerned with transportation equation representations and their approximations. Such representations of nonlinear stochastic delay models have been crucial in the development of numerical algorithms with much reduced memory and computational requirements. The representations for the new models are not obvious and are developed. They also provide a template for the adaptation of the Markov chain approximation numerical methods.


Nonlinear stochastic delay equations Numerical methods Optimal stochastic control Transportation equation representation 


  1. 1.
    Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, New York (1986) zbMATHGoogle Scholar
  2. 2.
    Kushner, H.J.: Numerical algorithms for optimal controls for nonlinear stochastic systems with delays. IEEE Trans. Autom. Control 55, 2170–2176 (2010) MathSciNetCrossRefGoogle Scholar
  3. 3.
    Kushner, H.J.: Numerical methods for controls for nonlinear stochastic systems with delays and jumps: applications to admission control. Stoch., Int. J. Probab. Stat. 83, 277–310 (2011) MathSciNetzbMATHGoogle Scholar
  4. 4.
    Kushner, H.J.: Numerical methods for the optimization of nonlinear stochastic delay systems, and an application to Internet regulation. In: Proceedings of the 49th Conference on Decision and Control. IEEE Press, New York (2010) Google Scholar
  5. 5.
    Krstic, M.: Compensation of infinite-dimesional actuator and sensor dynamics: nonlinear and adaptive systems. IEEE Control Syst. Mag. 30, 22–41 (2010) MathSciNetCrossRefGoogle Scholar
  6. 6.
    Kushner, H.J.: Controlled nonlinear stochastic delay equations: Part I: Modeling and approximations. Appl. Math. Optim. (2012). doi: 10.1007/s00245-012-9163-3 Google Scholar
  7. 7.
    Kushner, H.J.: Numerical Methods for Controlled Stochastic Delay Systems. Birkhäuser, Boston (2008) zbMATHCrossRefGoogle Scholar
  8. 8.
    Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, Berlin (1992), 2nd edn. (2001) zbMATHGoogle Scholar
  9. 9.
    Vinter, R.B., Kwong, R.H.: The infinite time quadratic control problem for linear systems with state and control delays: an evolution equation approach. SIAM J. Control Optim. 19, 139–153 (1981) MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Applied MathBrown UniversityProvidenceUSA

Personalised recommendations