Abstract
Stochastic control for systems with an unknown parameter is considered in this paper. The underlying problem is to minimize a functional subject to a system described by a singularly perturbed differential equation with an unknown parameter process driven by fast fluctuating random disturbances. This problem arises in the context of stochastic adaptive control, adaptive signal processing, and failure-prone manufacturing systems. Due to the nature of the wide-bandwidth noise processes, identifying the parameter process for eacht is very hard since the driving noise changes very rapidly. An alternative approach is used, and an auxiliary control problem is introduced to overcome the difficulties. By means of weak convergence methods and comparison control techniques, nearly optimal controls are obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Billingsley P (1968) Convergence of Probability Measures. Wiley, New York
Blankenship GB, Papanicolaou GC (1978) Stability and control of stochastic systems with wideband noise disturbances. SIAM J Appl Math 42:437–476
Caines PE, Chen HF (1985) Optimal adaptive LQG control for systems with finite state process parameters. IEEE Trans Automat Control 30:185–189
Ethier SN, Kurtz TG (1986) Markov Processes: Characterization and Convergence. Wiley, New York
Fleming WH (1982) Generalized solution in optimal stochastic control. Proceedings of the URI Conference on Control, pp 147–165
Fleming WH, Nisio M (1984) On stochastic relaxed controls for partially observable diffusions. Nagoya Math J 93:71–108
Fleming WH, Rishel RW (1975) Deterministic and Stochastic Optimal Control. Springer-Verlag, New York
Gerencsér L, Caines PE, eds (1991) Topics in Stochastic Systems: Modeling, Estimation and Adaptive Control. Springer-Verlag, New York
Kurtz TG (1975) Semigroups of conditional shifts and approximations of Markov processes. Ann Probab 4:618–642
Kushner HJ (1984) Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory. MIT Press, Cambridge, MA
Kushner HJ (1991) Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems. Birkhäuser, Boston
Kushner HJ, Runggaldier W (1987) Nearly optimal state feedback controls for stochastic systems with wideband noise disturbances. SIAM J Control Optim 25:289–315
Papanicolaou GC, Stroock D, Varadhan SRS (1977) Martingale approach to some limit theorems. Proceedings of the Conference on Statistical Mechanics, Dynamical Systems and Turbulence (Reed M, ed). Duke University Mathematical Series, vol 3. Duke University Press, Durham, NC
Warga J (1962) Relaxed variational problems. J Math Anal Appl 4:111–128
Zhang Q (1991) Controlled diffusions with rapidly oscillating unknown parameter processes. Proceedings of the Workshop on Stochastic Theory-Adaptive Control (Duncan T, Pasik-Duncan B, eds). Springer-Verlag, New York, pp 490–500
Author information
Authors and Affiliations
Additional information
This research was supported in part by the National Science Foundation under Grant DMS-9022139 and DMS-9224372.
Rights and permissions
About this article
Cite this article
Yin, G., Zhang, Q. Near optimality of stochastic control in systems with unknown parameter processes. Appl Math Optim 29, 263–284 (1994). https://doi.org/10.1007/BF01189478
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01189478