Abstract
Example of dual control of linear uncertain system have been presented. The control task with short horizon (N=2) were solved using dynamic programming. It was shown that the optimal solution is ambiguous, the cost function is non-convex and has many local minima. Optimal control depends in a discontinuous manner on the initial conditions. It was also observed that active learning occurs only when the uncertainty of the initial state exceeds a certain threshold. In this case, the amount of information transmitted from sensor to the controller is much greater than in the case of passive learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
[1] Alpcan T., Shames I., (2015): An Information-Based Learning Approach to Dual Control . IEEE Trans. on Neural Networks and Learning Systems Vol.26, Issue 11 Pages: 2736–2748.
[2] Alpcan T.; Shames I; Cantoni M.; Nair G. (2013): Learning and information for dual control 2013 9th Asian Control Conference (ASCC), Pages: 1–6
[3] Åström K, Wittenmark B. (1995): Adaptive Control. Addison–Wesley, 2nd edition.
[4] Åström K., Helmersson A. (1986): Dual control of an integrator with unknown gain. Comp. & Maths. With Appls.,12A:6, pp. 653–662.
[5] Åström K., Wittenmark B. (1971): Problems of identification and control. Journal of Mathematical Analysis and Applications, 34, pp. 90–113.
[6] Banek T. (2010): Incremental value of information for discrete–time partially observed stochastic systems Control and Cybernetics vol. 39, No. 3.
[7] Banek T., Kozłowski E. (2005): Active and passive learning in control processes application of the entropy concept, Systems Sciences, vol. 31, N.2, pp. 29–44.
[8] Banek T., Kozłowski E. (2006): Adaptive control of system entropy Contr. and Cybernetics 35, 2.
[9] Banek T., Kozłowski E. (2010): Active learning in discrete–time stochastic systems In: Jerzy Jozefczyk and Donat Orski (ed.), Knowledge-based Intelligent System Advancements: Systemic and Cybernetic Approaches, pp.350–371.
[10] Bania P., Baranowski J. (2016): Field Kalman Filter and its approximation. In proc of 55th IEEE Conference on Decision and Control December 12–14, Las Vegas, USA.
[11] Bar-Shalom Y., Tse E. (1976): Caution, probing, and the value of information in the control of uncertain systems. Ann. Econ. Social. Measurement. Vol 5. pp. 323–337.
[12] Bar-Shalom Y. (1981): Stochastic Dynamic Programming: Caution and Probing. IEEE trans Aut. Contr. Vol AC-26, No 5 pp. 1184–1195.
[13] Bensoussan, A. (1983), Maximum Principle and Dynamic Programming Approach of the Optimal control of Partially Observed Diffusions, Stochastics, vol.9, issue 3, 169–222.
[14] Bernhardsson B. (1989): Dual control of a first-order system with two possible gains. Int. J. of Adaptive Control and Signal Processing, 3, pp. 15–22.
[15] Bohlin T. (1969): Optimal dual control of a simple process with unknown gain. Report PT 18.196, IBM Nordic Laboratory, Lidingö, Sweden.
[16] Cao S., Qian F., Wang X. (2016): Exact optimal solution for a class of dual control problems. International Journal of Systems Science, 47:9, 2078–2087.
[17] Casiello F., Loparo K. A. (1989): Optimal control of unknown parameter systems. IEEE Trans. Aut. Contr. AC-34, pp.1092–1094.
[18] Chen R., Loparo K. A. (1991): Dual control of linear stochastic systems with unknown parameters. IEEE International Conference on Systems Engineering pp. 65–68.
[19] Chen, R., (1990): Dual Control of Linear Stochastic Systems with Unknown Parameters. Ph.D. Dissertation, Systems Engineering, Case Western Reserve University.
[20] Fang S., Chen J., Hideaki I., (2017): Towards integrating control and information theories. Lecture notes in control and information sciences 465. Springer.
[21] Feldbaum A.A. (1965): Optimal control systems. Elsevier Science.
[22] Feldbaum, A.A. (1960): Dual Control Theory I–II, J. Aut. Remote Cont., 21, 874–880, 1033–1039.
[23] Feldbaum, A.A. (1961), Dual Control Theory III–IV, J. Aut. Remote Cont., 22, 1–12, 109–121.
[24] Feng X., Loparo K., (1997): Optimal State Estimation for Stochastic Systems: An Information Theoretic Approach IEEE Transactions on Automatic Control, Vol. 42, No. 6.
[25] Filatov N. M, Unbehauen H. (2000): Survey of adaptive dual control methods. IEE Proceedings - Control Theory and Applications 147(1):118 – 128.
[26] Filatov, N.M., Unbehauen, H. (2004): Adaptive Dual Control: Theory and Applications, Lecture Notes in Control and Information Sciences No. 302.
[27] Fleming, W.H., Pardoux, H. (1982): Optimal Control of Partially Observed Diffusion, SIAM Journal on Control and Optimization, 20, 261–285.
[28] Heirung T. A., Ydstie B., Foss B. (2017): Dual Adaptive Model Predictive Control. Automatica, Feb. 2017, to appear.
[29] Hijab O. (1984): Entropy and dual control Proc. of 23rd C DC, Las Vegas NV.
[30] Hordjewicz T., Kozłowski E. (2006): Comparison of stochastic optimal controls with different level of self-learning Annales UMCS Informatica AI 5 (2006) 343–356.
[31] Hordjewicz T., Kozłowski E. (2007): The self-learning active problem in dynamic systems Annales UMCS Informatica AI 7 (2007) 171–180
[32] Kumar K, Heirung T. A,. N., Patwardhan S. C., Foss B. (2015): Experimental Evaluation of a MIMO Adaptive Dual MPC. 9th IFAC Symposium on Advanced Control of Chemical Processes ADCHEM 2015 Whistler, Canada, 7–10 June 7 – 10, 2015, IFAC-PapersOnLine Volume 48, Issue 8, 545–550 .
[33] Li D.; Qian F.; Fu P. (2008): Optimal nominal dual control for discrete-time linear-quadratic Gaussian problems with unknown parameters Automatica 44, 119–127
[34] Li D; Fu P.; Qian F. (2003): Optimal nominal dual control for discrete-time LQG problem with unknown parameters, SICE Annual Conference, http://ieeexplore.ieee.org/document/1323396/
[35] Lindoff B., Holst J., Wittenmark B. (1999): Analysis of approximations of dual control. Int. J. Adapt. Control Signal Process. 13, 593–620.
[36] Potschka H. C, Schloder J. P., Bock H. G. (2016): Dual Control and Information Gain in Controlling Uncertain Processes. 11th IFAC Symposium on Dynamics and Control of Process Systems, including Biosystems June 6-8, 2016. NTNU, Trondheim, Norway..
[37] Rishel, R. (1986): An Exact Formula for a Linear Quadratic Adaptive Stochastic Optimal Control Law, SIAM Journal on Control and Optimization, 24, 667–674.
[38] Rishel, R. (1990), A Comment on a Dual Control Problem in Proc. of 19th IEEE Conference on Decision and Control Including the Symposium on Adaptive Processes, pp. 337–340.
[39] Sagawa T., Ueda M. (2013): Role of mutual information in entropy production under information exchanges. New J. Phys. 15 125012,
[40] Saridis G. N. (1988): Entropy formulation of optimal and adaptive control. IEEE Trans. Aut. Cont. Vol: 33, Issue: 8.
[41] Saridis G. N. (2001): Entropy in control engineering. Series in Inteligent Control and Intelligent Automation. World Scientific Publishing.
[42] Scardovi L. (2005): Information based control for state and parameter estimation. PhD thesis. University of Genoa Dep. of Communication, Faculty of Engineering Computer and System Sciences.
[43] Sternby J. (1976): A simple dual control problem with an analytical solution. IEEE Trans.Aut. Cont., 21(6):840–844.
[44] Tenno R. (2010), Dual adaptive controls for linear system with unknown constant parameters, International Journal of Control, 83:11, 2232–2240
[45] Touchette H., Lloyd S. (2000): Information-theoretic limits of control. Phys Rev Lett. 2000 Feb 7;84(6):1156–9.
[46] Touchette H., Lloyd S. (2004): Information-theoretic approach to the study of control systems. Phys. A 331, 140–172.
[47] Tsai Y. A., Casiello F. A., Loparo K. A. (1992): Discrete-time entropy formulation of optimal and adaptive control problems. IEEE Trans. Aut. Cont. vol 37, No. 7.
[48] Tse E. (1974): Adaptive Dual Control Methods. Annals of Economic and Social Measurement, Vol. 3. No. 1. (1974)
[49] Tse, E., and Bar-Shalom, Y. (1973), An Actively Adaptive Control for Linear Systems with Random Parameters via the Dual Control Approach, IEEE Trans. Aut. Control, 18, 109–117.
[50] Tse, E., and Bar-Shalom, Y. (1976), Actively Adaptive Control for Nonlinear Stochastic Systems, Proceedings of the IEEE, 64, 1172–1181.
[51] Tse, E., Bar-Shalom, Y., Meier L. (1973a): Wide sense adaptive dual control for nonlinear stochastic systems. IEEE Trans. Aut. Control, 18, 98–108.
[52] Uciński D., Patan M. (2016): D-optimal spatio-temporal sampling design for identification of distributed parameter systems In proc of 55th IEEE Conference on Decision and Control December 12–14, Las Vegas, USA, 3985–3990.
[53] Wittenmark B. (1995): Adaptive dual control methods; an overview. Proc. Of 5th IFAC symposium of adaptive systems in control and signal processing. Budapest, pp. 67–72.
[54] Zabczyk J. (1996): Chance and decision. Stochastic control in discrete time. Quaderni Scuola Normale di Pisa
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bania, P. (2017). Simple example of dual control problem with almost analytical solution. In: Mitkowski, W., Kacprzyk, J., Oprzędkiewicz, K., Skruch, P. (eds) Trends in Advanced Intelligent Control, Optimization and Automation. KKA 2017. Advances in Intelligent Systems and Computing, vol 577. Springer, Cham. https://doi.org/10.1007/978-3-319-60699-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-60699-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60698-9
Online ISBN: 978-3-319-60699-6
eBook Packages: EngineeringEngineering (R0)