Abstract
We consider the general problem of optimal stochastic control of a dedicated-platform that processes one primary function or task (target-task). The dedicated-platform has two modes of action at each period of time: it can attempt to process the target-task at the given period of time, or suspend the target-task for later completion. We formulate the optimal trade-off between the processing cost and the latency in completion of the target-task as a Partially Observable Markov Decision Process (POMDP). By reformulating this POMDP as a Markovian search problem, we prove that the optimal control policies are threshold in nature. Threshold policies are computationally efficient and inexpensive to implement in real time systems. Numerical results demonstrate the effectiveness of these threshold based operating algorithms as compared to non-optimal heuristic algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
MacPhee, I., Jordan, B.: Optimal search for a moving target. Probability in the Engineering and Information Sciences 9, 159–182 (1995)
Ross, S.: Introduction to Stochastic Dynamic Programming. Academic Press, London (2000)
Weber, R.R.: Optimal search for a randomly moving object. Journal of Applied Probability 23, 708–717 (1986)
Benkoski, S.J., Monticino, M.G., Weisinger, J.R.: A survey of the search theory literature. Naval Research Logistics 38, 469–494 (1991)
Johnston, L.A., Krishnamurthy, V.: Optimality of threshold transmission policies in Gilbert Elliott fading channels. In: IEEE International Conference on Communications, ICC 2003, May, vol. 2, pp. 1233–1237 (2003)
Bertsekas, D.P.: Dynamic Programming and Optimal Control, 2nd edn. Athena Scientific (2000)
Lovejoy, A.: A survey of algorithmic methods for partially observed Markov decision processes. Annals of Operations Research 28, 47–66 (1991)
Zhang, D., Wasserman, K.M.: Energy efficient data communication over fading channels. In: IEEE Wireless Communications and Networking Conference, pp. 986–991 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Farrokh, A., Krishnamurthy, V. (2005). Optimal Threshold Policies for Operation of a Dedicated-Platform with Imperfect State Information – A POMDP Framework. In: Godo, L. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2005. Lecture Notes in Computer Science(), vol 3571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11518655_18
Download citation
DOI: https://doi.org/10.1007/11518655_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27326-4
Online ISBN: 978-3-540-31888-0
eBook Packages: Computer ScienceComputer Science (R0)