Abstract
Resource allocation is a widely studied class of problems in Operation Research and Artificial Intelligence. Specially, constrained stochastic resource allocation problems, where the assignment of a constrained resource do not automatically imply the realization of the task. This kind of problems are generally addressed with Markov Decision Processes (mdps). In this paper, we present efficient lower and upper bounds in the context of a constrained stochastic resource allocation problem for a heuristic search algorithm called Focused Real Time Dynamic Programming (frtdp). Experiments show that this algorithm is relevant for this kind of problems and that the proposed tight bounds reduce the number of backups to perform comparatively to previous existing bounds.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Bradtke, S.J., Singh, S.P.: Learning to act using real-time dynamic programming. Artificial Intelligence 72(1), 81–138 (1995)
Blodgett, D., Plamondon, P., Chaib-draa, B., Kropf, P., Bossé, E.: A method to optimize ship maneuvers for the coordination of hardkill and softkill weapons within a frigate. In: 7th International Command and Control Research and Technology Symposium (7th ICCRTS), Quebec, QC (September 2002)
Bonet, B., Geffner, H.: Faster heuristic search algorithms for planning with uncertainty and full feedback. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03) (August 2003)
Bonet, B., Geffner, H.: Labeled rtdp: Improving the convergence of real-time dynamic programming. In: Proceeding of the 13Th International Conference on Automated Planning & Scheduling (ICAPS-03), Trento, Italy, pp. 12–21 (2003), citeseer.nj.nec.com/article/bonet01planning.html
Hansen, E.A., Zilberstein, S.: lao ⋆ : A heuristic search algorithm that finds solutions with loops. Artificial Intelligence 129(1-2), 35–62 (2001), citeseer.ist.psu.edu/hansen01lao.html
McMahan, H.B., Likhachev, M., Gordon, G.J.: Bounded real-time dynamic programming: rtdp with monotone upper bounds and performance guarantees. In: ICML ’05: Proceedings of the 22nd international conference on Machine learning, Bonn, Germany, pp. 569–576. ACM Press, New York (2005), doi:10.1145/1102351.1102423
Mercier, L., Hentenryck, P.V.: Performance analysis of online anticipatory algorithms for large multistage stochastic integer programs. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, IJCAI-07 (2007)
Singh, S., Cohn, D.: How to dynamically merge markov decision processes. In: Advances in neural information processing systems, Denver, Colorado, United States, vol. 10, pp. 1057–1063. MIT Press, Cambridge (1998)
Smith, T., Simmons, R.: Focused real-time dynamic programming for mdps: Squeezing more out of a heuristic. In: Proceedings of the National Conference on Artificial Intelligence (AAAI), Boston, USA (2006)
Zhang, W.: Modeling and solving a resource allocation problem with soft constraint techniques. Technical report: wucs-2002-13, Washington University, Saint-Louis, Missouri (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Besse, C., Plamondon, P., Chaib-draa, B. (2007). R-FRTDP: A Real-Time DP Algorithm with Tight Bounds for a Stochastic Resource Allocation Problem. In: Kobti, Z., Wu, D. (eds) Advances in Artificial Intelligence. Canadian AI 2007. Lecture Notes in Computer Science(), vol 4509. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72665-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-72665-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72664-7
Online ISBN: 978-3-540-72665-4
eBook Packages: Computer ScienceComputer Science (R0)