Summary
This paper presents the Parameterised POMDP (PPOMDP) algorithm: a method for planning in the space of continuous parameterised functions. The novel contribution is an approach to transitioning parameterised beliefs using Monte Carlo methods. By re-using prediction and observation calculations, the transition function can be computed efficiently. An analysis of scalability suggests that the approach is likely to scale to physically larger environments than algorithms which rely on an underlying discretisation. Experimental results in a simulated robot navigation problem show that the algorithm compares favourably with existing approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aberdeen, D., Baxter, J.: Scaling internal-state policy-gradient methods for POMDPs. In: Proc. Intl. Conf. on Machine Learning, pp. 3–10 (2002)
Bertsekas, D.: Dynamic Programming and Optimal Control, vol. 1. Athena Scientific, Belmont (2000)
Brooks, A.: Parametric POMDPs for planning in continuous state spaces. Technical Report ACFR-TR-2007001, Australian Centre for Field Robotics, University of Sydney (2007)
Brooks, A., Makarenko, A., Williams, S., Durrant-Whyte, H.: Parametric POMDPs for planning in continuous state spaces. Robotics and Autonomous Systems 54(11), 887–897 (2006)
Durrant-Whyte, H., Bailey, T.: Simultaneous localisation and mapping (SLAM): Part I - the essential algorithms. Robotics and Automation Magazine 13(2), 99–110 (2006)
Gordon, G.: Stable function approximation in dynamic programming. In: Proc. Intl. Conf. on Machine Learning, pp. 261–268 (1995)
Hauskrecht, M.: Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 13, 33–94 (2000)
Kaelbling, L., Littman, M., Cassandra, A.: Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 99–134 (1998)
LaValle, S.: Planning Algortihms. Cambridge University Press, Cambridge (2006)
Moore, D.: Simplicial Mesh Generation with Applications. PhD thesis, Report no. 92-1322, Cornell University (1992)
Nourbakhsh, I., Powers, R., Birchfield, S.: Dervish: An office-navigating robot. AI Magazine 16(2), 53–60 (1995)
Pineau, J., Gordon, G., Thrun, S.: Point-based value iteration: An anytime algorithm for POMDPs. In: Proc. Intl. Joint Conf. on Artificial Intelligence, pp. 1025–1032 (2003)
Roy, N.: Finding Approximate POMDP Solutions Through Belief Compression. PhD thesis, Robotics Institute, Carnegie Mellon University (2003)
Spaan, M., Vlassis, N.: Perseus: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence research 24, 195–220 (2005)
Thrun, S.: Monte carlo POMDPs. Advances in Neural Information Processing Systems, 1064–1070 (2000)
Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT Press, Cambridge (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brooks, A., Williams, S. (2010). A Monte Carlo Update for Parametric POMDPs. In: Kaneko, M., Nakamura, Y. (eds) Robotics Research. Springer Tracts in Advanced Robotics, vol 66. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14743-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-14743-2_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14742-5
Online ISBN: 978-3-642-14743-2
eBook Packages: EngineeringEngineering (R0)