Inference and Learning for Active Sensing, Experimental Design and Control

  • Hendrik Kueck
  • Matt Hoffman
  • Arnaud Doucet
  • Nando de Freitas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5524)


In this paper we argue that maximum expected utility is a suitable framework for modeling a broad range of decision problems arising in pattern recognition and related fields. Examples include, among others, gaze planning and other active vision problems, active learning, sensor and actuator placement and coordination, intelligent human-computer interfaces, and optimal control. Following this remark, we present a common inference and learning framework for attacking these problems. We demonstrate this approach on three examples: (i) active sensing with nonlinear, non-Gaussian, continuous models, (ii) optimal experimental design to discriminate among competing scientific models, and (iii) nonlinear optimal control.


Utility Function Expected Utility Markov Decision Process Active Sensing Sequential Decision 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bernardo, J.: Expected information as expected utility. The Annals of Statistics 7(3), 686–690 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bertsekas, D.P.: Dynamic Programming and Optimal Control. Athena Scientific (1995)Google Scholar
  3. 3.
    Chaloner, K., Verdinelli, I.: Bayesian experimental design: A review. Statistical Science 10(3), 273–304 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Dayan, P., Hinton, G.E.: Using EM for reinforcement learning. Neural Computation 9, 271–278 (1997)CrossRefzbMATHGoogle Scholar
  5. 5.
    Green, P.: Reversible jump Markov Chain Monte Carlo computation and Bayesian model determination. Biometrika 82(4), 711–732 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Hoffman, M., Doucet, A., de Freitas, N., Jasra, A.: Bayesian policy learning with trans-dimensional MCMC. In: NIPS (2007)Google Scholar
  7. 7.
    Hoffman, M., Doucet, A., de Freitas, N., Jasra, A.: On solving general state-space sequential decision problems using inference algorithms. Technical Report TR-2007-04, University of British Columbia, Computer Science (2007)Google Scholar
  8. 8.
    Kueck, H., de Freitas, N., Doucet, A.: SMC samplers for Bayesian optimal nonlinear design. Nonlinear Statistical Signal Processing (2006)Google Scholar
  9. 9.
    Loredo, T.J.: Bayesian adaptive exploration. Bayesian Inference And Maximum Entropy Methods In Science And Engineering, 330–346 (2003)Google Scholar
  10. 10.
    Müller, P., Sansó, B., de Iorio, M.: Optimal Bayesian design by inhomogeneous Markov chain simulation. Journal of the American Statistical Association 99, 788–798 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Myung, J.I., Pitt, M.A.: Optimal experimental design for model discrimination (under review)Google Scholar
  12. 12.
    Rubin, D., Hinton, S., Wenzel, A.: The precise time course of retention. Journal of experimental psychology. Learning, memory, and cognition 25(5), 1161–1176 (1999)CrossRefGoogle Scholar
  13. 13.
    Rubin, D.C., Wenzel, A.E.: One hundred years of forgetting: A quantitative description of retention. Psychological review 103, 734–760 (1996)CrossRefGoogle Scholar
  14. 14.
    Shoham, Y., Leyton-Brown, K.: Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2009)zbMATHGoogle Scholar
  15. 15.
    Toussaint, M., Storkey, A.: Probabilistic inference for solving discrete and continuous state Markov Decision Processes. In: ICML (2006)Google Scholar
  16. 16.
    von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behaviour. Princeton University Press, Princeton (1947)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Hendrik Kueck
    • 1
  • Matt Hoffman
    • 1
  • Arnaud Doucet
    • 1
  • Nando de Freitas
    • 1
  1. 1.Department of Computer ScienceUBCCanada

Personalised recommendations