Advertisement

Autonomous Robots

, Volume 18, Issue 1, pp 59–80 | Cite as

Active Appearance-Based Robot Localization Using Stereo Vision

  • J.M. Porta
  • J.J. Verbeek
  • B.J.A. Kröse
Article

Abstract

A vision-based robot localization system must be robust: able to keep track of the position of the robot at any time even if illumination conditions change and, in the extreme case of a failure, able to efficiently recover the correct position of the robot. With this objective in mind, we enhance the existing appearance-based robot localization framework in two directions by exploiting the use of a stereo camera mounted on a pan-and-tilt device. First, we move from the classical passive appearance-based localization framework to an active one where the robot sometimes executes actions with the only purpose of gaining information about its location in the environment. Along this line, we introduce an entropy-based criterion for action selection that can be efficiently evaluated in our probabilistic localization system. The execution of the actions selected using this criterion allows the robot to quickly find out its position in case it gets lost. Secondly, we introduce the use of depth maps obtained with the stereo cameras. The information provided by depth maps is less sensitive to changes of illumination than that provided by plain images. The main drawback of depth maps is that they include missing values: points for which it is not possible to reliably determine depth information. The presence of missing values makes Principal Component Analysis (the standard method used to compress images in the appearance-based framework) unfeasible. We describe a novel Expectation-Maximization algorithm to determine the principal components of a data set including missing values and we apply it to depth maps. The experiments we present show that the combination of the active localization with the use of depth maps gives an efficient and robust appearance-based robot localization system.

localization appearance-based modeling active vision depth maps stereo vision 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arbel, T. and Ferrie, F.P. 1999. Entropy-based gaze planning. In Proceedings of the Second IEEE Workshop on Perception for Mobile Agents, in Association with the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,Fort Collins, Colorado.Google Scholar
  2. Arras, K.O., Castellanos, J.A., and Siegwart, R. 2002. Feature-based multi-hypothesis localization and tracking for mobile robots using geometric constraints. In Proceedings of IEEE International Conference on Robotics and Automation, pp. 1371–1377.Google Scholar
  3. Bunschoten, R. and Kr¨ ose, B. 2003. Robust scene reconstruction from an omnidirectional vision system. IEEE Transactions on Robotics and Automation, 19(2):351–357.Google Scholar
  4. Burgard, W., Fox, D., Hennig, D., and Schmidt, T. 1996. Estimating the absolute position of a mobile robot using position probability grids. In Proceedings of the National Conference on Artificial Intelligence (AAAI), pp. 896–901.Google Scholar
  5. Burgard, W., Fox, D., and Thrun, S. 1997. Active mobile robot localization. In Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI).Google Scholar
  6. Chow, Y.H. and Chung, R. 2002. VisionBug: A hexapod robot controlled by stereo cameras. Autonomous Robots, 13:259–276.Google Scholar
  7. Clemen, R.T. 1989. Combining forecasts: A review of annotated bibliography. International Journal of Forecasting, 5:559–583.Google Scholar
  8. Clemen, R.T. and Winkler, R.L. 1999. Combining probability distributions from experts in risk analysis. Risk Analysis, 19(2):187–203.Google Scholar
  9. Cobzas, D. and Zhang, H. 2001. Mobile robot localization using planar patches and a stereo panoramic model. In Proceedings of the Vision Interface Annual Conference, Ottawa, Canada, pp. 94–99.Google Scholar
  10. Cobzas, D., Zhang, H., and Jagersand, M. 2003. Image-based localization with depth-enhanced image map. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1570–1575.Google Scholar
  11. Cox, I.J. and Leonard, J.J. 1994. Modeling a dynamic environment using a multiple hypothesis approach. Artificial Intelligence, 66(2):311–344.Google Scholar
  12. Crowley, J., Wallner, F., and Schiele, B. 1998. Position estimation using principal components of range data. Robotics and Autonomous Systems, 23(4):267–276.Google Scholar
  13. Crowley, J., Wallner, F., and Schiele, B. 1998. Position estimation using principal components of range data. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 3121–3128.Google Scholar
  14. Crowley, J.L. and Pourraz, F. 2001. Continuity properties of the appearance manifold for mobile robot position estimation. Image and Vision Computing, 19:741–752.Google Scholar
  15. Davison, A.J. 1999. Mobile robot navigation using active vision. PhD thesis, Robotics Research Group, University of Oxford.Google Scholar
  16. Dellaert, F., Burgard, W., Fox, D., and Thrun, S. 1999. Using the CONDENSATION algorithm for robust, vision-based mobile robot localization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  17. Dellaert, F., Fox, D., Burgard, W., and Thrun, S. 1999. Monte-Carlo localization for mobile robots. In Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1322–1328.Google Scholar
  18. Dempster, A.P., Laird, N.M., and Rubin, D.B. 1977. Maximum-likelihood for incomplete data via the EM algorithm. J. Royal Statistical Society, Series B, 39(1):1–38.Google Scholar
  19. Elfes, A. 1987. Sonar-based real-world mapping and navigation. IEEE Journal of Robotics and Automation, 3(3):249–265.Google Scholar
  20. Faugeras, O. and Luong, Q.-T. 2001. The geometry of multiple images. MIT Press.Google Scholar
  21. Fox, D., Burgard, W., and Thrun, S. 1998. Active markov localization for mobile robots.vol. 25, pp. 195–207.Google Scholar
  22. Fox, D., Burgard, W., and Thrun, S. 1999. Markov localization for mobile robots in dynamic environments. Journal of Artificial Intelligence Research, 11:391–427.Google Scholar
  23. Fox, D., Burgard, W., Thrun, S., and Cremers, A.B. 1998. Position estimation for mobile robots in dynamic environments. In Proc. of the Fifteenth National Conference on Artificial Intelligence (AAAI-98).Google Scholar
  24. Gaspar, J., Winters, N., and Santos-Victor, J. 2000. Vision-based navigation and environmental representations with an omnidirectional camera. IEEE Transactions on Robotics and Automation, 16(6):890–898.Google Scholar
  25. Genest, C. and Zidek, J.V. 1986. Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 1(1):114–148.Google Scholar
  26. Gonzalez, R. and Woods, R. 1992. Digital image processing. Addison-Wesley Publishing Company.Google Scholar
  27. Horswill, I. 1998. Artificial Intelligence and Mobile Robots.Chapter The Polly System, MIT Press: Cambridge, MA, pp. 125–139.Google Scholar
  28. Isard, M. and Blake, A. 1998. Condensation-Conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1):5–28.Google Scholar
  29. Jensfelt, P. and Kristensen, S. 2001. Active global localization for a mobile robot using multiple hypothesis tracking. IEEE Transactions on Robotics and Automation, 17(5):748–760.Google Scholar
  30. Jogan, M. and Leonardis, A. 2000. Robust localization using eigenspace of spinning-images. In Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina, pp. 37–44.Google Scholar
  31. Jogan, M., Leonardis, A., Wildenauer, H., and Bischof, H. 2002. Mobile robot localization under varying illumination. In Proceedings of the International Conference on Pattern Recognition (ICPR), Quebec, Canada, pp. 741–744.Google Scholar
  32. Jolliffe, I.T. 2002. Principal component analysis. Springer Verlag.Google Scholar
  33. Kaelbling, L.P., Cassandra, A.R., and Kurien, J.A. 1996. Acting under uncertainty: Discrete bayesian models for mobile-robot navigation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Google Scholar
  34. Kleinberg, J. 1994. The localization problem for mobile robots. In Proceedings of the IEEE Symposium on Foundations of Computer Science, pp. 521–531.Google Scholar
  35. Konolige, K. 1997. Small vision system: Hardware and implementation. In Proceedings of the 8th International Symposium on Robotics Research, Japan.Google Scholar
  36. Kristensen, S. and Jensfelt, P. 2003. An experimental comparison of localization methods, the MHL session. In Proceedings of the International Conference on Robotics and Intelligent Systems (IROS), Las Vegas, USA, pp. 992–997.Google Scholar
  37. Kr¨ ose, B.J.A. and Bunschoten, R. 1999. Probabilistic localization by appearance models and active vision. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2255–2260.Google Scholar
  38. Kr¨ ose, B.J.A., Vlassis, N., and Bunschoten, R. 2002. Omnidirectional vision for appearance-based robot localization. Lecture Notes in Computer Science, 2238:39–50.Google Scholar
  39. Kr¨ ose, B.J.A., Vlassis, N., Bunschoten, R., and Motomura, Y. 2001. A probabilistic model for appearance-based robot localization. Image and Vision Computing, 19(6):381–391.Google Scholar
  40. Kwok, C., Fox, D., and Meila, M. 2003. Adaptive real-time particle filters for robot localization. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2836–2841.Google Scholar
  41. Lamon, P., Nourbakhsh, I., Jensen, B., and Siegwart, R. 2001. Deriving and matching image fingerprint sequences for mobile robot localization. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seoul, Korea, pp. 1111–1116.Google Scholar
  42. Lenser, S. and Veloso, M. 2000. Sensor resetting localization for poorly modelled mobile robots. In Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 1225–1232.Google Scholar
  43. Leonard, J.J. and Durrant-Whyte, H.F. 1991. Mobile robot localization by tracking geometric beacons. IEEE Transactions on Robotics and Automation, 7(3):376–382.Google Scholar
  44. Little, J., Lu, J., and Murray, D. 1998. Selecting stable image features for robot localization using stereo. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1072–1077.Google Scholar
  45. Maeda, S., Kuno, Y., and Shirai, Y. 1997. Active vision based on eigenspace analysis. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1018–1023.Google Scholar
  46. Menegatti, E., Zoccarato, M., Pagello, E., and Ishiguro, H. 2003. Image-based Monte-carlo localisation without a map. In Proceedings Conference of the Italian Association of Artificial Intelligence.Google Scholar
  47. Minka, T. 1998. Expected-Maximization as lower bound maximization. http://www.stat.cmu.edu/~minka/ papers/em.html.Google Scholar
  48. Montemerlo, M., Thrun, S., Koller, D., and Webgreit, B. 2002. Fast-SLAM: A factored solution to the simultaneous localization and mapping problem. In Proceedings of the 18th AAAI National Conference on Artificial Intelligence, pp. 593–598.Google Scholar
  49. Montemerlo, M., Thrun, S., Koller, D., and Webgreit, B. 2003. Fast-SLAM2.0: An improved particle filtering algorithm for simultaneous localization and mapping that provably converges. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).Google Scholar
  50. Moravec, H.P. 1988. Sensor fusion in certainty grids for mobile robots. AI Magazine, 9(2):61–74.Google Scholar
  51. Murase, H., Kimura, F., Yoshimura, M., and Miyake, Y. 1981. An improvement of the autocorrelation matrix in the pattern matching method and its application to handprinted HIRAGANA recognition. Transactions of the IECEJ, J64-D(3):276–283.Google Scholar
  52. Murase, H. and Nayar, S.K. 1994. Illumination planning for object recognition using parametric eigenspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 16(12):1219–1227.Google Scholar
  53. Murase, H. and Nayar, S.K. 1995. Visual learning and recognition of 3-D objects from appearance. International Journal of Computer Vision, 14:5–24.Google Scholar
  54. Murray, D. and Little, J. 2000. Using real-time stereo vision for mobile robot navigation. Autonomous Robots, 8(2):161–171.Google Scholar
  55. Olson, C.F. 2000. Probabilistic self-localization for mobile robots. IEEE Transactions on Robotics and Automation, 16(1):55–66.Google Scholar
  56. Pitt, M.K. and Shephard, N. 1999. Filtering via simulation: Auxiliary particle filters. J. Amer. Statist. Assoc., 94(446):590–599.Google Scholar
  57. Porta, J.M. and Kr¨ ose, B.J.A. 2004. Appearance-based concurrent map building and localization. In Proceedings of the 8th International Conference on Intelligent Autonomous Systems (IAS), pp. 1022–1029.Google Scholar
  58. Roweis, S. 1998. EMAlgorithms for PCA and SPCA. In Advances in Neural Information Processing Systems,vol. 10, The MIT Press, pp. 626–632.Google Scholar
  59. Roy, N. and Gordon, G. 2003. Exponential family PCAfo belief compression in POMDPs. In S. Becker, S. Thrun, and K. Obermayer (Eds.), Advances in Neural Information Processing Systems. MIT Press.Google Scholar
  60. Sim, R. and Dudek, G. 2003. Comparing image-based localization methods. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI), Acapulco, MexicoGoogle Scholar
  61. Sirovich, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 4(3):519–524.Google Scholar
  62. Sobel, I. 1990. Machine vision of three-dimensional scenes. Chapter An Isotropic 3 × 3 Images Gradient Operator, Academic Press, pp. 376–379.Google Scholar
  63. Sujan, V.A. and Dubowsky, S. 2002. Visually built task models for robot teams in unstructured environments. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Washington D.C., USA, pp. 1782–1787.Google Scholar
  64. Terwijn, B., Porta, J.M., and Kr¨ ose, B.J.A. 2004. A particle filter to estimate non-markovian states. In Proceedings of the 8th International Conference on Intelligent Autonomous Systems (IAS), pp. 1062–1069.Google Scholar
  65. Thrun, S., Burgard, W., and Fox, D. 1998. Aprobabilistic approach to concurrent mapping and localization for mobile robots. Machine Learning, 31:29–53.Google Scholar
  66. Thrun, S., Fox, D., and Burgard, W. 2000. Monte Carlo localization with mixture proposal distribution. In Proceedings of the AAAI National Conference on Artificial Intelligence, pp. 859–865.Google Scholar
  67. Thrun, S., Fox, D., Burgard, W., and Dellaert, F. 2001. Robust Monte Carlo localization for mobile robots. Artificial Intelligence, 28(1/2):99–141.Google Scholar
  68. van Breemen, A.J.N., Crucq, K., Kr¨ ose, B.J.A., Nuttin, M., Porta, J.M., and Demeester, E. 2003. A user-interface robot for ambient intelligent environments. In Proceedings of the 1st International Workshop on Advances in Service Robotics (ASER), Bardolino, March 13-15.Google Scholar
  69. Verbeek, J.J. 2002. Probabilistic PCA with missing values. Matlab source code. http://www.science.uva.nl/~jverbeek/software.Google Scholar
  70. Vlassis, N., Terwijn, B., and Kr¨ ose, B.J.A. 2002. Auxiliary particle filter robot localization from high-dimensional sensor. observations. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Washington D.C., USA, pp. 7–12.Google Scholar
  71. Wand, M.P. and Jones, M.C. 1995. Kernel smoothing. Chapman & Hall: London.Google Scholar

Copyright information

© Kluwer Academic Publishers 2005

Authors and Affiliations

  • J.M. Porta
    • 1
  • J.J. Verbeek
    • 1
  • B.J.A. Kröse
    • 1
  1. 1.IAS GroupUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations