Active Sensorimotor Object Recognition in Three-Dimensional Space

  • David Nakath
  • Tobias Kluth
  • Thomas Reineking
  • Christoph Zetzsche
  • Kerstin Schill
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8684)


Spatial interaction of biological agents with their environment is based on the cognitive processing of sensory as well as motor information. There are many models for sole sensory processing but only a few for integrating sensory and motor information into a unifying sensorimotor approach. Additionally, neither the relations shaping the integration are yet clear nor how the integrated information can be used in an underlying representation. Therefore, we propose a probabilistic model for integrated processing of sensory and motor information by combining bottom-up feature extraction and top-down action selection embedded in a Bayesian inference approach. The integration of sensory perceptions and motor information brings about two main advantages: (i) Their statistical dependencies can be exploited by representing the spatial relationships of the sensor information in the underlying joint probability distribution and (ii) a top-down process can compute the next most informative region according to an information gain strategy. We evaluated our system in two different object recognition tasks. We found that the integration of sensory and motor information significantly improves active object recognition, in particular when these movements have been chosen by an information gain strategy.


sensorimotor object recognition Bayesian inference information gain 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gibson, J.: The ecological approach to visual perception. Houghton Mifflin, Boston (1992)Google Scholar
  2. 2.
    Neisser, U.: Cognition and reality: Principles and implications of cognitive psychology. WH Freeman/Times Books/Henry Holt & Co. (1976)Google Scholar
  3. 3.
    Bajcsy, R.: Active perception. Proceedings of the IEEE 76(8), 966–1005 (1988)CrossRefGoogle Scholar
  4. 4.
    Aloimonos, J., Weiss, I., Bandyopadhyay, A.: Active vision. International Journal of Computer Vision 1(4), 333–356 (1988)CrossRefGoogle Scholar
  5. 5.
    Ballard, D.H.: Animate vision. Artificial intelligence 48(1), 57–86 (1991)CrossRefGoogle Scholar
  6. 6.
    O’Regan, J.K., Noë, A.: A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24(5), 939–972 (2001)CrossRefGoogle Scholar
  7. 7.
    Noë, A.: Action in Perception. MIT Press (2004)Google Scholar
  8. 8.
    Prinz, W.: A common coding approach to perception and action. Springer (1990)Google Scholar
  9. 9.
    Hommel, B., Müsseler, J., Aschersleben, G., Prinz, W.: The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences 24(05), 849–878 (2001)CrossRefGoogle Scholar
  10. 10.
    O’Regan, J.K.: What it is like to see: A sensorimotor theory of perceptual experience. Synthese 129(1), 79–103 (2001)CrossRefGoogle Scholar
  11. 11.
    Noton, D., Stark, L.: Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research 11(9), 929–IN8 (1971)Google Scholar
  12. 12.
    Stark, L.W., Choi, Y.S.: Experimental metaphysics: The scanpath as an epistemological mechanism. In: Zangemeister, W.H., Stiehl, H.S., Freksa, C. (eds.) Visual Attention and Cognition. Advances in Psychology, vol. 116, pp. 3–69. North-Holland (1996)Google Scholar
  13. 13.
    Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
  14. 14.
    Cassandra, A.R., Kaelbling, L.P., Kurien, J.A.: Acting under uncertainty: Discrete Bayesian models for mobile-robot navigation. In: Proceedings of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 1996, vol. 2, pp. 963–972. IEEE (1996)Google Scholar
  15. 15.
    Stachniss, C., Grisetti, G., Burgard, W.: Information Gain-based Exploration Using Rao-Blackwellized Particle Filters. In: Robotics: Science and Systems, vol. 2, pp. 65–72 (2005)Google Scholar
  16. 16.
    Oaksford, M., Chater, N.: Information gain explains relevance which explains the selection task. Cognition 57(1), 97–108 (1995)CrossRefGoogle Scholar
  17. 17.
    Friston, K., Kilner, J., Harrison, L.: A free energy principle for the brain. Journal of Physiology-Paris 100(1-3), 70–87 (2006); heoretical and Computational Neuroscience: Understanding Brain FunctionsGoogle Scholar
  18. 18.
    Schill, K., Pöppel, E., Zetzsche, C.: Completing knowledge by competing hierarchies. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 348–352. Morgan Kaufmann Publishers Inc. (1991)Google Scholar
  19. 19.
    Schill, K.: Decision support systems with adaptive reasoning strategies. In: Freksa, C., Jantzen, M., Valk, R. (eds.) Foundations of Computer Science. LNCS, vol. 1337, pp. 417–427. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  20. 20.
    Zetzsche, C., Schill, K., Deubel, H., Krieger, G., Umkehrer, E., Beinlich, S.: Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach. In: Proceedings of the 5th International Conference of Simulation of Adaptive Behaviour, vol. 5, pp. 120–126 (1998)Google Scholar
  21. 21.
    Schill, K., Umkehrer, E., Beinlich, S., Krieger, G., Zetzsche, C.: Scene analysis with saccadic eye movements: top-down and bottom-up modeling. Journal of Electronic Imaging 10(1), 152–160 (2001)CrossRefGoogle Scholar
  22. 22.
    Zetzsche, C., Wolter, J., Schill, K.: Sensorimotor representation and knowledge-based reasoning for spatial exploration and localisation. Cognitive Processing 9(4), 283–297 (2008)CrossRefGoogle Scholar
  23. 23.
    Reineking, T., Wolter, J., Gadzicki, K., Zetzsche, C.: Bio-inspired Architecture for Active Sensorimotor Localization. In: Hölscher, C., Shipley, T.F., Olivetti Belardinelli, M., Bateman, J.A., Newcombe, N.S. (eds.) Spatial Cognition VII. LNCS, vol. 6222, pp. 163–178. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Schill, K., Zetzsche, C., Hois, J.: A belief-based architecture for scene analysis: From sensorimotor features to knowledge and ontology. Fuzzy Sets and Systems 160(10), 1507–1516 (2009)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 Object Category Dataset. Technical report, California Institute of Technology (2007)Google Scholar
  26. 26.
    Oliva, A., Torralba, A.: Building the gist of a scene: The role of global image features in recognition. Prorgess in Brain Research 155, 23–36 (2006)CrossRefGoogle Scholar
  27. 27.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  28. 28.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Computer Vision and Image Understanding 110(3), 346–359 (2008)CrossRefGoogle Scholar
  29. 29.
    Roy, N., Burgard, W., Fox, D., Thrun, S.: Coastal navigation-mobile robot navigation with uncertainty in dynamic environments. In: Proceedings of the 1999 IEEE International Conference on Robotics and Automation, vol. 1, pp. 35–40. IEEE (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • David Nakath
    • 1
  • Tobias Kluth
    • 1
  • Thomas Reineking
    • 1
  • Christoph Zetzsche
    • 1
  • Kerstin Schill
    • 1
  1. 1.Cognitive NeuroinformaticsUniversity of BremenBremenGermany

Personalised recommendations