Knowledge-Driven Saliency: Attention to the Unseen

  • M. Zaheer Aziz
  • Michael Knopf
  • Bärbel Mertsching
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6915)


This paper deals with attention in 3D environments based upon knowledge-driven cues. Using learned 3D scenes as top-down influence, the proposed system is able to mark high saliency to locations occupied by objects that are new, changed, or even missing from their location as compared to the already learned situation. The proposal addresses a system level solution covering learning of 3D objects and scenes using visual, range and odometry sensors, storage of spatial knowledge using multiple-view theory from psychology, and validation of scenes using recognized objects as anchors. The proposed system is designed to handle the complex scenarios of recognition with partially visible objects during revisit to the scene from an arbitrary direction. Simulation results have shown success of the designed methodology under ideal sensor readings from range and odometry sensors.


Visual Attention Object Representation Sensory Memory Canonical View View Descriptor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Congnitive Psychology 12, 97–136 (1980)CrossRefGoogle Scholar
  2. 2.
    Posner, M., Snyder, C., Davidson, B.: Attention and the detection of signals. Journal of Experimental Psychology: General 109(2), 160–174 (1980)CrossRefGoogle Scholar
  3. 3.
    Chen, X., Zelinsky, G.: Real-world visual search is dominated by top-down guidance. Vision Research 46(24), 4118–4133 (2006)CrossRefGoogle Scholar
  4. 4.
    Chen, X., Zelinsky, G.J.: Is visual search a top-down or bottom-up process? Journal of Vision 6, 447 (2006)CrossRefGoogle Scholar
  5. 5.
    Blanz, V., Tarr, M., Bülthoff, H., Vetter, T.: What object attributes determine canonical views? PERCEPTION-LONDON 28, 575–600 (1999)CrossRefGoogle Scholar
  6. 6.
    Endres, D., Vintila, F., Bruce, N., Bouecke, J., Kornprobst, P., Neumann, H., Giese, M.: Hooligan detection: the effects of saliency and expert knowledge. Perception 39 ECVP Abstract Supplement 39, 193 (2010)Google Scholar
  7. 7.
    Olivers, C., Meijer, F., Theeuwes, J.: Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology 32(5), 1243–1265 (2006)Google Scholar
  8. 8.
    Palmer, S.: Vision science: Photons to phenomenology, vol. 2. MIT Press, Cambridge (1999)Google Scholar
  9. 9.
    Oman, C.M., Shebilske, W.L., Richards, J.T., Tubré, T.C., Bealli, A.C., Natapoffi, A.: Three dimensional spatial memory and learning in real and virtual environments. Spatial Cognition and Computation 2, 355–372 (2000)CrossRefGoogle Scholar
  10. 10.
    Shelton, A.L., Mcnamara, T.P.: Spatial memory and perspective taking. Memory & Cognition 32, 416–426 (2004)CrossRefGoogle Scholar
  11. 11.
    Aivar, M.P., Hayhoe, M.M., Chizk, C.L., Mruczek, R.E.B.: Spatial memory and saccadic targeting in a natural task. Journal of Vision 5, 177–193 (2005)CrossRefGoogle Scholar
  12. 12.
    Hoshino, E., Taya, F., Mogi, K.: Memory formation of object representation: Natural scenes. In: Wang, R., et al. (eds.) Advances in Cognitive Neurodynamics, pp. 457–462 (2008)Google Scholar
  13. 13.
    Follet, B., Le Meur, O., Baccino, T.: Modeling visual attention on scenes. Studia Informatica Universalis 8, 150–167 (2010)Google Scholar
  14. 14.
    Fay, R., Kaufmann, U., Markert, H., Palm, G.: Adaptive visual attention based object recognition. In: Proceedings of the IEEE SMC UK-RI Chapter Conference (2005)Google Scholar
  15. 15.
    Rasolzadeh, B., Tavakoli, A.T., Eklundh, J.O.: Attention in cognitive systems. theories and systems from an interdisciplinary viewpoint, pp. 123–140. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Navalpakkam, V., Itti, L.: An integrated model of top-down and bottom-up attention for optimizing detection speed. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2049–2056. IEEE, Los Alamitos (2006)Google Scholar
  17. 17.
    Lee, S., Kim, K., Kim, J., Kim, M., Yoo, H.: Familiarity based unified visual attention model for fast and robust object recognition. Pattern Recognition 43, 1116–1128 (2010)CrossRefzbMATHGoogle Scholar
  18. 18.
    Atmosukarto, I., Shapiro, L.: A learning approach to 3D object representation for classification. Structural, Syntactic, and Statistical Pattern Recognition, 267–276 (2010)Google Scholar
  19. 19.
    Denton, T., Demirci, M., Abrahamson, J., Shokoufandeh, A., Dickinson, S.: Selecting canonical views for view-based 3-D object recognition. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 273–276. IEEE, Los Alamitos (2004)CrossRefGoogle Scholar
  20. 20.
    Büker, U., Hartmann, G.: Knowledge-based view control of a neural 3-D object recognition system. In: ICPR, vol. 4, pp. 24–29. IEEE, Los Alamitos (2002)Google Scholar
  21. 21.
    Ruhnke, M., Steder, B., Grisetti, G., Burgard, W.: Unsupervised learning of 3d object models from partial views. In: ICRA 2009, pp. 801–806. IEEE, Los Alamitos (2009)Google Scholar
  22. 22.
    Salamanca, S., Cerrada, C., Adan, A., Cerrada, J., Adán, M.: Free-Shaped Object Recognition Method from Partial Views Using Weighted Cone Curvatures. Progress in Pattern Recognition, Image Analysis and Applications, 222–232 (2005)Google Scholar
  23. 23.
    Kutter, O., Hilker, C., Simon, A., Mertsching, B.: Modeling and simulating mobile robots environments. In: 3rd International Conference on Computer Graphics Theory and Applications (GRAPP 2008), Funchal, Portugal (2008)Google Scholar
  24. 24.
    Kotthäuser, T., Mertsching, B.: Validating Vision and Robotic Algorithms for Dynamic Real World Environments. In: Second International Conference on Simulation, Modeling and Programming for Autonomous Robot, SIMPAR (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • M. Zaheer Aziz
    • 1
  • Michael Knopf
    • 1
  • Bärbel Mertsching
    • 1
  1. 1.GET LABUniversität PaderbornPaderbornGermany

Personalised recommendations