Semantic Reasoning for Scene Interpretation

  • Lars B. W. Jensen
  • Emre Baseski
  • Sinan Kalkan
  • Nicolas Pugeault
  • Florentin Wörgötter
  • Norbert Krüger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5329)


In this paper, we propose a hierarchical architecture for representing scenes, covering 2D and 3D aspects of visual scenes as well as the semantic relations between the different aspects. We argue that labeled graphs are a suitable representational framework for this representation and demonstrate its potential by two applications. As a first application, we localize lane structures by the semantic descriptors and their relations in a Bayesian framework. As the second application, which is in the context of vision based grasping, we show how the semantic relations can be associated to actions that allow for grasping without using any object knowledge.


cognitive vision semantic reasoning bayesian classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aarno, D., Sommerfeld, J., Kragic, D., Pugeault, N., Kalkan, S., Wörgötter, F., Kraft, D., Krüger, N.: Early reactive grasping with second order 3D feature relations. In: Lee, S., Suh, I.H., Kim, M.S. (eds.) Recent Progress in Robotics; ViableRobotic Service to Human, ICAR 2007. LNCIS. Springer, Heidelberg (2007)Google Scholar
  2. 2.
    Echtler, F., Huber, M., Pustka, D., Keitler, P., Klinker, G.: Splitting the scene graph – using spatial relationship graphs instead of scene graphs in augmented reality. In: GRAPP 2008: Int. Conference on Computer Graphics Theory and Applications (2008)Google Scholar
  3. 3.
    Jaromczyk, J.W., Toussaint, G.T.: Relative neighborhood graphs and their relatives. Proceedings of the IEEE 80(9), 1502–1517 (1992)CrossRefGoogle Scholar
  4. 4.
    Mucke, E.P.: Shapes and implementations in three-dimensional geometry. Technical report, University of Illinois at Urbana-Champaign, Champaign, IL, USA (1993)Google Scholar
  5. 5.
    Korting, T.S., Fonseca, L.M.G., Dutra, L.V., da Silva, F.C.: Image re-segmentation – a new approach applied to urban imagery. In: VISAPP 2008: Int. Conference on Computer Vision Theory and Applications (2008)Google Scholar
  6. 6.
    Tremeau, A., Colantoni, P.: Regions adjacency graph applied to color image segmentation. IEEE Transactions on Image Processing 9(4), 735–744 (2000)CrossRefGoogle Scholar
  7. 7.
    Hancock, E.R., Wilson, R.C.: Graph-based methods for vision: A yorkist manifesto. In: Proc. of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, London, UK, pp. 31–46. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Kovesi, P.: Image features from phase congruency. Videre: Journal of Computer Vision Research 1(3), 1–26 (1999)Google Scholar
  9. 9.
    Nagel, H.H.: On the estimation of optic flow: Relations between different approaches and some new results. Artificial Intelligence 33, 299–324 (1987)CrossRefGoogle Scholar
  10. 10.
    Sabatini, S.P., Gastaldi, G., Solari, F., Diaz, J., Ros, E., Pauwels, K., Hulle, K.M.M.V., Pugeault, N., Krüger, N.: Compact and accurate early vision processing in the harmonic space. In: International Conference on Computer Vision Theory and Applications (VISAPP) (2007)Google Scholar
  11. 11.
    Felsberg, M., Sommer, G.: The monogenic signal. IEEE Transactions on Signal Processing 49(12), 3136–3144 (2001)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Krüger, N., Lappe, M., Wörgötter, F.: Biologically motivated multi-modal processing of visual primitives. Interdisciplinary Journal of Artificial Intelligence & the Simulation of Behaviour, AISB Journal 1(5), 417–427 (2004)Google Scholar
  13. 13.
    Pugeault, N.: Early Cognitive Vision: Feedback Mechanisms for the Disambiguation of Early Visual Representation. Ph.D thesis, Informatics Institute, University of Göttingen (2008)Google Scholar
  14. 14.
    Kalkan, S., Pugeault, N., Krüger, N.: Perceptual operations and relations between 2d or 3d visual entities. Technical Report 2007-3, Technical report of the Robotics Group, Maersk Institute, University of Southern Denmark (2007)Google Scholar
  15. 15.
    Piegl, L., Tiller, W.: The NURBS book, 2nd edn. Springer, New York (1997)CrossRefMATHGoogle Scholar
  16. 16.
    Kalkan, S., Yan, S., Krüger, V., Wörgötter, F., Krüger, N.: A signal-symbol loop mechanism for enhanced edge extraction. In: VISAPP 2008: Int. Conference on Computer Vision Theory and Applications (2008)Google Scholar
  17. 17.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, Inc., San Francisco (1988)MATHGoogle Scholar
  18. 18.
    Popović, M.: An early grasping reflex in a cognitive robot vision system. Master’s thesis, University of Southern Denmark (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Lars B. W. Jensen
    • 1
  • Emre Baseski
    • 1
  • Sinan Kalkan
    • 2
  • Nicolas Pugeault
    • 3
  • Florentin Wörgötter
    • 2
  • Norbert Krüger
    • 1
  1. 1.University of Southern DenmarkOdenseDenmark
  2. 2.University of GöttingenGöttingenGermany
  3. 3.University of EdinburghEdinburghUnited Kingdom

Personalised recommendations