Combining Statistical and Symbolic Reasoning for Active Scene Categorization

  • Thomas Reineking
  • Niclas Schult
  • Joana Hois
Part of the Communications in Computer and Information Science book series (CCIS, volume 128)


One of the reasons why humans are so successful at interpreting everyday situations is that they are able to combine disparate forms of knowledge. Most artificial systems, by contrast, are restricted to a single representation and hence fail to utilize the complementary nature of multiple sources of information. In this paper, we introduce an information-driven scene categorization system that integrates common sense knowledge provided by a domain ontology with a learned statistical model in order to infer a scene class from recognized objects. We show how the unspecificity of coarse logical constraints and the uncertainty of statistical relations and the object detection process can be modeled using Dempster-Shafer theory and derive the resulting belief update equations. In addition, we define an uncertainty minimization principle for adaptively selecting the most informative object detectors and present classification results for scenes from the LabelMe image database.


Recognition Rate Object Detection Description Logic Object Class Domain Ontology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 145–175 (2001)CrossRefzbMATHGoogle Scholar
  2. 2.
    Schill, K., Zetzsche, C., Hois, J.: A belief-based architecture for scene analysis: From sensorimotor features to knowledge and ontology. Fuzzy Sets and Systems 160, 1507–1516 (2009)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Martínez Mozos, Ó., Triebel, R., Jensfelt, P., Rottmann, A., Burgard, W.: Supervised semantic labeling of places using information extracted from sensor data. Robotics and Autonomous Systems 55, 391–402 (2007)CrossRefGoogle Scholar
  4. 4.
    Kollar, T., Roy, N.: Utilizing object-object and object-scene context when planning to find things. In: International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  5. 5.
    Maillot, N.E., Thonnat, M.: Ontology based complex object recognition. Image and Vision Computing 26, 102–113 (2008)CrossRefGoogle Scholar
  6. 6.
    Russell, B., Torralba, A., Murphy, K., Freeman, W.: LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision 77, 157–173 (2008)CrossRefGoogle Scholar
  7. 7.
    Baader, F., Calvanese, D., McGuinness, D., Nardi, D., Patel-Schneider, P.: The Description Logic Handbook. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  8. 8.
    Motik, B., Patel-Schneider, P.F., Grau, B.C.: OWL 2 Web Ontology Language: Direct Semantics. Technical report, W3C (2008),
  9. 9.
    Horrocks, I., Kutz, O., Sattler, U.: The Even More Irresistible SROIQ. In: Knowledge Representation and Reasoning (KR). AAAI Press, Menlo Park (2006)Google Scholar
  10. 10.
    Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: Pellet: A practical OWL-DL reasoner. In: Web Semantics: Science, Services and Agents on the World Wide Web, vol. 5, pp. 51–53 (2007)Google Scholar
  11. 11.
    Kutz, O., Lücke, D., Mossakowski, T.: Heterogeneously Structured Ontologies—Integration, Connection, and Refinement. In: Meyer, T., Orgun, M.A. (eds.) Advances in Ontologies, Proc. of the Knowledge Representation Ontology Workshop (KROW 2008), pp. 41–50. ACS (2008)Google Scholar
  12. 12.
    Masolo, C., Borgo, S., Gangemi, A., Guarino, N., Oltramari, A.: Ontologies library. WonderWeb Deliverable D18, ISTC-CNR (2003)Google Scholar
  13. 13.
    Konev, B., Lutz, C., Walther, D., Wolter, F.: Formal properties of modularisation. In: Stuckenschmidt, H., Parent, C., Spaccapietra, S. (eds.) Modular Ontologies. LNCS, vol. 5445, pp. 25–66. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Vernon, D.: Cognitive vision: The case for embodied perception. Image and Vision Computing 26, 127–140 (2008)CrossRefGoogle Scholar
  15. 15.
    Horridge, M., Patel-Schneider, P.F.: Manchester OWL syntax for OWL 1.1. In: OWL: Experiences and Directions (OWLED 2008), DC, Gaithersberg, Maryland (2008)Google Scholar
  16. 16.
    Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: Pellet: A practical OWL-DL reasoner. In: Web Semantics: Science, Services and Agents on the World Wide Web, vol. 5, pp. 51–53 (2007)Google Scholar
  17. 17.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)zbMATHGoogle Scholar
  18. 18.
    Smets, P., Kennes, R.: The transferable belief model. Artificial intelligence 66, 191–234 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Smets, P.: Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem. International Journal of Approximate Reasoning 9, 1–35 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Delmotte, F., Smets, P.: Target identification based on the transferable belief model interpretation of Dempster-Shafer model. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 34, 457–471 (2004)CrossRefGoogle Scholar
  21. 21.
    Dubois, D., Prade, H.: On the unicity of Dempster’s rule of combination. International Journal of Intelligent Systems 1, 133–142 (1986)CrossRefzbMATHGoogle Scholar
  22. 22.
    Smets, P.: The nature of the unnormalized beliefs encountered in the transferable belief model. In: Uncertainty in Artificial Intelligence, pp. 292–297 (1992)Google Scholar
  23. 23.
    Pal, N., Bezdek, J., Hemasinha, R.: Uncertainty measures for evidential reasoning II: A new measure of total uncertainty. International Journal of Approximate Reasoning 8, 1–16 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Smets, P.: Decision making in the TBM: the necessity of the pignistic transformation. International Journal of Approximate Reasoning 38, 133–147 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Reineking, T., Schult, N., Hois, J.: Evidential combination of ontological and statistical information for active scene classification. In: International Conference on Knowledge Engineering and Ontology Development (KEOD) (2009)Google Scholar
  26. 26.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Annals of Statistics 28 (1998), 2000Google Scholar
  27. 27.
    Henderson, J., Hollingworth, A.: High-level scene perception. Annual Review of Psychology 50, 243–271 (1999)CrossRefGoogle Scholar
  28. 28.
    Schill, K., Umkehrer, E., Beinlich, S., Krieger, G., Zetzsche, C.: Scene analysis with saccadic eye movements: Top-down and bottom-up modeling. Journal of Electronic Imaging 10, 152–160 (2001)CrossRefGoogle Scholar
  29. 29.
    Zetzsche, C., Wolter, J., Schill, K.: Sensorimotor representation and knowledge-based reasoning for spatial exploration and localisation. Cognitive Processing 9, 283–297 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Thomas Reineking
    • 1
  • Niclas Schult
    • 1
  • Joana Hois
    • 2
  1. 1.Cognitive NeuroinformaticsUniversity of BremenBremenGermany
  2. 2.Research Center on Spatial Cognition SFB/TR 8University of BremenBremenGermany

Personalised recommendations