Predicting Reasoning Performance Using Ontology Metrics

  • Yong-Bin Kang
  • Yuan-Fang Li
  • Shonali Krishnaswamy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7649)


A key issue in semantic reasoning is the computational complexity of inference tasks on expressive ontology languages such as OWL DL and OWL 2 DL. Theoretical works have established worst-case complexity results for reasoning tasks for these languages. However, hardness of reasoning about individual ontologies has not been adequately characterised. In this paper, we conduct a systematic study to tackle this problem using machine learning techniques, covering over 350 real-world ontologies and four state-of-the-art, widely-used OWL 2 reasoners. Our main contributions are two-fold. Firstly, we learn various classifiers that accurately predict classification time for an ontology based on its metric values. Secondly, we identify a number of metrics that can be used to effectively predict reasoning performance. Our prediction models have been shown to be highly effective, achieving an accuracy of over 80%.


Description Logic Reasoning Performance Reasoning Task Feature Selector Ontology Engineering 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Baader, F., Brandt, S., Lutz, C.: Pushing the \(\mathcal{EL}\) envelope further. In: Clark, K., Patel-Schneider, P.F. (eds.) Proceedings of the OWLED 2008 DC Workshop on OWL: Experiences and Directions (2008)Google Scholar
  2. 2.
    Baader, F., Sattler, U.: An overview of tableau algorithms for description logics. Studia Logica 69(1), 5–40 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bock, J., Haase, P., Ji, Q., Volz, R.: Benchmarking OWL reasoners. In: ARea2008 - Workshop on Advancing Reasoning on the Web: Scalability and Commonsense (June 2008)Google Scholar
  4. 4.
    Dentler, K., Cornet, R., ten Teije, A., de Keizer, N.: Comparison of reasoners for large ontologies in the OWL 2 EL profile. Semantic Web Journal 2(2), 71–87 (2011)Google Scholar
  5. 5.
    García, J., García, F., Therón, R.: Defining Coupling Metrics among Classes in an OWL Ontology. In: García-Pedrajas, N., Herrera, F., Fyfe, C., Benítez, J.M., Ali, M. (eds.) IEA/AIE 2010, Part II. LNCS, vol. 6097, pp. 12–17. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Gardiner, T., Horrocks, I., Tsarkov, D.: Automated benchmarking of description logic reasoners. In: Proceedings of the 2006 International Workshop on Description Logics (DL 2006) (2006)Google Scholar
  7. 7.
    Grau, B.C., Horrocks, I., Motik, B., Parsia, B., Patel-Schneider, P., Sattler, U.: OWL 2: The next step for OWL. Journal of Web Semantics: Science, Services and Agents on the World Wide Web 6, 309–322 (2008)CrossRefGoogle Scholar
  8. 8.
    Heinsohn, J., Kudenko, D., Nebel, B., Profitlich, H.-J.: An empirical analysis of terminological representation systems. In: Proceedings of the Tenth National Conference on Artificial Intelligence, AAAI 1992, pp. 767–773. AAAI Press (1992)Google Scholar
  9. 9.
    Horrocks, I., Patel-Schneider, P.F.: DL systems comparison (summary relation). In: Proceedings of the 1998 International Workshop on Description Logics (DL 1998). CEUR Workshop Proceedings, vol. 11. (1998)Google Scholar
  10. 10.
    Horrocks, I., Patel-Schneider, P.F., van Harmelen, F.: From \(\mathcal{SHIQ}\) and RDF to OWL: The Making of a Web Ontology Language. Journal of Web Semantics 1(1), 7–26 (2003)CrossRefGoogle Scholar
  11. 11.
    Kang, Y.-B., Li, Y.-F., Krishnaswamy, S.: A rigorous characterization of reasoning performance – a tale of four reasoners. In: Proceedings of the 1st International Workshop on OWL Reasoner Evaluation (ORE 2012) (June 2012)Google Scholar
  12. 12.
    Pan, Z.: Benchmarking DL reasoners using realistic ontologies. In: Grau, B.C., Horrocks, I., Parsia, B., Patel-Schneider, P.F. (eds.) OWLED. CEUR Workshop Proceedings, vol. 188. (2005)Google Scholar
  13. 13.
    Ren, Y., Pan, J.Z., Zhao, Y.: Soundness preserving approximation for tbox reasoning. In: Fox, M., Poole, D. (eds.) AAAI. AAAI Press (2010)Google Scholar
  14. 14.
    Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information Processing and Management: an International Journal 24(5), 513–523 (1988)CrossRefGoogle Scholar
  15. 15.
    Shearer, R., Motik, B., Horrocks, I.: HermiT: A Highly-Efficient OWL Reasoner. In: Proceedings of the 5th International Workshop on OWL: Experiences and Directions, OWLED 2008 (2008)Google Scholar
  16. 16.
    Sirin, E., Parsia, B., Grau, B., Kalyanpur, A., Katz, Y.: Pellet: A practical OWL-DL reasoner. Web Semantics: Science, Services and Agents on the World Wide Web 5(2), 51–53 (2007)CrossRefGoogle Scholar
  17. 17.
    Tempich, C., Volz, R.: Towards a benchmark for semantic web reasoners - an analysis of the DAML ontology library. In: Sure, Y., Corcho, Ó. (eds.) EON. CEUR Workshop Proceedings, vol. 87. (2003)Google Scholar
  18. 18.
    Thomas, E., Pan, J.Z., Ren, Y.: TrOWL: Tractable OWL 2 Reasoning Infrastructure. In: Aroyo, L., Antoniou, G., Hyvönen, E., ten Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC 2010, Part II. LNCS, vol. 6089, pp. 431–435. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Tsarkov, D., Horrocks, I.: FaCT++ Description Logic Reasoner: System Description. In: Furbach, U., Shankar, N. (eds.) IJCAR 2006. LNCS (LNAI), vol. 4130, pp. 292–297. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    Wang, T.D., Parsia, B.: Ontology Performance Profiling and Model Examination: First Steps. In: Aberer, K., Choi, K.-S., Noy, N., Allemang, D., Lee, K.-I., Nixon, L.J.B., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., Cudré-Mauroux, P. (eds.) ISWC/ASWC 2007. LNCS, vol. 4825, pp. 595–608. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  21. 21.
    Witten, I.H., Frank, E.: Data mining: Practical machine learning tools and techniques with Java implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
  22. 22.
    Yang, Z., Zhang, D., Ye, C.: Evaluation metrics for ontology complexity and evolution analysis. In: IEEE International Conference on E-Business Engineering, pp. 162–170 (2006)Google Scholar
  23. 23.
    Zhang, D., Ye, C., Yang, Z.: An Evaluation Method for Ontology Complexity Analysis in Ontology Evolution. In: Staab, S., Svatek, V. (eds.) EKAW 2006. LNCS (LNAI), vol. 4248, pp. 214–221. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  24. 24.
    Zhang, H., Li, Y.-F., Tan, H.B.K.: Measuring Design Complexity of Semantic Web Ontologies. Journal of Systems and Software 83(5), 803–814 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yong-Bin Kang
    • 1
  • Yuan-Fang Li
    • 1
  • Shonali Krishnaswamy
    • 1
    • 2
  1. 1.Faculty of ITMonash UniversityAustralia
  2. 2.Institute for Infocomm ResearchA*STARSingapore

Personalised recommendations