Making Entailment Set Changes Explicit Improves the Understanding of Consequences of Ontology Authoring Actions

  • Nicolas MatentzogluEmail author
  • Markel Vigo
  • Caroline Jay
  • Robert Stevens
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10024)


The consequences of adding or removing axioms are difficult to apprehend for ontology authors using the Web Ontology Language (OWL). Consequences of modelling actions range from unintended inferences to outright defects such as incoherency or even inconsistency. One of the central ontology authoring activities is verifying that a particular modelling step has had the intended consequences, often with the help of reasoners. For users of Protégé, this involves, for example, exploring the inferred class hierarchy.

We explore the hypothesis that making changes to key entailment sets explicit improves verification compared to the standard static hierarchy/frame-based approach. We implement our approach as a Protégé plugin and conduct an exploratory study to isolate the authoring actions for which users benefit from our approach. In a second controlled study we address our hypothesis and find that, for a set of key authoring problems, making entailment set changes explicit improves the understanding of consequences both in terms of correctness and speed, and is rated as the preferred way to track changes compared to a static hierarchy/frame-based view.


Ontology engineering Ontology authoring Reasoning 



This research has been funded by the EPSRC project WhatIf: Answering “What if...” questions for Ontology Authoring, EPSRC reference EP/J014176/1.


  1. 1.
    Consortium, T.G.O.: Gene ontology annotations and resources. Nucleic Acids Res. 41(D1), D530–D535 (2013)CrossRefGoogle Scholar
  2. 2.
    Denaux, R., Thakker, D., Dimitrova, V., Cohn, A.G.: Interactive semantic feedback for intuitive ontology authoring. In: Formal Ontology in Information Systems - Proceedings of the Seventh International Conference, FOIS 2012, Gray, Austra, 24–27 July 2012, pp. 160–173 (2012)Google Scholar
  3. 3.
    Dzbor, M., Motta, E., Buil, C., Gomez, J.M., Görlitz, O., Lewen, H.: Developing ontologies in OWL: an observational study. In: Proceedings of the OWLED 2006 Workshop on OWL: Experiences and Directions, Athens, Georgia, USA, 10–11 November 2006 (2006)Google Scholar
  4. 4.
    Horridge, M., Bail, S., Parsia, B., Sattler, U.: The cognitive complexity of OWL justifications. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011, Part I. LNCS, vol. 7031, pp. 241–256. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Knublauch, H., Fergerson, R.W., Noy, N.F., Musen, M.A.: The protégé OWL plugin: an open development environment for semantic web applications. In: McIlraith, S.A., Plexousakis, D., Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 229–243. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Lambrix, P., Habbouche, M., Pérez, M.: Evaluation of ontology development tools for bioinformatics. Bioinformatics 19(12), 1564–1571 (2003)CrossRefGoogle Scholar
  7. 7.
    Lee, M., Matentzoglu, N., Parsia, B., Sattler, U.: A multi-reasoner, justification-based approach to reasoner correctness. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 393–408. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-25010-6_26 CrossRefGoogle Scholar
  8. 8.
    McGuinness, D.L., Patel-Schneider, P.F.: Usability issues in knowledge representation systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence and Tenth Innovative Applications of Artificial Intelligence Conference, AAAI 1998, IAAI 1998, 26–30 July 1998, Madison, Wisconsin, USA, pp. 608–614 (1998)Google Scholar
  9. 9.
    Parsia, B., Matentzoglu, N., Gonçalves, R.S., Glimm, B., Steigmiller, A.: The OWL reasoner evaluation (ORE) 2015 competition report. In: Proceedings of the 11th International Workshop on Scalable Semantic Web Knowledge Base Systems (SSWS-2015), Bethlehem, Pennsylvania, USA, 11 October 2015 (2015)Google Scholar
  10. 10.
    Parvizi, A., Mellish, C., van Deemter, K., Ren, Y., Pan, J.Z.: Selecting ontology entailments for presentation to users. In: KEOD 2014 - Proceedings of the International Conference on Knowledge Engineering and Ontology Development, Rome, Italy, 21–24 October 2014, pp. 382–387 (2014)Google Scholar
  11. 11.
    Tudorache, T., Nyulas, C.I., Noy, N.F., Musen, M.A.: WebProtégé: a collaborative ontology editor and knowledge acquisition tool for the Web. Semant. Web 4, 89–99 (2013)Google Scholar
  12. 12.
    Vigo, M., Bail, S., Jay, C., Stevens, R.D.: Overcoming the pitfalls of ontology authoring: strategies and implications for tool design. Int. J. Hum.-Comput. Stud. 72(12), 835–845 (2014)CrossRefGoogle Scholar
  13. 13.
    Vigo, M., Jay, C., Stevens, R.: Design insights for the next wave ontology authoring tools. In: CHI Conference on Human Factors in Computing Systems, CHI 2014, Toronto, ON, Canada, April 26–May 01 2014, pp. 1555–1558 (2014)Google Scholar
  14. 14.
    Vigo, M., Jay, C., Stevens, R.: Constructing conceptual knowledge artefacts: activity patterns in the ontology authoring process. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, Seoul, Republic of Korea, 18–23 April 2015, pp. 3385–3394 (2015)Google Scholar
  15. 15.
    Wang, H., Tudorache, T., Dou, D., Noy, N.F., Musen, M.A.: Analysis of user editing patterns in ontology development projects. In: Meersman, R., Panetto, H., Dillon, T., Eder, J., Bellahsene, Z., Ritter, N., Leenheer, P., Dou, D. (eds.) ODBASE 2013. LNCS, vol. 8185, pp. 470–487. Springer, Heidelberg (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Nicolas Matentzoglu
    • 1
    Email author
  • Markel Vigo
    • 1
  • Caroline Jay
    • 1
  • Robert Stevens
    • 1
  1. 1.The University of ManchesterManchesterUK

Personalised recommendations