Skip to main content
SpringerLink
Log in
Book cover

OTM Confederated International Conferences "On the Move to Meaningful Internet Systems"

OTM 2009: On the Move to Meaningful Internet Systems: OTM 2009 Workshops pp 594–603Cite as

Ontology Evaluation through Usability Measures

An Experiment with the SUS Scale in the Legal Domain

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 5872))

Abstract

Current ontology methodologies offer guidance towards knowledge acquisition, ontology development (design and conceptualization), formalization, evaluation, evolution and maintenance. Nevertheless, these methodologies describe most of expert involvements within ontology validation rather vaguely. The use of tailored usability methods for ontology evaluation could offer the establishment of certain quality measurements and aid the evaluation of modelling decisions, prior ontology implementation. This paper describes the experimental evaluation of a legal ontology, the Ontology of Professional Judicial Knowledge (OPJK), with the SUS questionnaire, a usability evaluation questionnaire tailored to ontology evaluation.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Gómez-Pérez, A.: Evaluation of ontologies. International Journal of Intelligent Systems 16(3), 391–409 (2001)

    Article  MATH  Google Scholar 

  2. Sure, Y.: Methodology, Tools and Case Studies for Ontology Based Knowledge Management. PhD thesis, Fakultät für Wirschaftwissenschaften der Universität Fridericiana zu Karlsruhe (2003)

    Google Scholar 

  3. Grüninger, M., Fox, M.: Methodology for the Design and Evaluation of Ontologies. In: Proceedings of Int. Joint Conf. AI 1995, Workshop on Basic Ontological Issues in Knowledge Sharing (1995)

    Google Scholar 

  4. Velardi, P., Navigli, R., Cucchiarelli, A., Neri, F.: Evaluation of OntoLearn, a methodology for automatic population of domain ontologies. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning from Text: Methods, Evaluation and Applications. Frontiers in Artificial Intelligence and Applications Series, vol. 123. IOS Press, Amsterdam (2005)

    Google Scholar 

  5. Sclano, F., Velardi, P.: Termextractor: a web application to learn the common terminology of interest groups and research communities. In: Proceedings of the 9th Conference on Terminology and Artificial Intelligence (TIA 2007), Sophia Antinopolis (October 2007)

    Google Scholar 

  6. Gómez-Pérez, A., Fernández-López, M., Corcho, O.: Ontological Engineering. In: With examples from the areas of Knowledge Management, e-Commerce and the Semantic Web. Advanced Information and Knowlege Processing. Springer, London (2003)

    Google Scholar 

  7. Hartmann, J., Spyns, P., Gibboin, A., Maynard, D., Cuel, R., Suárez-Figueroa, M.C., Sure, Y.: D.1.2.3. methods for ontology evaluation. Deliverable IST-2004-507482 KWEB D.1.2.3., EU-IST Network of Excellence (NoE) Knowledge Web Consortium (January 2005)

    Google Scholar 

  8. Brank, J., Grobelnik, M., Mladenic, D.: D.1.6.1 ontology evaluation. SEKT IST-2003-506826 Deliverable 1.6.1, SEKT, EU-IST Project Jozef Stefan Institute (June 2005)

    Google Scholar 

  9. Lozano-Tello, A., Gómez-Pérez, A., Sosa, E.: Selection of ontologies for the semantic web. In: Lovelle, J.M.C., Rodríguez, B.M.G., Aguilar, L.J., Gayo, J.E.L. (eds.) ICWE 2003. LNCS, vol. 2722, pp. 413–416. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  10. Gangemi, A., Catenaccia, C., Ciaramita, M., Lehmann, J.: Qood grid: A metaontology-based framework for ontology evaluation and selection. In: Vrandečić, D., del Carmen Suárez-Figueroa, M., Gangemi, A., Sure, Y. (eds.) Proceedings of the 4th International Workshop on Evaluation of Ontologies for the Web (EON 2006) at the 15th International World Wide Web Conference (WWW 2006), Edinburgh, Scotland, May 2006, pp. 8–15 (2006)

    Google Scholar 

  11. Guarino, N., Welty, C.: Evaluating ontological decisions with ontoclean. Communications of the ACM 45(2), 61–65 (2002)

    Article  Google Scholar 

  12. Tartir, S., Arpinar, I.B., Moore, M., Sheth, A.P., Aleman-Meza, B.: OntoQA: Metric-based ontology quality analysis. In: Proceedings of IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources (2005)

    Google Scholar 

  13. Vrandečić, D., Sure, Y.: How to design better ontology metrics. In: Franconi, E., Kifer, M., May, W. (eds.) ESWC 2007. LNCS, vol. 4519, pp. 311–325. Springer, Heidelberg (2007)

    Google Scholar 

  14. Vrandečić, D., Gangemi, A.: Unit tests for ontologies. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM 2006 Workshops. LNCS, vol. 4278, pp. 1012–1020. Springer, Heidelberg (2006)

    Google Scholar 

  15. Benjamins, V.R., Casanovas, P., Contreras, J., López-Covo, J.M., Lemus, L.: Iuriservice: An intelligent frequently asked questions system to assist newly appointed judges. In: Benjamins, V.R., Casanovas, P., Breuker, J., Gangemi, A. (eds.) Law and the Semantic Web. LNCS (LNAI), vol. 3369, pp. 201–217. Springer, Heidelberg (2005)

    Google Scholar 

  16. Casellas, N., Blázquez, M., Kiryakov, A., Casanovas, P., Poblet, M., Benjamins, V.R.: OPJK into PROTON: Legal domain ontology integration into an upper-level ontology. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM-WS 2005. LNCS, vol. 3762, pp. 846–855. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  17. Casellas, N., Casanovas, P., Vallbé, J.J., Poblet, M., Blázquez, M., Contreras, J., López-Cobo, J.M., Benjamins, V.R.: Semantic enhancement for legal information retrieval: Iuriservice performance. In: Proceedings of the Eleventh International Conference on Artificial Intelligence and Law. ICAIL 2007, Stanford Law School, California, June 4-8, pp. 49–57. Association for Computing Machinery (2007)

    Google Scholar 

  18. ISO: Human-centred design processes for interactive systems. ISO 13407:1999, International Organization for Standardization (1999)

    Google Scholar 

  19. ISO: Ergonomics of human-system interaction – usability methods supporting human-centred design. ISO Standard TR 16982:2002, International Organization for Standardization (2002)

    Google Scholar 

  20. ISO/IEC: Ease of operation of everyday products - part 1: Design requirements for context of use and user characteristics. ISO Standard 20282-1:2006, ISO/IEC (2006)

    Google Scholar 

  21. ISO/IEC: Software and system engineering - guidelines for the design and preparation of user documentation for application software. ISO Standard 18019:2004, ISO/IEC (2004)

    Google Scholar 

  22. Brooke, J.: Sus: A ’quick and dirty’ usability scale. In: Jordan, P.W., Thomas, B., McClelland, I.L., Weerdmeester, B. (eds.) Usability Evaluation In Industry, pp. 189–194. Taylor & Francis, London (1996)

    Google Scholar 

  23. Casellas, N.: Modelling Legal Knowledge Through Ontologies. OPJK: the Ontology of Professional Judicial Knowledge. PhD thesis, Faculty of Law, Universitat Autònoma de Barcelona, Barcelona (2008)

    Google Scholar 

  24. Casanovas, P., Casellas, N., Vallbé, J.: An ontology-based decision support system for judges. In: Casanovas, P., Breuker, J., Klein, M., Francesconi, E. (eds.) Channelling the legal information flood. Legal ontologies and the Semantic Web. Frontiers in Artificial Intelligence and Applications, vol. 188, pp. 165–176. IOS Press, Amsterdam (2009)

    Google Scholar 

  25. Nielsen, J.: How to conduct a heuristic evaluation. Web (2005) (Available at 01-10-2008)

    Google Scholar 

  26. Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: UPA 2004: Connecting Communities, Minneapolis, Minnesota, June 7-11 (2004)

    Google Scholar 

  27. Nielsen, J.: Usability inspection methods. In: CHI 1994: Conference companion on Human factors in computing systems, pp. 413–414. ACM, New York (1994)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Law and Technology (IDT-UAB), Universitat Autònoma de Barcelona, Bellaterra, 08193, Spain

    Núria Casellas

Authors
  1. Núria Casellas
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. STARLab, Vrije Universiteit Brussel (VUB), Bldg G/10, Pleinlaan 2, 1050, Brussels, Belgium

    Robert Meersman

  2. Facultad de Informática, Universidad Politécnica de Madrid, Campus de Montegancedo S/N, 28660, Boadilla del Monte, Madrid, Spain

    Pilar Herrero

  3. DEBII - CBS, De Laeter Way, Curtin University of Technology, 6102, Bentley, WA, Australia

    Tharam Dillon

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Casellas, N. (2009). Ontology Evaluation through Usability Measures. In: Meersman, R., Herrero, P., Dillon, T. (eds) On the Move to Meaningful Internet Systems: OTM 2009 Workshops. OTM 2009. Lecture Notes in Computer Science, vol 5872. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05290-3_73

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05290-3_73

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05289-7

  • Online ISBN: 978-3-642-05290-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation