Skip to main content
SpringerLink
Log in

Analysis and Prediction of User Editing Patterns in Ontology Development Projects

  • Original Article
  • Published:
Journal on Data Semantics

Abstract

The development of real-world ontologies is a complex undertaking, commonly involving a group of domain experts with different expertise that work together in a collaborative setting. These ontologies are usually large scale and have complex structures. To assist in the authoring process, ontology tools are key at making the editing process as streamlined as possible. Being able to predict confidently what the users are likely to do next as they edit an ontology will enable us to focus and structure the user interface accordingly and to facilitate more efficient interaction and information discovery. In this paper, we use data mining, specifically the association rule mining, to investigate whether we are able to predict the next editing operation that a user will make based on the change history. We simulated and evaluated continuous prediction across time using sliding window model. We used the association rule mining to generate patterns from the ontology change logs in the training window and tested these patterns on logs in the adjacent testing window. We also evaluated the impact of different training and testing window sizes on the prediction accuracies. At last, we evaluated our prediction accuracies across different user groups and different ontologies. Our results indicate that we can indeed predict the next editing operation a user is likely to make. We will use the discovered editing patterns to develop a recommendation module for our editing tools, and to design user interface components that better fit with the user editing behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agichtein, E., Brill, E., Dumais, S.: Improving web search ranking by incorporating user behavior information. In: ACM SIGIR International Conference on Research and Development in Information Retrieval, pp. 19–26 (2006).

  2. Agrawal, R., Imielinski, T., Swami, A.N.: Mining association rules between sets of items in large databases. In: ACM SIGMOD International Conference on Management of Data, pp. 207–216 (1993).

  3. Agrawal, R., Srikant, R.: Fast algorithms for mining association rules. In: International Conference on Very Large Data Bases, pp. 487–499 (1994).

  4. Borges, J., Levene, M.: Data mining of user navigation patterns. In: Revised Papers from the International Workshop on Web Usage Analysis and User Profiling, pp. 92–111 (2000).

  5. Cosley, D., Frankowski, D., Terveen, L., Riedl, J.: Suggestbot: Using intelligent task routing to help people find work in wikipedia. In: International Conference on Intelligent User Interfaces, pp. 32–41 (2007).

  6. De Leenheer, P., Debruyne, C., Peeters, J.: Towards social performance indicators for community-based ontology evolution. In: Workshop on Collaborative Construction, Management and Linking of Structured Knowledge at the International Semantic Web Conference (2009).

  7. Falconer, S.M., Tudorache, T., Noy, N.F.: An analysis of collaborative patterns in large-scale ontology development projects. In: International Conference on Knowledge Capture, pp. 25–32 (2011).

  8. Gibson, A., Wolstencroft, K., Stevens, R.: Promotion of ontological comprehension: Exposing terms and metadata with web 2.0. In: Workshop on Social and Collaborative Construction of Structured Knowledge (2007).

  9. GO Consortium (2001) Creating the Gene Ontology resource: design and implementation. Genome Research 11(8):1425–1433

    Article  Google Scholar 

  10. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. SIGKDD Explorations 11(1):10–18

    Article  Google Scholar 

  11. Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers (2001).

  12. Hartung M, Kirsten T, Gross A, Rahm E (2009) Onex: Exploring changes in life science ontologies. BMC Bioinformatics 10(1):250

    Article  Google Scholar 

  13. Hipp J, Güntzer U, Nakhaeizadeh G (2000) Algorithms for association rule mining - A general survey and comparison. SIGKDD Explorations 2(1):58–64

    Article  Google Scholar 

  14. Malone J, Stevens R (2013) Measuring the level of activity in community built bio-ontologies. Journal of Biomedical Informatics 46(1):5–14

    Article  Google Scholar 

  15. Noy, N.F., Chugh, A., Liu, W., Musen, M.A.: A framework for ontology evolution in collaborative environments. In: International Semantic Web Conference, pp. 544–558 (2006).

  16. Noy NF, Sintek M, Decker S, Crubézy M, Fergerson RW, Musen MA (2001) Creating semantic web contents with protégé-2000. IEEE Intelligent Systems 16(2):60–71

    Article  Google Scholar 

  17. Perera D, Kay J, Koprinska I, Yacef K, Zaïane OR (2009) Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on Knowledge and Data Engineering 21(6):759–772

    Article  Google Scholar 

  18. Pesquita, C., Couto, F.M.: Predicting the extension of biomedical ontologies. PLoS Computational Biology 8(9) (2012).

  19. Pöschko, J., Strohmaier, M., Tudorache, T., Noy, N.F., Musen, M.A.: Pragmatic analysis of crowd-based knowledge production systems with iCAT analytics: Visualizing changes to the ICD-11 ontology. In: AAAI Spring Symposium on Wisdom of the Crowds, pp. 59–64 (2012).

  20. Rector, A.L., Drummond, N., Horridge, M., Rogers, J., Knublauch, H., Stevens, R., Wang, H., Wroe, C.: OWL pizzas: Practical experience of teaching OWL-DL: Common errors & common patterns. In: International Conference on Knowledge Engineering and Knowledge Management, pp. 63–81 (2004).

  21. Sebastian, A., Noy, N.F., Tudorache, T., Musen, M.A.: A generic ontology for collaborative ontology-development workflows. In: International Conference on Knowledge Engineering and Knowledge Management, pp. 318–328 (2008).

  22. Sioutos N, de Coronado S, Haber M, Hartel F, Shaiu W, Wright L (2007) NCI Thesaurus: A semantic model integrating cancer-related clinical and molecular information. Journal of Biomedical Informatics 40(1):30–43

    Article  Google Scholar 

  23. Strohmaier M, Walk S, Pöschko J, Lamprecht D, Tudorache T, Nyulas C, Musen MA, Noy NF (2013) How ontologies are made: Studying the hidden social dynamics behind collaborative ontology engineering projects. Journal of Web Semantics 20:18–34

  24. Tudorache, T., Falconer, S.M., Nyulas, C.I., Noy, N.F., Musen, M.A.: Will semantic web technologies work for the development of ICD-11? In: International Semantic Web Conference, pp. 257–272 (2010).

  25. Tudorache T, Nyulas C, Noy NF, Musen MA (2013) WebProtégé: A collaborative ontology editor and knowledge acquisition tool for the web. Semantic Web Journal 4(1):89–99

    Google Scholar 

  26. Walk S, Pöschko J, Strohmaier M, Andrews K, Tudorache T, Noy NF, Nyulas C, Musen MA (2013) Pragmatix: An interactive tool for visualizing the creation process behind collaboratively engineered ontologies. International Journal on Semantic Web and Information Systems 9(1):45–78

    Article  Google Scholar 

  27. Walk, S., Singer, P., Strohmaier, M., Tudorache, T., Musen, M., Noy, N.: Discovering beaten paths in collaborative ontology-engineering projects using markov chains. Accepted for Publication in Journal of Biomedical Informatics (2014).

  28. Wang, H., Tudorache, T., Dou, D., Noy, N.F., Musen, M.A.: Analysis of user editing patterns in ontology development projects. In: International Conference on Ontologies, Databases and Application of Semantics, pp. 470–487 (2013).

  29. World Health Organization: International classification of diseases (ICD). http://www.who.int/classifications/icd/revision/en/. Last accessed: Oct, 2014

Download references

Acknowledgments

This work was supported by grants GM086587, EB007684, and GM103309 from the US National Institutes of Health.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, 1202 University of Oregon, Eugene, OR, 97403, USA

    Hao Wang & Dejing Dou

  2. Stanford Center for Biomedical Informatics Research, Stanford University, 1265 Welch Road, Stanford, CA, 94305, USA

    Tania Tudorache, Natalya F. Noy & Mark A. Musen

Authors
  1. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tania Tudorache
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dejing Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Natalya F. Noy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark A. Musen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Tudorache, T., Dou, D. et al. Analysis and Prediction of User Editing Patterns in Ontology Development Projects. J Data Semant 4, 117–132 (2015). https://doi.org/10.1007/s13740-014-0047-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13740-014-0047-3

Keywords

Search

Navigation