Towards Evaluating Interactive Ontology Matching Tools

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7882)


With a growing number of ontologies used in the semantic web, agents can fully make sense of different datasets only if correspondences between those ontologies are known. Ontology matching tools have been proposed to find such correspondences. While the current research focus is mainly on fully automatic matching tools, some approaches have been proposed that involve the user in the matching process. However, there are currently no benchmarks and test methods to compare such tools. In this paper, we introduce a number of quality measures for interactive ontology matching tools, and we discuss means to automatically run benchmark tests for such tools. To demonstrate how those evaluation can be designed, we show examples on assessing the quality of interactive matching tools which involve the user in matcher selection and matcher parametrization.


Domain Expert Match System Ontology Match Partial Alignment Ontology Alignment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Aguirre, J.L., Eckert, K., Euzenat, J., Ferrara, A., van Hage, W.R., Hollink, L., Meilicke, C., Nikolov, A., Ritze, D., Scharffe, F., Shvaiko, P., Šváb Zamazal, O., Trojahn, C., Jiménez-Ruiz, E., Grau, B.C., Zapilko, B.: Results of the Ontology Alignment Evaluation Initiative 2012. In: Proc. of the 7th Int. Workshop on Ontology Matching (2012)Google Scholar
  2. 2.
    Chalupksy, H.: OntoMorph: A Translation System for Symbolic Knowledge. In: Proc. of the 17th Int. Conference on Knowledge Representation and Reasoning (2000)Google Scholar
  3. 3.
    Cruz, I.F., Stroe, C., Palmonari, M.: Interactive User Feedback in Ontology Matching Using Signature Vectors. In: Proc. of the 28th Int. Conference on Data Engineering, pp. 1321–1324 (2012)Google Scholar
  4. 4.
    David, J., Euzenat, J., Scharffe, F., Trojahn dos Santos, C.: The Alignment API 4.0. Semantic Web 2(1), 3–10 (2011)Google Scholar
  5. 5.
    de Freitas, J., Pappa, G.L., da Silva, A.S., Gonçalves, M.A., de Moura, E.S., Veloso, A., Laender, A.H.F., de Carvalho, M.G.: Active Learning Genetic Programming for Record Deduplication. In: Proc. of IEEE Congress on Evolutionary Computation, pp. 1–8 (2010)Google Scholar
  6. 6.
    Duan, S., Fokoue, A., Srinivas, K.: One Size Does Not Fit All: Customizing Ontology Alignment Using User Feedback. In: Patel-Schneider, P.F., Pan, Y., Hitzler, P., Mika, P., Zhang, L., Pan, J.Z., Horrocks, I., Glimm, B. (eds.) ISWC 2010, Part I. LNCS, vol. 6496, pp. 177–192. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Eckert, K., Meilicke, C., Stuckenschmidt, H.: Improving ontology matching using meta-level learning. In: Aroyo, L., et al. (eds.) ESWC 2009. LNCS, vol. 5554, pp. 158–172. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Ehrig, M., Staab, S., Sure, Y.: Bootstrapping ontology alignment methods with APFEL. In: Gil, Y., Motta, E., Richard Benjamins, V., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, pp. 186–200. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Euzenat, J., Shvaiko, P.: Ontology Matching. Springer (2007)Google Scholar
  10. 10.
    Falconer, S.M., Noy, N.F.: Interactive Techniques to Support Ontology Matching. In: Bellahsene, Z., Bonifati, A., Rahm, E. (eds.) Schema Matching and Mapping, pp. 29–51. Springer (2011)Google Scholar
  11. 11.
    Falconer, S.M., Storey, M.-A.: A cognitive support framework for ontology mapping. In: Aberer, K., et al. (eds.) ISWC/ASWC 2007. LNCS, vol. 4825, pp. 114–127. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Guyon, I., Cawley, G.C., Dror, G., Lemaire, V.: Results of the Active Learning Challenge. Journal of Machine Learning Research - Proceedings Track 16, 19–45 (2011)Google Scholar
  13. 13.
    Huber, J., Sztyler, T., Noessner, J., Meilicke, C.: CODI: Combinatorial Optimization for Data Integration - Results for OAEI 2011. In: Proc. of the 6th Int. Workshop on Ontology Matching (2011)Google Scholar
  14. 14.
    Isele, R., Jentzsch, A., Bizer, C.: Active learning of expressive linkage rules for the web of data. In: Brambilla, M., Tokuda, T., Tolksdorf, R. (eds.) ICWE 2012. LNCS, vol. 7387, pp. 411–418. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Jiménez-Ruiz, E., Grau, B.C., Horrocks, I.: LogMap and LogMapLt Results for OAEI 2012. In: Proc. of the 7th Int. Workshop on Ontology Matching (2012)Google Scholar
  16. 16.
    Jiménez-Ruiz, E., Grau, B.C., Zhou, Y., Horrocks, I.: Large-scale interactive ontology matching: Algorithms and implementation. In: Proc. of the 20th European Conference on Artificial Intelligence (2012)Google Scholar
  17. 17.
    Lambrix, P., Edberg, A.: Evaluation of Ontology Merging Tools in Bioinformatics. In: Proc. of the Pacific Symposium on Biocomputing, pp. 589–600 (2003)Google Scholar
  18. 18.
    McGuinness, D., Fikes, R., Rice, J., Wilder, S.: An environment for merging and testing large ontologies. In: Proc. of the 17th Int. Conference on Principles of Knowledge Representation and Reasoning (2000)Google Scholar
  19. 19.
    Miller, R., Haas, L., Hernandez, M.: Schema mapping as query discovery. In: Proc. of the 26th Int. Conference on Very Large Databases (2000)Google Scholar
  20. 20.
    Noy, N.F., Musen, M.A.: The PROMPT suite: interactive tools for ontology merging and mapping. Int. Journal of Human-Computer Studies 59(6), 983–1024 (2003)CrossRefGoogle Scholar
  21. 21.
    Paulheim, H.: WeSeE-Match results for OEAI 2012. In: Proc. of the 7th Int. Workshop on Ontology Matching (2012)Google Scholar
  22. 22.
    Paulheim, H., Rebstock, M., Fengel, J.: Context-Sensitive Referencing for Ontology Mapping Disambiguation. In: Proc. of the 2007 Workshop on Context and Ontologies Representation and Reasoning, pp. 47–56 (2007)Google Scholar
  23. 23.
    Ritze, D., Paulheim, H.: Towards an automatic parameterization of ontology matching tools based on example mappings. In: Proc. of the 6th Int. Workshop on Ontology Matching (2011)Google Scholar
  24. 24.
    Sarasua, C., Simperl, E., Noy, N.F.: CrowdMap: Crowdsourcing ontology alignment with microtasks. In: Cudré-Mauroux, P., et al. (eds.) ISWC 2012, Part I. LNCS, vol. 7649, pp. 525–541. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Settles, B.: Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison (2009)Google Scholar
  26. 26.
    Shchekotykhin, K., Friedrich, G., Fleiss, P., Rodler, P.: Interactive ontology debugging: two query strategies for efficient fault localization. Web Semantics: Science, Services and Agents on the World Wide Web 12(13), 88–103 (2012)CrossRefGoogle Scholar
  27. 27.
    Shi, F., Li, J., Tang, J., Xie, G., Li, H.: Actively learning ontology matching via user interaction. In: Bernstein, A., Karger, D.R., Heath, T., Feigenbaum, L., Maynard, D., Motta, E., Thirunarayan, K. (eds.) ISWC 2009. LNCS, vol. 5823, pp. 585–600. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  28. 28.
    Shvaiko, P., Euzenat, J.: Ontology Matching: State of the Art and Future Challenges. IEEE Transactions on Knowledge and Data Engineering 25(1), 158–176 (2013)CrossRefGoogle Scholar
  29. 29.
    Shvaiko, P., Euzenat, J.: Ten challenges for ontology matching. In: Meersman, R., Tari, Z. (eds.) OTM 2008, Part II. LNCS, vol. 5332, pp. 1164–1182. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Siorpaes, K., Thaler, S., Simperl, E.: SpotTheLink: A Game for Ontology Alignment. In: Proc. 6th Conference for Professional Knowledge Management (2011)Google Scholar
  31. 31.
    Thayasivam, U., Chaudhari, T., Doshi, P.: Optima+ Results for OAEI 2012. In: Proc. of the 7th Int. Workshop on Ontology Matching (2012)Google Scholar
  32. 32.
    To, H.-V., Ichise, R., Le, H.-B.: An Adaptive Machine Learning Framework with User Interaction for Ontology Matching. In: Proc. of the IJCAI 2009 Workshop on Information Integration on the Web, pp. 35–40 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Research Group Data and Web ScienceUniversity of MannheimGermany
  2. 2.Knowledge Engineering GroupTechnische Universität DarmstadtGermany

Personalised recommendations