A framework for evaluating ontology meta-matching approaches


Ontology matching has become a key issue to solve problems of semantic heterogeneity. Several researchers propose diverse techniques that can be used in distinct scenarios. Ontology meta-matching approaches are a specialization of ontology matching and have achieved good results in pairs of ontologies with different types of heterogeneities. However, developing a new ontology meta-matcher can be a costly process and a lot of experiments are often carried out to analyze the behavior of the matcher. This article presents a modularized framework that covers the main stages of the ontology meta-matching evaluation process. This framework aims to aid researchers to develop and analyze algorithms for ontology meta-matching, mainly metaheuristic-based supervised and unsupervised approaches. As the main contribution of the research, the framework proposed will facilitate the evaluation of ontology meta-matching approaches and, as the secondary contribution, a data provenance model that captures the main information generated and consumed throughout experiments is presented in the framework.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8


  1. 1.


  2. 2.


  3. 3.


  4. 4.


  5. 5.



  1. Acampora, G., Kaymak, U., Loia, V., & Vitiello, A. (2013). Applying nsga-ii for solving the ontology alignment problem. In Systems, man, and cybernetics (SMC), 2013 IEEE international conference on IEEE (pp. 1098–1103).

  2. Acampora, G., Loia, V., & Vittiello, A. (2013). Enhancing ontology alignment through a memetic aggregation of similarity measures. Information Sciences (Vol. 250, pp. 1–20). New York: Elsevier.

    Google Scholar 

  3. Banerjee, S., & Pedersen, T. (2003). Extended gloss overlaps as a measure of semantic relatedness. In Ijcai, (Vol. 3 pp. 805–810).

  4. Biniz, M., & Ayachi, R.E. (2018). Optimizing ontology alignments by using neural nsga-ii. Journal of Electronic Commerce in Organizations (JECO), IGI Global, 16(1), 29–42.

    Article  Google Scholar 

  5. Budanitsky, A., & Graeme, H. (2001). Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. Workshop on WordNet and other lexical resources. (Vol. 2).

  6. Chondrogiannis, E., Andronikou, V., Karanastasis, E., & Varvarigou, T. (2014). An intelligent ontology alignment tool dealing with complicated mismatches, (p. 1320). USA: CEUR Workshop Proceedings.

    Google Scholar 

  7. Damerau, F.J. (1964). A technique for computer detection and correction of spelling errors. Communications of the ACM, 7(3), 171–176.

    Article  Google Scholar 

  8. De Souza, J.F., Siqueira, S.W.M., & Nunes, B. (2019). A framework to aggregate multiple ontology matchers. International Journal of Web Information Systems.

  9. Euzenat, J., & Shvaiko, P. (2013). Ontology matching, 2nd edn. New York: Springer.

    Google Scholar 

  10. Euzenat, J., Shvaiko, P., & et al. (2007). Ontology matching Vol. 18. New York: Springer.

    Google Scholar 

  11. Faria, D., Pesquita, C., Santos, E., Palmonari, M., Cruz, I.F., & Couto, F.M. (2013). The agreementmakerlight ontology matching system. In OTM confederated international conferences” on the move to meaningful internet systems (pp. 527–541). New York: Springer.

  12. Freire, J., Koop, D., Santos, E., & Silva, C.T. (2008). Provenance for computational tasks: A survey Computing in Science & Engineering 10(3).

  13. Gruber, T.R. (1993). A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2), 199–220.

    Article  Google Scholar 

  14. Hertling, S., Portisch, J., & Paulheim, H. (2019). Melt-matching evaluation toolkit. In International conference on semantic systems (pp. 231–245). New York: Springer.

  15. Kuhn, H.W. (1955). The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2), 83–97.

    MathSciNet  Article  Google Scholar 

  16. Kureychik, V., & Semenova, A. (2017). Combined method for integration of heterogeneous ontology models for big data processing and analysis. In Computer Science on-line Conference (pp. 302–311). New York: Springer.

  17. Leacock, C., & Chodorow, M. (1988). Combining local context and WordNet similarity for word sense identification. WordNet: An electronic lexical database, 49(2), 265–283.

    Google Scholar 

  18. Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Zednik, S., & Zhao, J. (2013). Prov-o: the prov ontology. W3C recommendation 30.

  19. Levenshtein, V.I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady, (Vol. 10 pp. 707–710).

  20. Lim, C., Lu, S., Chebotko, A., & Fotouhi, F. (2010). Prospective and retrospective provenance collection in scientific workflow environments. In 2010 IEEE International conference on services computing, IEEE (pp. 449–456).

  21. Manning, C.D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. USA: Cambridge University Press. ISBN 0521865719.

    Google Scholar 

  22. Marjit, U. (2015). Aggregated similarity optimization in ontology alignment through multiobjective particle swarm optimization. International Journal of Advanced Research in Computer and Communication Engineering, 4(2).

  23. Martinez-Gil, J., Navas-Delgado, I., & Aldana-Montes, J.F. (2012). Maf: an ontology matching framework. Journal of Universal Computer Science, 18(2), 194–217.

    Google Scholar 

  24. McBride, B. (2002). Jena: a semantic web toolkit. IEEE Internet Computing, 6(6), 55–59.

    Article  Google Scholar 

  25. Melnik, S., Garcia-Molina, H., & Rahm, E. (2002). Similarity flooding: a versatile graph matching algorithm and its application to schema matching. In Proceedings 18th international conference on data engineering, IEEE (pp. 117–128).

  26. Mohammadi, M., Hofman, W., & Tan, Y.H. (2019). Simulated annealing-based ontology matching. ACM Transactions on Management Information Systems (TMIS), 10(1), 1–24.

    Article  Google Scholar 

  27. Ochieng, P., & Swaib, K. (2018). Large-scale ontology matching: state-of-the-art analysis (Vol. 51.4 , pp. 1–35). USA: ACM Computing Surveys (CSUR).

    Google Scholar 

  28. Otero-Cerdeira, L., Rodríguez-Martínez, F.J., & Gómez-Rodríguez, A. (2015). Ontology matching: A literature review. Expert Systems with Applications, 42(2), 949–971.

    Article  Google Scholar 

  29. Paulheim, H. (2019). Evaluating ontology matchers on real-world financial services data models.

  30. Philip R. (1995). Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th international joint conference on artificial intelligence - (IJCAI’95), (Vol. 1 pp. 448–453). San Francisco: Morgan Kaufmann Publishers Inc.

  31. Poli, R., Langdon, W., & McPhee, N. (2008). A field guide to genetic programming, 1st edn. San Francisco: Lulu Enterprises Uk Ltd.

    Google Scholar 

  32. Ramesh, M., Karthikeyan, I.P., Meenachi, I.D.N.M., & Baba, M.S. (2016). Optimizing ontology alignment for nuclear information system. International Journal of Emerging Technologies in Engineering Research, 4.

  33. Rouces, J., De Melo, G., & Hose, K. (2016). Complex schema mapping and linking data: Beyond binary predicates. LDOW@ WWW.

  34. Semenova, A., & Kureychik, V. (2016). Multi-objective particle swarm optimization for ontology alignment. In 2016 IEEE 10Th international conference on application of information and communication technologies (AICT), IEEE (pp. 1–7).

  35. Shvaiko, P., & Euzenat, J. (2013). Ontology matching: state of the art and future challenges. IEEE Transactions on Knowledge and Data Engineering, 25 (1), 158–176.

    Article  Google Scholar 

  36. Thiéblin, E., & et al. (2019). Survey on complex ontology matching. Semantic Web Preprint, 1–39.

  37. Winkler, W.E. (1999). The state of record linkage and current research problems. In Statistical research division, US census bureau. USA: Citeseer.

  38. Wu, Z., & Palmer, M. (1994). Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on association for computational linguistics, association for computational linguistics (pp. 133–138).

  39. Xue, X., & Chen, J. (2018). A preference-based multi-objective evolutionary algorithm for semiautomatic sensor ontology matching. International Journal of Swarm Intelligence Research (IJSIR), 9(2), 1–14.

    Article  Google Scholar 

  40. Xue, X., & Chen, J. (2019). Optimizing ontology alignment through hybrid population-based incremental learning algorithm. Memetic Computing, 11 (2), 209–217.

    MathSciNet  Article  Google Scholar 

  41. Xue, X., & Chen, J. (2019). Using compact evolutionary tabu search algorithm for matching sensor ontologies. Swarm and Evolutionary Computation, 48, 25–30.

    Article  Google Scholar 

  42. Xue, X., Chen, J., Chen, J., & Chen, D. (2018). A hybrid nsga-ii for matching biomedical ontology. In International conference on intelligent information hiding and multimedia signal processing (pp. 3–10). New York: Springer.

  43. Xue, X., & Jeng-Shyang, P. (2017). A segment-based approach for large-scale ontology matching. Knowledge and Information Systems, 52.2, 467–484.

    Article  Google Scholar 

  44. Xue, X., & Liu, J. (2017). Collaborative ontology matching based on compact interactive evolutionary algorithm. Knowledge-Based Systems, 137, 94–103.

    Article  Google Scholar 

  45. Xue, X., & Liu, S. (2017). Compact evolutionary algorithm based ontology meta-matching. In International conference on smart vehicular technology, transportation, communication and applications (pp. 213–221). New York: Springer.

  46. Xue, X., & Liu, J. (2018). Geo-spatial ontology matching through compact evolutionary algorithm. In International conference on smart vehicular technology, transportation, communication and applications. [S.l.] (pp. 11–18). New York: Springer.

  47. Xue, X., Lu, J., & Chen, J. (2019). Using nsga-iii for optimising biomedical ontology alignment. CAAI Transactions on Intelligence Technology, 4(3), 135–141.

    Article  Google Scholar 

  48. Xue, X., & Pan, J.S. (2018). A compact co-evolutionary algorithm for sensor ontology meta-matching. Knowledge and Information Systems, 56(2), 335–353.

    Article  Google Scholar 

  49. Xue, X., & Tang, Z. (2017). An evolutionary algorithm based ontology matching system. Journal of Information Hiding and Multimedia Signal Processing, 8(14), 551–556.

    Google Scholar 

  50. Xue, X., & Wang, Y. (2015). Optimizing ontology alignments through a Memetic algorithm using both MatchFmeasure and unanimous improvement ratio. Artificial intelligence (Vol. 223, pp. 65–81). New York: Elsevier.

    Google Scholar 

  51. Xue, X., & Wang, Y. (2016). Using memetic algorithm for instance coreference resolution. IEEE Transactions on Knowledge and Data Engineering, 28 (2), 580–591.

    Article  Google Scholar 

  52. Xue, X., Wang, Y., & Hao, W. (2014). Using moea/d for optimizing ontology alignments. Soft Computing Springer, 18(8), 1589–1601.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Nicolas Ferranti.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ferranti, N., Mouro, J.R., Mendonça, F.M. et al. A framework for evaluating ontology meta-matching approaches. J Intell Inf Syst (2020). https://doi.org/10.1007/s10844-020-00615-8

Download citation


  • Ontology matching
  • Evolutionary ontology matching
  • Ontology meta-matching
  • Metaheuristics
  • Data provenance