A Configurable Evaluation Framework for Node Embedding Techniques

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11762)


While Knowledge Graphs (KG) are graph shaped by nature, most traditional data mining and machine learning (ML) software expect data in a vector form. Several node embedding techniques have been proposed to represent each node in the KG as a low-dimensional feature vector. A node embedding technique should preferably be task independent. Therefore, when a new method has been developed, it should be tested on the tasks it was designed for as well as on other tasks. We present the design and implementation of a ready to use evaluation framework to simplify the node embedding technique testing phase. The provided tests range from ML tasks, semantic tasks to semantic analogies.


Evaluation framework Node embedding Machine learning Semantic tasks 


  1. 1.
    Cochez, M., Ristoski, P., Ponzetto, S.P., Paulheim, H.: Global RDF vector space embeddings. In: 16th International Semantic Web Conference ISWC (2017)Google Scholar
  2. 2.
    Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. Knowl.-Based Syst. 151, 78–94 (2018)CrossRefGoogle Scholar
  3. 3.
    Hoffart, J., Seufert, S., Nguyen, D.B., Theobald, M., Weikum, G.: KORE: keyphrase overlap relatedness for entity disambiguation. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management (2012)Google Scholar
  4. 4.
    Lee, M.D., Welsh, M.: An empirical evaluation of models of text document similarity. In: XXVII Annual Conference of the Cognititive Science Society (2005)Google Scholar
  5. 5.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: 27th Annual Conference on Neural Information Processing Systems (2013)Google Scholar
  6. 6.
    Ristoski, P., Rosati, J., Di Noia, T., De Leone, R., Paulheim, H.: RDF2Vec: RDF graph embeddings and their applications. Semant. Web J. 10(4), 721–752 (2018). in pressCrossRefGoogle Scholar
  7. 7.
    Ristoski, P., de Vries, G.K.D., Paulheim, H.: A collection of benchmark datasets for systematic evaluations of machine learning on the semantic web. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9982, pp. 186–194. Springer, Cham (2016). Scholar
  8. 8.
    Rulinda, J., de Dieu Tugirimana, J., Nzaramba, A., Aila, F.O., Langat, G.K.: An integrated platform to evaluate graph embedding. Int. J. Sci. Eng. Res. 9, 665–676 (2018) Google Scholar
  9. 9.
    Wang, D., Cui, P., Zhu, W.: Structural deep network embedding. In: Proceedings of the 22nd International Conference on Knowledge Discovery and Data Mining (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of SalernoFiscianoItaly
  2. 2.Fraunhofer FITSankt AugustinGermany
  3. 3.Information Systems and DatabasesRWTH Aachen UniversityAachenGermany
  4. 4.Faculty of Information TechnologyUniversity of JyvaskylaJyvaskylaFinland
  5. 5.IBM Research AlmadenSan JoseUSA

Personalised recommendations