Towards Encoding Time in Text-Based Entity Embeddings

  • Federico BianchiEmail author
  • Matteo Palmonari
  • Debora Nozza
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11136)


Knowledge Graphs (KG) are widely used abstractions to represent entity-centric knowledge. Approaches to embed entities, entity types and relations represented in the graph into vector spaces - often referred to as KG embeddings - have become increasingly popular for their ability to capture the similarity between entities and support other reasoning tasks. However, representation of time has received little attention in these approaches. In this work, we make a first step to encode time into vector-based entity representations using a text-based KG embedding model named Typed Entity Embeddings (TEEs). In TEEs, each entity is represented by a vector that represents the entity and its type, which is learned from entity mentions found in a text corpus. Inspired by evidence from cognitive sciences and application-oriented concerns, we propose an approach to encode representations of years into TEEs by aggregating the representations of the entities that occur in event-based descriptions of the years. These representations are used to define two time-aware similarity measures to control the implicit effect of time on entity similarity. Experimental results show that the linear order of years obtained using our model is highly correlated with natural time flow and the effectiveness of the time-aware similarity measure proposed to flatten the time effect on entity similarity.


  1. 1.
    Basile, P., Caputo, A., Rossiello, G., Semeraro, G.: Learning to rank entity relatedness through embedding-based features. In: Métais, E., Meziane, F., Saraee, M., Sugumaran, V., Vadera, S. (eds.) NLDB 2016. LNCS, vol. 9612, pp. 471–477. Springer, Cham (2016). Scholar
  2. 2.
    Bianchi, F., Palmonari, M.: Joint learning of entity and type embeddings for analogical reasoning with entities. In: NL4AI Workshop, Co-located with the International Conference of the Italian Association for Artificial Intelligence (AI* IA) (2017)Google Scholar
  3. 3.
    Bianchi, F., Palmonari, M., Cremaschi, M., Fersini, E.: Actively learning to rank semantic associations for personalized contextual exploration of knowledge graphs. In: Blomqvist, E., Maynard, D., Gangemi, A., Hoekstra, R., Hitzler, P., Hartig, O. (eds.) ESWC 2017. LNCS, vol. 10249, pp. 120–135. Springer, Cham (2017). Scholar
  4. 4.
    Bianchi, F., Soto, M., Palmonari, M., Cutrona, V.: Type vector representations from text: an empirical analysis. In: DL4KGS Workshop, Co-located with the ESWC (2018)Google Scholar
  5. 5.
    Bittner, T.: Approximate qualitative temporal reasoning. Ann. Math. Artif. Intell. 36(1–2), 39–80 (2002)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795 (2013)Google Scholar
  7. 7.
    Daiber, J., Jakob, M., Hokamp, C., Mendes, P.N.: Improving efficiency and accuracy in multilingual entity extraction. In: I-Semantics (2013)Google Scholar
  8. 8.
    Damasio, A.R.: Remembering when. Sci. Am. 287(3), 66–73 (2002)CrossRefGoogle Scholar
  9. 9.
    Esteban, C., Tresp, V., Yang, Y., Baier, S., Krompaß, D.: Predicting the co-evolution of event and knowledge graphs. In: 2016 19th International Conference on Information Fusion (FUSION), pp. 98–105, July 2016Google Scholar
  10. 10.
    Hamilton, W.L., Leskovec, J., Jurafsky, D.: Diachronic word embeddings reveal statistical laws of semantic change. arXiv preprint arXiv:1605.09096 (2016)
  11. 11.
    Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)CrossRefGoogle Scholar
  12. 12.
    Hoffart, J., Seufert, S., Nguyen, D.B., Theobald, M., Weikum, G.: Kore: keyphrase overlap relatedness for entity disambiguation. In: CIKM, pp. 545–554. ACM (2012)Google Scholar
  13. 13.
    Jiang, T., et al.: Encoding temporal information for time-aware link prediction. In: EMNLP, pp. 2350–2354 (2016)Google Scholar
  14. 14.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, pp. 2181–2187 (2015)Google Scholar
  15. 15.
    Ling, X., Weld, D.S.: Temporal information extraction. In: AAAI. vol. 10, pp. 1385–1390 (2010)Google Scholar
  16. 16.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)Google Scholar
  17. 17.
    Mohapatra, N., Iosifidis, V., Ekbal, A., Dietze, S., Fafalios, P.: Time-aware and corpus-specific entity relatedness. In: DL4KGS Workshop, Co-located with the ESWC (2018)Google Scholar
  18. 18.
    Nguyen, T.N., Kanhabua, N., Nejdl, W.: Multiple models for recommending temporal aspects of entities. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 462–480. Springer, Cham (2018). Scholar
  19. 19.
    Nickel, M., Rosasco, L., Poggio, T.A., et al.: Holographic embeddings of knowledge graphs. In: AAAI, pp. 1955–1961 (2016)Google Scholar
  20. 20.
    Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of ICML-11, pp. 809–816 (2011)Google Scholar
  21. 21.
    Ristoski, P., Paulheim, H.: RDF2Vec: RDF graph embeddings for data mining. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9981, pp. 498–514. Springer, Cham (2016). Scholar
  22. 22.
    Rizzo, G., Troncy, R.: NERD: a framework for unifying named entity recognition and disambiguation extraction tools. In: EACL, pp. 73–76. ACL (2012)Google Scholar
  23. 23.
    Rula, A., Palmonari, M., Harth, A., Stadtmüller, S., Maurino, A.: On the diversity and availability of temporal information in linked open data. In: Cudré-Mauroux, P., et al. (eds.) ISWC 2012. LNCS, vol. 7649, pp. 492–507. Springer, Heidelberg (2012). Scholar
  24. 24.
    Rula, A., Palmonari, M., Ngonga Ngomo, A.-C., Gerber, D., Lehmann, J., Bühmann, L.: Hybrid acquisition of temporal scopes for RDF data. In: Presutti, V., d’Amato, C., Gandon, F., d’Aquin, M., Staab, S., Tordai, A. (eds.) ESWC 2014. LNCS, vol. 8465, pp. 488–503. Springer, Cham (2014). Scholar
  25. 25.
    Sanampudi, S.K., Kumari, G.V.: Temporal reasoning in natural language processing: a survey. Int. J. Comput. Appl. 1(4), 68–72 (2010)Google Scholar
  26. 26.
    Snaider, J., McCall, R., Franklin, S.: Time production and representation in a conceptual and computational cognitive model. Cogn. Syst. Res. 13(1), 59–71 (2012)CrossRefGoogle Scholar
  27. 27.
    Szymanski, T.: Temporal word analogies: identifying lexical replacement with diachronic word embeddings. In: Association for Computational Linguistics, Vancouver, Canada. ACL, August 2017Google Scholar
  28. 28.
    Tran, N.K., Tran, T., Niederée, C.: Beyond time: dynamic context-aware entity recommendation. In: Blomqvist, E., Maynard, D., Gangemi, A., Hoekstra, R., Hitzler, P., Hartig, O. (eds.) ESWC 2017. LNCS, vol. 10249, pp. 353–368. Springer, Cham (2017). Scholar
  29. 29.
    Trivedi, R., Dai, H., Wang, Y., Song, L.: Know-Evolve: deep temporal reasoning for dynamic knowledge graphs. In: ICML, pp. 3462–3471 (2017)Google Scholar
  30. 30.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)Google Scholar
  31. 31.
    Van Beek, P.: Reasoning about qualitative temporal information. Artif. Intell. 58(1–3), 297–326 (1992)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Federico Bianchi
    • 1
    Email author
  • Matteo Palmonari
    • 1
  • Debora Nozza
    • 1
  1. 1.University of Milano - BicoccaMilanItaly

Personalised recommendations