On Joint Representation Learning of Network Structure and Document Content

  • Jörg SchlöttererEmail author
  • Christin Seifert
  • Michael Granitzer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10410)


Inspired by the advancements of representation learning for natural language processing, learning continuous feature representations of nodes in networks has recently gained attention. Similar to word embeddings, node embeddings have been shown to capture certain semantics of the network structure. Combining both research directions into a joint representation learning of network structure and document content seems a promising direction to increase the quality of the learned representations. However, research is typically focused on either word or network embeddings and few approaches that learn a joint representation have been proposed. We present an overview of that field, starting at word representations, moving over document and network node representations to joint representations. We make the connections between the different models explicit and introduce a novel model for learning a joint representation. We present different methods for the novel model and compare the presented approaches in an evaluation. This paper explains how the different models recently proposed in the literature relate to each other and compares their performance.


Representation learning Network embeddings Document embeddings 


  1. 1.
    Firth, J.R.: A synopsis of linguistic theory 1930–55. In: Studies in Linguistic Analysis (Special Volume of the Philological Society), vol. 1952–1959, pp. 1–32. The Philological Society, Oxford (1957)Google Scholar
  2. 2.
    Ganguly, S., Pudi, V.: Paper2vec: combining graph and text information for scientific paper representation. In: Jose, J.M., Hauff, C., Altıngovde, I.S., Song, D., Albakour, D., Watt, S., Tait, J. (eds.) ECIR 2017. LNCS, vol. 10193, pp. 383–395. Springer, Cham (2017). doi: 10.1007/978-3-319-56608-5_30 CrossRefGoogle Scholar
  3. 3.
    Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, NY, USA, pp. 855–864. ACM, New York (2016)Google Scholar
  4. 4.
    Harris, Z.: Distributional structure. Word 10(23), 146–162 (1954)CrossRefGoogle Scholar
  5. 5.
    Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning, PMLR, Bejing, China, 22–24 June 2014, pp. 1188–1196 (2014)Google Scholar
  6. 6.
    Levy, O., Goldberg, Y.: Neural word embedding as implicit matrix factorization. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, NIPS 2014, pp. 2177–2185. MIT Press, Cambridge (2014)Google Scholar
  7. 7.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS 2013, pp. 3111–3119. Curran Associates Inc., USA (2013)Google Scholar
  8. 8.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  9. 9.
    Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2014, NY, USA, pp. 701–710. ACM, New York (2014)Google Scholar
  10. 10.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS 2012, pp. 2951–2959. Curran Associates Inc., USA (2012)Google Scholar
  11. 11.
    Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, WWW 2015, pp. 1067–1077. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2015)Google Scholar
  12. 12.
    Tang, L., Liu, H.: Leveraging social media networks for classification. Data Min. Knowl. Discov. 23(3), 447–478 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Yang, C., Liu, Z., Zhao, D., Sun, M., Chang, E.Y.: Network representation learning with rich text information. In: Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI 2015, pp. 2111–2117. AAAI Press (2015)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Jörg Schlötterer
    • 1
    Email author
  • Christin Seifert
    • 1
  • Michael Granitzer
    • 1
  1. 1.University of PassauPassauGermany

Personalised recommendations