Skip to main content
SpringerLink
Log in

Multi-task network embedding

  • Regular Paper
  • Published:
International Journal of Data Science and Analytics Aims and scope Submit manuscript

Abstract

As there are various data mining applications involving network analysis, network embedding is frequently employed to learn latent representations or embeddings that encode the network structure. However, existing network embedding models are only designed for a single network scenario. It is common that nodes can have multiple types of relationships in big data era, which results in multiple networks, e.g., multiple social networks and multiple gene regulatory networks. Jointly embedding multiple networks thus may make network-specific embeddings more comprehensive and complete as the same node may expose similar or complementary characteristics in different networks. In this paper, we thus propose an idea of multi-task network embedding to jointly learn multiple network-specific embeddings for each node via enforcing an extra information-sharing embedding. We instantiate the idea in two types of models that are different in the mechanism for enforcing the information-sharing embedding. The first type enforces the information-sharing embedding as a common embedding shared by all tasks, which is similar to the concept of the common metric in multi-task metric learning, while the second type enforces the information-sharing embedding as a consensus embedding on which all network-specific embeddings agree. Moreover, we propose two mechanisms for embedding the network structure, which are first-order proximity preserving and second-order proximity preserving. We demonstrate through comprehensive experiments on three real-world datasets that the proposed models outperform recent network embedding models in applications including visualization, link prediction, and multi-label classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16(1), 1–3 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bakker, B., Heskes, T.: Task clustering and gating for bayesian multitask learning. J. Mach. Learn. Res. 4(May), 83–99 (2003)

    MATH  Google Scholar 

  3. Bakshy, E., Rosenn, I., Marlow, C., Adamic, L.: The role of social networks in information diffusion. In: Proceedings of the 21st International Conference on World Wide Web, pp. 519–528. ACM (2012)

  4. Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. NIPS 14, 585–591 (2001)

    Google Scholar 

  5. Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., et al.: Greedy layer-wise training of deep networks. Adv Neural Inf. Process. Syst. 19, 153 (2007)

    Google Scholar 

  6. Bezdek, JC., Hathaway, RJ.: Some notes on alternating optimization. In: AFSS International Conference on Fuzzy Systems, pp. 288–300. Springer (2002)

  7. Bhagat, S., Cormode, G., Muthukrishnan, S.: Node classification in social networks. In: Social Network Data Analytics, pp. 115–148. Springer (2011)

  8. Chen, B., Ding, Y., Wild, D.J.: Assessing drug target association using semantic linked data. PLoS Comput. Biol. 8(7), e1002–574 (2012)

    Article  Google Scholar 

  9. Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)

  10. Cox, T.F., Cox, M.A.: Multidimensional Scaling. CRC Press, Boca Raotn (2000)

    Book  MATH  Google Scholar 

  11. Dong, Y., Chawla, NV., Swami, A.: metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2017)

  12. Evgeniou, T., Pontil, M.: Regularized multi–task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)

  13. Grover, A., Leskovec, J.: node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)

  14. Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Assoc. Inf. Sci. Technol. 58(7), 1019–1031 (2007)

    Article  Google Scholar 

  15. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)

    MATH  Google Scholar 

  16. McAuley, J., Leskovec, J.: Image labeling on a network: using social-network metadata for image classification. In: European Conference on Computer Vision, pp. 828–841. Springer (2012)

  17. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. (2013). arXiv preprint arXiv:1301.3781

  18. Ni, J., Tong, H., Fan, W., Zhang, X.: Flexible and robust multi-network clustering. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 835–844. ACM (2015)

  19. Ou, M., Cui, P., Pei, J., Zhang, Z., Zhu, W.: Asymmetric transitivity preserving graph embedding. In: KDD, pp. 1105–1114 (2016)

  20. Parameswaran, S., Weinberger, KQ.: Large margin multi-task metric learning. In: Advances in Neural Information Processing Systems, pp. 1867–1875 (2010)

  21. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. ACM (2014)

  22. Philip, SY., Zhang, J.: Mcd: Mutual clustering across multiple social networks. In: 2015 IEEE International Congress on Big Data (BigData Congress), pp. 762–771. IEEE (2015)

  23. Read, J., Reutemann, P., Pfahringer, B., Holmes, G.: Meka: a multi-label/multi-target extension to weka. J. Mach. Learn. Res. 17(21), 1–5 (2016)

    MathSciNet  MATH  Google Scholar 

  24. Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., Su, Z.: Arnetminer: extraction and mining of academic social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2008)

  25. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: Large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee (2015)

  26. Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  27. Wang, X., Cui, P., Wang, J., Pei, J., Zhu, W., Yang, S.: Community preserving network embedding. In: AAAI, pp. 203–209 (2017)

  28. Wei, X., Xie, S., Yu, PS.: Efficient partial order preserving unsupervised feature selection on networks. In: Proceedings of the 2015 SIAM International Conference on Data Mining, Vancouver, BC, Canada, April 30–May 2, 2015, pp. 82–90 (2015)

  29. Wei, X., Cao, B., Shao, W., Lu, C., Yu, PS.: Community detection with partially observable links and node attributes. In: 2016 IEEE International Conference on Big Data, BigData 2016, Washington DC, USA, December 5–8, 2016, pp. 773–782 (2016)

  30. Wei, X., Xu, L., Cao, B., Yu, PS.: Cross view link prediction by learning noise-resilient representation consensus. In: Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3–7, 2017, pp. 1611–1619 (2017)

  31. Xu, L., Wei, X., Cao, J., Philip, SY.: Disentangled link prediction for signed social networks via disentangled representation learning. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 676–685. IEEE (2017a)

  32. Xu, L., Wei, X., Cao, J., Philip, SY.: Multi-task network embedding. In: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 571–580. IEEE (2017b)

  33. Xu, L., Wei, X., Cao, J., Yu, PS.: Embedding identity and interest for social networks. In: Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, April 3–7, 2017, pp. 859–860 (2017c)

  34. Xu, L., Wei, X., Cao, J., Yu, PS.: Embedding of embedding (eoe): Joint embedding for coupled heterogeneous networks. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 741–749. ACM (2017d)

  35. Xu, L., Wei, X., Cao, J., Yu, PS.: Multiple social role embedding. In: Proceedings of the International Conference on Data Science and Advanced Analytics. IEEE (2017e)

  36. Xu, L., Wei, X., Cao, J., Philip, SY.: Interaction content aware network embedding via co-embedding of nodes and edges. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 183–195. Springer (2018a)

  37. Xu, L., Wei, X., Cao, J., Yu, PS.: On exploring semantic meanings of links for embedding social networks. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 479–488. International World Wide Web Conferences Steering Committee (2018b)

  38. Yang, C., Liu, Z., Zhao, D., Sun, M., Chang, EY.: Network representation learning with rich text information. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, pp. 2111–2117 (2015)

  39. Zhang, J., Philip, SY.: Integrated anchor and social link predictions across social networks. In: IJCAI, pp. 2125–2132 (2015)

  40. Zhang, J., Yu, PS., Zhou, ZH.: Meta-path based multi-network collective link prediction. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1286–1295. ACM (2014)

Download references

Acknowledgements

The work described in this paper was partially supported by National Key R & D Program of China—2018 YFB1004801, RGC General Research Fund under Grant PolyU 152199/17E, the funding for Project of Strategic Importance provided by The Hong Kong Polytechnic University (Project Code: 1-ZE26), NSF through Grants IIS-1526499, IIS-1763325, CNS-1626432, and NSFC 61672313.

Author information

Authors and Affiliations

  1. Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Linchuan Xu & Jiannong Cao

  2. Facebook Inc., 1 Hacker Way, Menlo Park, CA, USA

    Xiaokai Wei

  3. Department of Computer Science, University of Illinois at Chicago, Chicago, IL, USA

    Philip S. Yu

Authors
  1. Linchuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaokai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiannong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip S. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linchuan Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extension version of the DSAA’2017 Research Track paper titled “Multi-task Network Embedding” [32].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Wei, X., Cao, J. et al. Multi-task network embedding. Int J Data Sci Anal 8, 183–198 (2019). https://doi.org/10.1007/s41060-018-0166-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41060-018-0166-2

Keywords

Search

Navigation