Skip to main content
SpringerLink
Log in

Learning Representations for Bipartite Graphs Using Multi-task Self-supervised Learning

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14171))

  • 919 Accesses

Abstract

Representation learning for bipartite graphs is a challenging problem due to its unique structure and characteristics. The primary challenge is the lack of extensive supervised data and the bipartite graph structure, where two distinct types of nodes exist with no direct connections between the nodes of the same kind. Hence, recent algorithms utilize Self Supervised Learning (SSL) to learn effective node embeddings without needing costly labeled data. However, conventional SSL methods learn through a single pretext task, making the trained model specific to the downstream task. This paper proposes a novel approach for learning generalized representations of bipartite graphs using multi-task SSL. The proposed method utilizes multiple self-supervised tasks to learn improved embeddings that capture different aspects of the bipartite graphs, such as graph structure, node features, and local-global information. We utilize deep multi-task learning (MTL) to further assist in learning generalizable self-supervised solution. To mitigate negative transfer when related and unrelated tasks are trained in MTL, we propose a novel DST++ algorithm. The proposed DST++ optimization algorithm improves existing DST by considering task affinities and groupings for better initialization and training. The end-to-end proposed method with complimentary SSL tasks and DST++ multi-task optimization is evaluated on three tasks: node classification, link prediction, and node regression, using four publicly available benchmark datasets. The results demonstrate that our proposed method outperforms state-of-the-art methods for representation learning in bipartite graphs. Specifically, our method achieves up to 12% improvement in accuracy for node classification and up to 9% improvement in AUC for link prediction tasks compared to the baseline methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Berg, R.V.D., Kipf, T.N., Welling, M.: Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017)

  2. Bommasani, R., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

  3. Cai, H., Zheng, V.W., Chang, K.C.C.: A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE Trans. Knowl. Data Eng. 30(9), 1616–1637 (2018)

    Article  Google Scholar 

  4. Cao, J., Lin, X., Guo, S., Liu, L., Liu, T., Wang, B.: Bipartite graph embedding via mutual information maximization. In: ACM International Conference on Web Search and Data Mining, pp. 635–643 (2021)

    Google Scholar 

  5. Chen, Z., Badrinarayanan, V., Lee, C.Y., Rabinovich, A.: Gradnorm: gradient normalization for adaptive loss balancing in deep multitask networks. In: International Conference on Machine Learning, pp. 794–803 (2018)

    Google Scholar 

  6. Doersch, C., Zisserman, A.: Multi-task self-supervised visual learning. In: IEEE International Conference on Computer Vision, pp. 2051–2060 (2017)

    Google Scholar 

  7. Dong, Y., Chawla, N.V., Swami, A.: metapath2vec: scalable representation learning for heterogeneous networks. In: ACM International Conference on Knowledge Discovery and Data Mining, pp. 135–144 (2017)

    Google Scholar 

  8. Fifty, C., Amid, E., Zhao, Z., Yu, T., Anil, R., Finn, C.: Efficiently identifying task groupings for multi-task learning. Adv. Neural. Inf. Process. Syst. 34, 27503–27516 (2021)

    Google Scholar 

  9. Gao, M., Chen, L., He, X., Zhou, A.: Bine: bipartite network embedding. In: ACM Conference on Research & Development in Information Retrieval, pp. 715–724 (2018)

    Google Scholar 

  10. Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: ACM International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)

    Google Scholar 

  11. Guo, M., Haque, A., Huang, D.A., Yeung, S., Fei-Fei, L.: Dynamic task prioritization for multitask learning. In: European Conference on Computer Vision, pp. 270–287 (2018)

    Google Scholar 

  12. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  13. Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. arXiv preprint arXiv:1709.05584 (2017)

  14. Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 1–19 (2015)

    Article  Google Scholar 

  15. He, C., et al.: Cascade-BGNN: toward efficient self-supervised representation learning on large-scale bipartite graphs. arXiv preprint arXiv:1906.11994 (2019)

  16. Hjelm, R.D., et al.: Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018)

  17. Hou, Z., Liu, X., Dong, Y., Wang, C., Tang, J., et al.: Graphmae: self-supervised masked graph autoencoders. arXiv preprint arXiv:2205.10803 (2022)

  18. Hu, W., et al.: Strategies for pre-training graph neural networks. arXiv preprint arXiv:1905.12265 (2019)

  19. Jean, S., Firat, O., Johnson, M.: Adaptive scheduling for multi-task learning. arXiv preprint arXiv:1909.06434 (2019)

  20. Jin, W., Liu, X., Zhao, X., Ma, Y., Shah, N., Tang, J.: Automated self-supervised learning for graphs. arXiv preprint arXiv:2106.05470 (2021)

  21. Jing, B., Yan, Y., Zhu, Y., Tong, H.: Coin: co-cluster infomax for bipartite graphs. arXiv preprint arXiv:2206.00006 (2022)

  22. Ju, M., et al.: Multi-task self-supervised graph neural networks enable stronger task generalization. arXiv preprint arXiv:2210.02016 (2022)

  23. Liebel, L., Körner, M.: Auxiliary tasks in multi-task learning. arXiv preprint arXiv:1805.06334 (2018)

  24. Liu, S., Liang, Y., Gitter, A.: Loss-balanced task weighting to reduce negative transfer in multi-task learning. In: AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 9977–9978 (2019)

    Google Scholar 

  25. Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., Feris, R.: Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5334–5343 (2017)

    Google Scholar 

  26. Malhotra, A., et al.: Multi-task driven explainable diagnosis of COVID-19 using chest X-ray images. Pattern Recogn. 122, 108243 (2022)

    Article  Google Scholar 

  27. Malhotra, A., Vatsa, M., Singh, R.: Dropped scheduled task: mitigating negative transfer in multi-task learning using dynamic task dropping. Trans. Mach. Learn. Res. (2022)

    Google Scholar 

  28. Mavromatis, C., Karypis, G.: Graph infoclust: maximizing coarse-grain mutual information in graphs. In: Advances in Knowledge Discovery and Data Mining, pp. 541–553 (2021)

    Google Scholar 

  29. Ni, M., et al.: M3P: learning universal representations via multitask multilingual multimodal pre-training. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3977–3986 (2021)

    Google Scholar 

  30. Park, C., Kim, D., Han, J., Yu, H.: Unsupervised attributed multiplex network embedding. In: AAAI Conference on Artificial Intelligence, vol. 34, no. 04, pp. 5371–5378 (2020)

    Google Scholar 

  31. Peng, Z., et al.: Graph representation learning via graphical mutual information maximization. In: The Web Conference, pp. 259–270 (2020)

    Google Scholar 

  32. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: ACM International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)

    Google Scholar 

  33. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

    Google Scholar 

  34. Sanh, V., et al.: Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 (2021)

  35. Stratigi, M., Li, X., Stefanidis, K., Zhang, Z.: Ratings vs. reviews in recommender systems: a case study on the amazon movies dataset. In: New Trends in Databases and Information Systems, pp. 68–76 (2019)

    Google Scholar 

  36. Sun, J., Cheng, Z., Zuberi, S., Pérez, F., Volkovs, M.: HGCF: hyperbolic graph convolution networks for collaborative filtering. In: The Web Conference, pp. 593–601 (2021)

    Google Scholar 

  37. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: International Conference on World Wide Web, pp. 1067–1077 (2015)

    Google Scholar 

  38. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  39. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  40. Wan, H., Zhang, Y., Zhang, J., Tang, J.: Aminer: search and mining of academic social networks. Data Intell. 1(1), 58–76 (2019)

    Article  Google Scholar 

  41. Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: ACM International Conference on Research and Development in Information Retrieval, pp. 165–174 (2019)

    Google Scholar 

  42. Yamanishi, Y., Kotera, M., Kanehisa, M., Goto, S.: Drug-target interaction prediction from chemical, genomic and pharmacological data in an integrated framework. Bioinformatics 26(12), i246–i254 (2010)

    Article  Google Scholar 

  43. Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: ACM International Conference on Knowledge Discovery & Data Mining, pp. 974–983 (2018)

    Google Scholar 

  44. You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst. 33, 5812–5823 (2020)

    Google Scholar 

  45. Zamir, A.R., et al.: Robust learning through cross-task consistency. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11197–11206 (2020)

    Google Scholar 

  46. Zhang, M., Chen, Y.: Inductive matrix completion based on graph neural networks. arXiv preprint arXiv:1904.12058 (2019)

  47. Zhang, Y., Xiong, Y., Kong, X., Zhu, Y.: Learning node embeddings in interaction graphs. In: ACM Conference on Information and Knowledge Management, pp. 397–406 (2017)

    Google Scholar 

  48. Zhang, Y., Yeung, D.Y.: A regularization approach to learning task relationships in multitask learning. ACM Trans. Knowl. Discov. Data (TKDD) 8(3), 1–31 (2014)

    Article  Google Scholar 

  49. Zhang, Y., Wang, D., Zhang, Y.: Neural IR meets graph embedding: a ranking model for product search. In: The World Wide Web Conference, pp. 2390–2400 (2019)

    Google Scholar 

  50. Zhang, Z., Cui, P., Wang, X., Pei, J., Yao, X., Zhu, W.: Arbitrary-order proximity preserved network embedding. In: ACM International Conference on Knowledge Discovery & Data Mining, pp. 2778–2786 (2018)

    Google Scholar 

  51. Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020)

  52. Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: The Web Conference, pp. 2069–2080 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. AI Garage, Mastercard, Gurugram, India

    Akshay Sethi, Sonia Gupta, Aakarsh Malhotra & Siddhartha Asthana

Authors
  1. Akshay Sethi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sonia Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aakarsh Malhotra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Siddhartha Asthana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akshay Sethi .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethics Statement

Our proposed algorithm does not raise any ethical concerns, however, it is important to note that ethical applications of graphs can potentially benefit from the improved task generalization and performance provided by our work. To ensure positive and socially beneficial outcomes of machine learning algorithms, it is crucial to exercise caution and responsibility.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sethi, A., Gupta, S., Malhotra, A., Asthana, S. (2023). Learning Representations for Bipartite Graphs Using Multi-task Self-supervised Learning. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14171. Springer, Cham. https://doi.org/10.1007/978-3-031-43418-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43418-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43417-4

  • Online ISBN: 978-3-031-43418-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation