Skip to main content
SpringerLink
Log in

Contrastive sequential interaction network learning on co-evolving Riemannian spaces

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The sequential interaction network usually find itself in a variety of applications, e.g., recommender system. Herein, inferring future interaction is of fundamental importance, and previous efforts are mainly focused on the dynamics in the classic zero-curvature Euclidean space. Despite the promising results achieved by previous methods, a range of significant issues still largely remains open: On the bipartite nature, is it appropriate to place user and item nodes in one identical space regardless of their inherent difference? On the network dynamics, instead of a fixed curvature space, will the representation spaces evolve when new interactions arrive continuously? On the learning paradigm, can we get rid of the label information costly to acquire? To address the aforementioned issues, we propose a novel Contrastive model for Sequential Interaction Network learning on Co-Evolving RiEmannian spaces, CSincere. To the best of our knowledge, we are the first to introduce a couple of co-evolving representation spaces, rather than a single or static space, and propose a co-contrastive learning for the sequential interaction network. In CSincere, we formulate a Cross-Space Aggregation for message-passing across representation spaces of different Riemannian geometries, and design a Neural Curvature Estimator based on Ricci curvatures for modeling the space evolvement over time. Thereafter, we present a Reweighed Co-Contrast between the temporal views of the sequential network, so that the couple of Riemannian spaces interact with each other for the interaction prediction without labels. Empirical results on 5 public datasets show the superiority of CSincere over the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets of MOOC, Wikipedia, Reddit, LastFM and Movielen are publicly available.

Notes

  1. In Riemannian geometry, the negative curvature space is termed as the hyperbolic space, and positive curvature space is termed as the spherical space.

  2. We use manifold and space interchangeable throughout this paper.

References

  1. He X, Gao M, Kan M-Y, Liu Y, Sugiyama K (2014) Predicting the popularity of web 2.0 items based on user comments. In: Proceedings of the 37th SIGIR, pp 233–242

  2. Peng H, Yang R, Wang Z, Li J, He L, Yu P, Zomaya A, Ranjan R (2021) Lime: low-cost incremental learning for dynamic heterogeneous information networks. IEEE Trans Comput 71:628–642

    Article  Google Scholar 

  3. Peng H, Zhang R, Dou Y, Yang R, Zhang J, Yu PS (2021) Reinforced neighborhood selection guided multi-relational graph neural networks. ACM Trans Inf Syst 40(69):1–46

    Google Scholar 

  4. Yang M, Zhou M, Kalander M, Huang Z, King I (2021) Discrete-time temporal network embedding via implicit hierarchical learning in hyperbolic space. In: Proceedings of KDD, pp 1975–1985

  5. Wang Y, Chang Y-Y, Liu Y, Leskovec J, Li P (2021) Inductive representation learning in temporal networks via causal anonymous walks. In: Proceedings of ICLR

  6. Peng H, Zhang R, Li S, Cao Y, Pan S, Yu PS (2023) Reinforced, incremental and cross-lingual event detection from social messages. IEEE Trans Pattern Anal Mach Intell (early access) 45(1):980–998

  7. Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv NeurIPS 32:4868–4879

  8. Mathieu E, Le Lan C, Maddison CJ, Tomioka R, Teh YW (2019) Continuous hierarchical representations with poincaré variational auto-encoders. In: Advances in NeurIPS, pp 12544–12555

  9. Gulcehre C, Denil M, Malinowski M, Razavi A, Pascanu R, Hermann KM, Battaglia P, Bapst V, Raposo D, Santoro A, Freitas N (2019) Hyperbolic attention networks. In: Proceedings of ICLR, pp 1–15

  10. Zhang Y, Wang X, Shi C, Liu N, Song G (2021) Lorentzian graph convolutional networks. In: Proceedings of WWW, pp 1249–1261

  11. Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. In: Advances in NeurIPS, pp 6338–6347

  12. Ganea O, Bécigneul G, Hofmann T (2018) Hyperbolic neural networks. In: Advances in NeurIPS, pp 5345–5355

  13. Defferrard M, Milani M, Gusset F, Perraudin N (2020) Deepsphere: a graph-based spherical cnn. In: Proceedings of ICLR

  14. Rezende DJ, Papamakarios G, Racaniere S, Albergo M, Kanwar G, Shanahan P, Cranmer K (2020) Normalizing flows on tori and spheres. In: Proceedings of ICML, pp 8083–8092

  15. Fan Z, Liu Z, Zhang J, Xiong Y, Zheng L, Yu PS (2021) Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th CIKM, pp 433–442

  16. Cao J, Lin X, Guo S, Liu L, Liu T, Wang B (2021) Bipartite graph embedding via mutual information maximization. In: Proceedings of the 14th CIKM, pp 635–643

  17. Dai H, Wang Y, Trivedi R, Song L (2016) Deep coevolutionary network: embedding user and item features for recommendation. arXiv:1609.03675

  18. Nguyen GH, Lee JB, Rossi RA, Ahmed NK, Koh E, Kim S (2018) Continuous-time dynamic network embeddings. In: Companion proceedings of the Web conference 2018, pp 969–976

  19. Kefato Z, Girdzijauskas S, Sheikh N, Montresor A (2021) Dynamic embeddings for interaction prediction. In: Proceedings of the Web conference 2021, pp 1609–1618

  20. Beutel A, Covington P, Jain S, Xu C, Li J, Gatto V, Chi EH (2018) Latent cross: making use of context in recurrent recommender systems. In: Proceedings of the 11th WSDM, pp 46–54

  21. Sreejith R, Mohanraj K, Jost J, Saucan E, Samal A (2016) Forman curvature for complex networks. J Stat Mech Theory Exp 2016(6):063206

    Article  MathSciNet  Google Scholar 

  22. Bachmann G, Becigneul G, Ganea O (2020) Constant curvature graph convolutional networks. In: Proceedings of the 37th ICML, vol 119, pp 486–496

  23. Gromov M (1987) In: Gersten SM (ed) Hyperbolic groups. Springer, New York, pp 75–263

  24. Vinh Tran L, Tay Y, Zhang S, Cong G, Li X (2020) HyperML: a boosting metric learning approach in hyperbolic space for recommender systems, pp 609–617

  25. Kumar S, Zhang X, Leskovec J (2019) Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of the 25th KDD, pp 1269–1278

  26. Chen H, Xiong Y, Zhu Y, Yu PS (2021) Highly liquid temporal interaction graph embeddings. In: Proceedings of the Web conference 2021, pp 1639–1648

  27. Zhu Y, Li H, Liao Y, Wang B, Guan Z, Liu H, Cai D (2017) What to do next: modeling user behaviors by time-lstm. In: Proceedings of the 26th IJCAI, vol 17, pp 3602–3608

  28. Baytas IM, Xiao C, Zhang X, Wang F, Jain AK, Zhou J (2017) Patient subtyping via time-aware lstm networks. In: Proceedings of the 23rd KDD, pp 65–74

  29. Ye J, Zhang Z, Sun L, Yan Y, Wang F, Ren F (2023) Sincere: sequential interaction networks representation learning on co-evolving Riemannian manifolds. In: Proceedings of ACM The Web conference (WWW), pp 1–10

  30. Ungar AA (2008) A gyrovector space approach to hyperbolic geometry. Synth Lect Math Stat 1(1):1–194

    MathSciNet  Google Scholar 

  31. Ye Z, Liu KS, Ma T, Gao J, Chen C (2020) Curvature graph network. In: Proceedings of ICLR

  32. Lee JM (2018) Introduction to Riemannian manifolds, Springer Cham, Series ISSN: 0072-5285. https://doi.org/10.1007/978-3-319-91755-9

  33. Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: Proceedings of CVPR, pp 154–163

  34. Chen W, Han X, Lin Y, Zhao H, Liu Z, Li P, Sun M, Zhou J (2022) Fully hyperbolic neural networks. In: Proceedings of the 60th ACL, pp 5672–5686

  35. Ollivier Y (2009) Ricci curvature of Markov chains on metric spaces. J Funct Anal 256(3):810–864

    Article  MathSciNet  Google Scholar 

  36. Forman R (2003) Bochner’s method for cell complexes and combinatorial Ricci curvature. Discrete Comput Geom 29(3):323–374

    Article  MathSciNet  Google Scholar 

  37. Ungar AA (2010) Barycentric calculus in Euclidean and hyperbolic geometry: A comparative introduction. World Scientific. https://doi.org/10.1142/7740

  38. Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K (2020) Inductive representation learning on temporal graphs. In: Proceedings of ICLR

  39. Sia J, Jonckheere E, Bogdan P (2019) Ollivier–Ricci curvature-based method to community detection in complex networks. Sci Rep 9(1):1–12

    Article  ADS  CAS  Google Scholar 

  40. Gu A, Sala F, Gunel B, Re C (2019) Learning mixed-curvature representations in product spaces. In: Proceedings of ICLR

  41. Fu X, Li J, Wu J, Sun Q, Ji C, Wang S, Tan J, Peng H, Yu PS (2021) Ace-hgnn: adaptive curvature exploration hyperbolic graph neural network. In: Proceedings of ICDM, pp 111–120

  42. Yang H, Chen H, Pan S, Li L, Yu PS, Xu G (2022) Dual space graph contrastive learning. In: Proceedings of The ACM Web conference, pp 1238–1247

  43. Hassani K, Ahmadi AHK (2020) Contrastive multi-view representation learning on graphs. In: Proceedings of ICML, vol 119, pp 4116–4126

  44. Qiu J, Chen Q, Dong Y, Zhang J, Yang H, Ding M, Wang K, Tang J (2020) GCC: graph contrastive coding for graph neural network pre-training. In: Proceedings of KDD, pp 1150–1160

  45. Sun L, Ye J, Peng H, Yu PS (2022) A self-supervised Riemannian GNN with time varying curvature for temporal graph learning. In: Proceedings of the 31st CIKM, pp 1827–1836

  46. Tian S, Wu R, Shi L, Zhu L, Xiong T (2021) Self-supervised representation learning on dynamic graphs. In: Proceedings of the 30th CIKM, pp 1814–1823

  47. Oord AVD, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding, pp 1–13. arXiv: 1807.03748

  48. Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: Proceedings of ICLR, pp 1–24

  49. Robinson JD, Chuang C, Sra S, Jegelka S (2021) Contrastive learning with hard negative samples. In: Proceedings of the 9th ICLR

  50. Xia J, Wu L, Wang G, Chen J, Li SZ (2022) Progcl: rethinking hard negative mining in graph contrastive learning. In: Proceedings of ICML, vol 162, pp 24332–24346

  51. Ni C, Lin Y, Luo F, Gao (2019)Community detection on networks with Ricci flow. Nat Sci Rep 9(9984):1–12

  52. Ye Z, Liu KS, Ma T, Gao J, C (2020) Curvature graph network. In: Proceedings of the 8th ICLR

  53. Wu C-Y, Ahmed A, Beutel A, Smola AJ, Jing H (2017) Recurrent recommender networks. In: Proceedings of the 10th WSDM, pp 495–503

  54. Sun J, Cheng Z, Zuberi S, Perez F, Volkovs M (2021) Hgcf: Hyperbolic graph convolution networks for collaborative filtering. In: Proceedings of the Web conference 2021, pp 593–601

  55. Shimizu R, Mukuta Y, Harada T (2021) Hyperbolic neural networks++. In: Proceedings of ICLR, pp 1–25

  56. Lee JM (2013) Introduction to smooth manifolds, 2nd edn. Springer New York, NY. https://doi.org/10.1007/978-1-4419-9982-5

    Google Scholar 

  57. Wang Y, Cai Y, Liang Y, Ding H, Wang C, Bhatia S, Hooi B (2021) Adaptive data augmentation on temporal graphs. In: Advances in NeurIPS, vol 34, pp 1440–1452

  58. Zuo Y, Liu G, Lin H, Guo J, Hu X, Wu J (2018) Embedding temporal network via neighborhood formation. In: Proceedings of KDD, pp 2857–2866

  59. Gupta S, Manchanda S, Bedathur S, Ranu S (2022) Tigger: scalable generative modelling for temporal interaction graphs. In: Proceedings of AAAI, vol 36, pp 6819–6828

  60. Xia W, Li Y, Li S (2023) Graph neural point process for temporal interaction prediction. IEEE Trans Knowl Data Eng 35(5):4867–4879

    Google Scholar 

  61. Zhang Y, Xiong Y, Liao Y, Sun Y, Jin Y, Zheng X, Zhu Y (2023) TIGER: temporal interaction graph embedding with restarts. In: Proceedings of the ACM Web conference 2023 (WWW), pp 478–488

  62. Kazemi SM, Goel R, Jain K, Kobyzev I, Sethi A, Forsyth P, Poupart P (2020) Representation learning for dynamic graphs: a survey. J Mach Learn Res 21(70):1–73

    MathSciNet  Google Scholar 

  63. Aggarwal C, Subbian K (2014) Evolutionary network analysis: a survey. ACM Comput Surv: CSUR 47(1):1–36

    Article  Google Scholar 

  64. Suzuki R, Takahama R, Onoda S (2019) Hyperbolic disk embeddings for directed acyclic graphs. In: Proceedings of ICML, pp 6066–6075

  65. Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. In: Advances in NeurIPS, pp 4869–4880

  66. Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. In: Advances in NeurIPS, pp 8228–8239

  67. Bachmann G, Bécigneul G, Ganea O (2020) Constant curvature graph convolutional networks. In: Proceedings of ICML, vol 119, pp 486–496

  68. Xiong B, Zhu S, Nayyeri M, Xu C, Pan S, Zhou C, Staab S (2022)Ultrahyperbolic knowledge graph embeddings. In: Proceedings of KDD, pp 2130–2139

  69. Xiong, B., Zhu, S., Potyka, N., Pan, S., Zhu, C., Staab, S.: Pseudo-riemannian graph convolutional networks. In: Advances in 36th NeurIPS, pp. 1–21 (2022)

  70. Law M (2021) Ultrahyperbolic neural networks. In: Advances in NeurIPS, vol 34, pp 22058–22069

  71. Gu A, Sala F, Gunel B, Ré C (2019) Learning mixed-curvature representations in product spaces. In: Proceedings of ICLR, pp 1–21

  72. Wang S, Wei X, Santos CN, Wang Z, Nallapati R, Arnold AO, Xiang B, Yu PS, Cruz IF (2021) Mixed-curvature multi-relational graph neural network for knowledge graph completion. In: Proceedings of The ACM Web conference, pp 1761–1771

  73. Skopek O, Ganea O-E, Becigneul G (2020) Mixed-curvature variational autoencoders. In: Proceedings of ICLR

  74. Sun L, Zhang Z, Ye J, Peng H, Zhang J, Su S, Yu PS (2022) A self-supervised mixed-curvature graph neural network. In: Proceedings of AAAI, vol 36, pp 4146–4155

  75. Cruceru C, Bécigneul G, Ganea O (2021) Computationally tractable Riemannian manifolds for graph embeddings. In: Proceedings of AAAI, pp 7133–7141

  76. Zhu S, Pan S, Zhou C, Wu J, Cao Y, Wang B (2020) Graph geometry interaction learning. In: Advances in NeurIPS, vol 33, pp 7548–7558

  77. Sun L, Zhang Z, Zhang J, Wang F, Peng H, Su S, Yu PS (2021) Hyperbolic variational graph neural network for modeling dynamic graphs. In: Proceedings of the 35th AAAI, pp 4375–4383

  78. Sun L, Ye J, Peng H, Wang F, Yu PS (2023) Self-supervised continual graph learning in adaptive riemannian spaces. In: Proceedings of the 37th AAAI, pp 4633–4642

  79. Sun L, Ye J, Peng H, Yu PS (2022) A self-supervised Riemannian GNN with time varying curvature for temporal graph learning. In: Proceedings of the 31st CIKM, pp 1827–1836

  80. Sun L, Wang F, Ye J, Peng H, Yu PS (2023) CONGREGATE: contrastive graph clustering in curvature spaces. In: Proceedings of the 32nd IJCAI, pp 2296–2305

Download references

Acknowledgements

The authors of this paper were supported in part by National Natural Science Foundation of China under Grant 62202164, S &T Program of Hebei through Grant 20310101D, and the Fundamental Research Funds for the Central Universities 2022MS018. Prof. Philip S. Yu is supported in part by NSF under Grant III-2106758.

Author information

Authors and Affiliations

  1. North China Electric Power University, Beijing, 102206, China

    Li Sun

  2. Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Junda Ye & Feiyang Wang

  3. University of California, Davis, Davis, CA, 95616, USA

    Jiawei Zhang

  4. State Grid Handan Electric Power Supply Company, Handan, 056000, Hebei, China

    Yong Yang

  5. Shijiazhuang Institute of Railway Technology, Shijiazhuang, 050041, Hebei, China

    Mingsheng Liu

  6. University of Illinois at Chicago, Chicago, IL, 60607, USA

    Philip S. Yu

Authors
  1. Li Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junda Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiawei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingsheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Feiyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Philip S. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Li Sun or Mingsheng Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Ye, J., Zhang, J. et al. Contrastive sequential interaction network learning on co-evolving Riemannian spaces. Int. J. Mach. Learn. & Cyber. 15, 1397–1413 (2024). https://doi.org/10.1007/s13042-023-01974-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01974-8

Keywords

Search

Navigation