Skip to main content
SpringerLink
Log in

High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

The sequential recommendation task based on the multi-interest framework aims to model multiple interests of users from different aspects to predict their future interactions. However, researchers rarely consider the differences in features between the interests generated by the model. In extreme cases, all interest capsules have the same meaning, leading to the failure of modeling users with multiple interests. To address this issue, we propose the High-level Preferences as positive examples in Contrastive Learning for multi-interest Sequence Recommendation framework (HPCL4SR), which uses contrastive learning to distinguish differences in interests based on user item interaction information. In order to find high-quality comparative examples, this paper introduces the category information to construct a global graph, learning the association between categories for high-level preference interest of users. Then, a multi-layer perceptron is used to adaptively fuse the low-level preference interest features of the user’s items and the high-level preference interest features of the categories. Finally, user multi-interest contrastive samples are obtained through item sequence information and corresponding categories, which are fed into contrastive learning to optimize model parameters and generate multi-interest representations that are more in line with the user sequence. In addition, when modeling the user’s item sequence information, in order to increase the differentiation between item representations, the category of the item is used to supervise the learning process. Extensive experiments on three real datasets demonstrate that our method outperforms existing multi-interest recommendation models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

Notes

  1. https://nijianmo.github.io/amazon/

  2. https://tianchi.aliyun.com/dataset/dataDetail?dataId=42

  3. https://www.kaggle.com/chiranjivdas09/ta-feng-grocery-dataset

References

  1. Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q.Z., Orgun, M.A.: Sequential recommender systems: Challenges, progress and prospects. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence: IJCAI, pp. 6332–6338 (2019). https://doi.org/10.24963/ijcai.2019/883

  2. Quadrana, M., Cremonesi, P., Jannach, D.: Sequence-aware recommender systems. ACM Comput. Surv. 51(4), 1–36 (2018). https://doi.org/10.1145/3190616

    Article  Google Scholar 

  3. Pi, Q., Bian, W., Zhou, G., Zhu, X., Gai, K.: Practice on long sequential user behavior modeling for click-through rate prediction. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining: KDD, pp. 2671–2679 (2019). https://doi.org/10.1145/3292500.3330666

  4. Zhang, Y., Liu, Y., Xiong, H., Liu, Y., Yu, F., He, W., Xu, Y., Cui, L., Miao, C.: Cross-domain disentangled learning for e-commerce live streaming recommendation. In: 39th IEEE International conference on data engineering: ICDE, pp. 2955–2968 (2023). https://doi.org/10.1109/ICDE55515.2023.00226

  5. Chu, Q., Liu, G., Sun, H., Zhou, C.: Next news recommendation via knowledge-aware sequential model. In: Chinese computational linguistics - 18th China national conference: CCL. Lecture Notes in Computer Science, vol. 11856, pp. 221–232 (2019). https://doi.org/10.1007/978-3-030-32381-3_18

  6. Wu, C., Wu, F., Qi, T., Huang, Y.: User modeling with click preference and reading satisfaction for news recommendation. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence: IJCAI, pp. 3023–3029 (2020). https://doi.org/10.5555/3491440.3491858

  7. Chaves, P.D.V., Pereira, B.L., Santos, R.L.T.: Efficient online learning to rank for sequential music recommendation. In: WWW ’22: The ACM Web Conference 2022, pp. 2442–2450 (2022). https://doi.org/10.1145/3485447.3512116

  8. Li, Q., Wang, X., Wang, Z., Xu, G.: Be causal: De-biasing social network confounding in recommendation. ACM Trans. Knowl. Discov. Data 17(1), 1–23 (2023). https://doi.org/10.1145/3533725

    Article  Google Scholar 

  9. Song, W., Wang, S., Wang, Y., Wang, S.: Next-item recommendations in short sessions. In: RecSys ’21: fifteenth acm conference on recommender systems, pp. 282–291 (2021). https://doi.org/10.1145/3460231.3474238

  10. Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management: CIKM, pp. 843–852 (2018). https://doi.org/10.1145/3269206.3271761

  11. Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J.M., He, X.: A simple convolutional generative network for next item recommendation. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining: WSDM, pp. 582–590 (2019). https://doi.org/10.1145/3289600.3290975

  12. Kang, W., McAuley, J.J.: Self-attentive sequential recommendation. In: IEEE International Conference on Data Mining: ICDM, pp. 197–206 (2018). https://doi.org/10.1109/ICDM.2018.00035

  13. Wang, S., Cao, L., Hu, L., Berkovsky, S., Huang, X., Xiao, L., Lu, W.: Hierarchical attentive transaction embedding with intra- and inter-transaction dependencies for next-item recommendation. IEEE Intell. Syst. 36(4), 56–64 (2021). https://doi.org/10.1109/MIS.2020.2997362

    Article  Google Scholar 

  14. Wang, N., Wang, S., Wang, Y., Sheng, Q.Z., Orgun, M.A.: Modelling local and global dependencies for next-item recommendations. In: Web information systems engineering: WISE. Lecture Notes in Computer Science, vol. 12343, pp. 285–300 (2020). https://doi.org/10.1007/978-3-030-62008-0_20

  15. Zheng, X., Su, J., Liu, W., Chen, C.: DDGHM: dual dynamic graph with hybrid metric training for cross-domain sequential recommendation. In: MM ’22: The 30th ACM international conference on multimedia: MM, pp. 471–481 (2022). https://doi.org/10.1145/3503161.3548072

  16. Li, C., Liu, Z., Wu, M., Xu, Y., Zhao, H., Huang, P., Kang, G., Chen, Q., Li, W., Lee, D.L.: Multi-interest network with dynamic routing for recommendation at tmall. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2615–2623 (2019). https://doi.org/10.1145/3357384.3357814

  17. Xiao, Z., Yang, L., Jiang, W., Wei, Y., Hu, Y., Wang, H.: Deep multi-interest network for click-through rate prediction. In: CIKM ’20: The 29th ACM international conference on information and knowledge management, pp. 2265–2268 (2020). https://doi.org/10.1145/3340531.3412092

  18. Cen, Y., Zhang, J., Zou, X., Zhou, C., Yang, H., Tang, J.: Controllable multi-interest framework for recommendation. In: KDD ’20: The 26th ACM SIGKDD conference on knowledge discovery and data mining (KDD)), pp. 2942–2951 (2020). https://doi.org/10.1145/3394486.3403344

  19. Tang, Z., Wang, L., Zou, L., Zhang, X., Zhou, J., Li, C.: Towards multi-interest pre-training with sparse capsule network. In: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 311–320 (2023). https://doi.org/10.1145/3539618.3591778

  20. Xie, Y., Gao, J., Zhou, P., Ye, Q., Hua, Y., Kim, J.B., Wu, F., Kim, S.: Rethinking multi-interest learning for candidate matching in recommender systems. In: Proceedings of the 17th ACM conference on recommender systems: RecSys, pp. 283–293 (2023). https://doi.org/10.1145/3604915.3608766

  21. Lv, F., Jin, T., Yu, C., Sun, F., Lin, Q., Yang, K., Ng, W.: SDM: sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2635–2643 (2019). https://doi.org/10.1145/3357384.3357818

  22. Tan, Q., Zhang, J., Yao, J., Liu, N., Zhou, J., Yang, H., Hu, X.: Sparse-interest network for sequential recommendation. In: WSDM ’21, The Fourteenth ACM international conference on web search and data mining: WSDM, pp. 598–606 (2021). https://doi.org/10.1145/3437963.3441811

  23. Chen, G., Zhang, X., Zhao, Y., Xue, C., Xiang, J.: Exploring periodicity and interactivity in multi-interest framework for sequential recommendation. In: Proceedings of the thirtieth international joint conference on artificial intelligence: IJCAI, pp. 1426–1433 (2021). https://doi.org/10.24963/ijcai.2021/197

  24. Qiu, R., Huang, Z., Yin, H., Wang, Z.: Contrastive learning for representation degeneration problem in sequential recommendation. In: WSDM ’22: The fifteenth acm international conference on web search and data mining: WSDM, pp. 813–823 (2022). https://doi.org/10.1145/3488560.3498433

  25. Chai, Z., Chen, Z., Li, C., Xiao, R., Li, H., Wu, J., Chen, J., Tang, H.: User-aware multi-interest learning for candidate matching in recommenders. In: SIGIR ’22: The 45th international acm sigir conference on research and development in information retrieval: SIGIR, pp. 1326–1335 (2022). https://doi.org/10.1145/3477495.3532073

  26. Li, L., Zhang, Y., Liu, D., Chen, L.: Large language models for generative recommendation: A survey and visionary discussions. CoRR abs/2309.01157 (2023) https://doi.org/10.48550/ARXIV.2309.01157 https://arxiv.org/abs/2309.011572309.01157

  27. Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J.J., Zhao, W.X.: Large language models are zero-shot rankers for recommender systems. CoRR abs/2305.08845 (2023) https://doi.org/10.48550/ARXIV.2305.08845 https://arxiv.org/abs/2305.088452305.08845

  28. Luo, S., He, B., Zhao, H., Huang, Y., Zhou, A., Li, Z., Xiao, Y., Zhan, M., Song, L.: c: Instruction tuning large language model as ranker for top-k recommendation. arXiv preprint arXiv:2312.16018 (2023) https://doi.org/10.48550/arXiv.2312.16018

  29. Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., He, X.: Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In: Zhang, J., Chen, L., Berkovsky, S., Zhang, M., Noia, T.D., Basilico, J., Pizzato, L., Song, Y. (eds.) Proceedings of the 17th ACM conference on recommender systems, RecSys 2023, Singapore, Singapore, September 18-22, 2023, pp. 1007–1014. ACM, ??? (2023). https://doi.org/10.1145/3604915.3608857

  30. Hénaff, O.J.: Data-efficient image recognition with contrastive predictive coding. In: Proceedings of the 37th international conference on machine learning:icml. proceedings of machine learning research, vol. 119, pp. 4182–4192 (2020). https://proceedings.mlr.press/v119/henaff20a.html

  31. Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018) https://doi.org/10.48550/arXiv.1807.03748

  32. Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization. In: 7th International conference on learning representations: ICLR (2019). https://openreview.net/forum?id=Bklr3j0cKX

  33. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.B.: Momentum contrast for unsupervised visual representation learning. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition: CVPR, pp. 9726–9735 (2020). https://doi.org/10.1109/CVPR42600.2020.00975

  34. Chen, X., He, K.: Exploring simple siamese representation learning. In: IEEE Conference on computer vision and pattern recognition: CVPR, pp. 15750–15758 (2021). https://doi.org/10.1109/CVPR46437.2021.01549

  35. Wang, X., Huang, Y., Zeng, D., Qi, G.: Caco: Both positive and negative samples are directly learnable via cooperative-adversarial contrastive learning. IEEE Trans. Pattern Anal. Mach. Intell. 45(9), 10718–10730 (2023). https://doi.org/10.1109/TPAMI.2023.3262608

    Article  Google Scholar 

  36. Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., Xu, W.: Consert: A contrastive framework for self-supervised sentence representation transfer. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: ACL/IJCNLP, pp. 5065–5075 (2021). https://doi.org/10.18653/v1/2021.acl-long.393

  37. Gao, T., Yao, X., Chen, D.: Simcse: Simple contrastive learning of sentence embeddings. In: Proceedings of the 2021 conference on empirical methods in natural language processing: EMNLP, pp. 6894–6910 (2021). https://doi.org/10.18653/v1/2021.emnlp-main.552

  38. Zhang, Y., Zhu, H., Wang, Y., Xu, N., Li, X., Zhao, B.: A contrastive framework for learning sentence representations from pairwise and triple-wise perspective in angular space. In: Proceedings of the 60th annual meeting of the association for computational linguistics: ACL, pp. 4892–4903 (2022). https://doi.org/10.18653/v1/2022.acl-long.336

  39. Chen, Y., Liu, Z., Li, J., McAuley, J.J., Xiong, C.: Intent contrastive learning for sequential recommendation. In: WWW ’22: The ACM web conference 2022: WWW, pp. 2172–2182 (2022). https://doi.org/10.1145/3485447.3512090

  40. Du, H., Shi, H., Zhao, P., Wang, D., Sheng, V.S., Liu, Y., Liu, G., Zhao, L.: Contrastive learning with bidirectional transformers for sequential recommendation. In: Proceedings of the 31st ACM international conference on information & knowledge management: CIKM, pp. 396–405 (2022). https://doi.org/10.1145/3511808.3557266

  41. Yang, Y., Huang, C., Xia, L., Huang, C., Luo, D., Lin, K.: Debiased contrastive learning for sequential recommendation. In: Proceedings of the ACM Web Conference 2023: WWW, pp. 1063–1073 (2023). https://doi.org/10.1145/3543507.3583361

  42. Yuan, X., Duan, D., Tong, L., Shi, L., Zhang, C.: ICAI-SR: item categorical attribute integrated sequential recommendation. In: SIGIR ’21: The 44th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 1687–1691 (2021). https://doi.org/10.1145/3404835.3463060

  43. Xie, Y., Zhou, P., Kim, S.: Decoupled side information fusion for sequential recommendation. In: SIGIR ’22: The 45th international ACM SIGIR conference on research and development in information retrieval :SIGIR, pp. 1611–1621 (2022). https://doi.org/10.1145/3477495.3531963

  44. Wang, G., Ying, R., Huang, J., Leskovec, J.: Multi-hop attention graph neural networks. In: Proceedings of the thirtieth international joint conference on artificial intelligence: IJCAI, pp. 3089–3096 (2021). https://doi.org/10.24963/ijcai.2021/425

  45. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017: NIPS, pp. 3856–3866 (2017). https://proceedings.neurips.cc/paper_files/paper/2017/file/2cad8fa47bbef282badbb8de5374b894-Paper.pdf

  46. Deng, J., Guo, J., Yang, J., Xue, N., Kotsia, I., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 5962–5979 (2022). https://doi.org/10.1109/TPAMI.2021.3087709

    Article  Google Scholar 

  47. Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the asian federation of natural language processing: ACL, pp. 1–10 (2015). https://doi.org/10.3115/v1/P15-1001

  48. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: 4th International conference on learning representations: ICLR (2016). https://doi.org/10.48550/arXiv.1511.06939

  49. Fan, S., Wang, Y., Pang, X., Chen, L., Han, P., Shang, S.: Uamc: user-augmented conversation recommendation via multi-modal graph learning and context mining. World Wide Web, pp. 1–21 (2023). https://doi.org/10.1007/s11280-023-01219-2

  50. Xu, S., Hua, W., Zhang, Y.: Openp5: Benchmarking foundation models for recommendation. CoRR abs/2306.11134 (2023). https://doi.org/10.48550/ARXIV.2306.11134

Download references

Funding

National Natural Science Foundation of China (NSFC) (61972455)

Author information

Authors and Affiliations

  1. School of New Media and Communication, Tianjin University, Tianjin, China

    Zizhong Zhu

  2. College of Intelligence and Computing, Tianjin University, Tianjin, China

    Shuang Li, Yaokun Liu, Xiaowang Zhang, Zhiyong Feng & Yuexian Hou

Authors
  1. Zizhong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaokun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiyong Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuexian Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zizhong.Zhu. and Shuang.Li. wrote the main manuscript text, Yaokun.Liu. revised the figure, Xiaowang.Zhang. served as the corresponding author to coordinate and promote the completion of the paper, and Zhiyong.Feng. and Yuexian.Hou. provided reasonable revision opinions. All authors reviewed the manuscript.

Corresponding author

Correspondence to Xiaowang Zhang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Advancing recommendation systems with foundation models Guest Editors: Kai Zheng, Renhe Jiang, and Ryosuke Shibasaki

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Z., Li, S., Liu, Y. et al. High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation. World Wide Web 27, 21 (2024). https://doi.org/10.1007/s11280-024-01263-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11280-024-01263-6

Keywords

Search

Navigation