Abstract
Recommendation methods predicting potential items for user has evolved from linear factor models to non-linear factor deep learning models. Deep generative model, especially variational autoencoder(VAE), has been used in a wide range of recommendation systems such as MultVAE and RecVAE. Despite effectiveness, we argue that they suffer from two limitations: (1)sparse and noisy user-item interactions will affect the performance of VAE-based recommendation models; and (2)incorporating simple priors,e.g.,isotropic Gaussian, in VAE couldn’t extract personalized user preference,as user’s preference may be highly complex. In this paper, we propose a Nested Self-supervised Variational Autoencoder (NSVAE) model for recommendation to enhance generalization and accuracy of VAE-based recommendation models. Besides using VAE for predicting user interests, NSVAE supplements supervised task of recommendation with nested self-supervised task, which consider both partial and entire preference. Nested self-supervised task is composed of inside and outside pretext tasks. Inside pretext task aligns the representations learned from different views, where views contain user partial preference, and outside pretext task discriminates entire preference from other user. Recommendation task and pretext tasks can be seamlessly integrated and enhance each other. Extensive experiment results on three real-world benchmarks validate the superiority of our NSVAE model to state-of-the-art VAE-based recommendation models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ba LJ, Kiros JR, Hinton GE (2016) Layer normalization. CoRR arXiv:1607.06450
Blei DM, Kucukelbir A, Mcauliffe JD (2016) Variational inference: a review for statisticians. CoRR
Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp 4171–4186
Gao T, Cui Y, Ding F (2021) Seqvae: sequence variational autoencoder with policy gradient. Appl Intell 51(12):9030–9037
Gopalan P, Hofman JM, Blei DM (2015) Scalable recommendation with hierarchical poisson factorization. In: UAI, pp 326–335
Gupta K, Raghuprasad MY, Kumar P (2018) A hybrid variational autoencoder for collaborative filtering. CoRR
Harper FM, Konstan JA (2016) The movielens datasets: history and context. ACM Trans Interact Intell Syst:19:1–19:19
Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: ICDM
Kingma DP, Welling M (2014) Auto-encoding variational bayes. In: ICLR
Koren Y, Bell RM, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer
Lee W, Song K, Moon I (2017) Augmented variational autoencoders for collaborative filtering with auxiliary information. In: CIKM, pp 1139–1148
Liang D, Krishnan RG, Hoffman MD, Jebara T (2018) Variational autoencoders for collaborative filtering. In: WWW, pp 689–698
Liu H, Wen J, Jing L, Yu J (2019) Deep generative ranking for personalized recommendation. In: RecSys, pp 34–42
Liu H, Jing L, Wen J, Wu Z, Sun X, Wang J, Xiao L, Yu J (2020) Deep global and local generative model for recommendation. In: WWW, pp 551–561
Lobel S, Li C, Gao J, Carin L (2020) Ract: toward amortized ranking-critical training for collaborative filtering. In: ICLR
Ma J, Zhou C, Cui P, Yang H, Zhu W (2019) Learning disentangled representations for recommendation. In: NeurIPS, pp 5712–5723
Massa P, Avesani P (2007) Trust-aware recommender systems. In: RecSys, pp 17–24
Pang B, Yang M, Wang C (2019) A novel top-n recommendation approach based on conditional variational auto-encoder. In: PAKDD, pp 357–368
Rakesh V, Wang S, Shu K, Liu H (2019) Linked variational autoencoders for inferring substitutable and supplementary items. In: WSDM, pp 438–446
Ramachandran P, Zoph B, Le QV (2018) Searching for activation functions. In: 6th International conference on learning representations, ICLR 2018, Vancouver, BC, Canada, 30 Apr 3 May 2018, workshop track proceedings
Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: bayesian personalized ranking from implicit feedback. In: UAI, pp 452–461
Salakhutdinov R, Mnih A (2007) Probabilistic matrix factorization. In: Advances in neural information processing, pp 1257–1264
Shenbin I, Alekseev A, Tutubalina E, Malykh V, Nikolenko SI (2020) Recvae: a new variational autoencoder for top-n recommendations with implicit feedback. In: WSDM, pp 528– 536
Truong Q, Salah A, Lauw HW (2021) Bilateral variational autoencoder for collaborative filtering. In: WSDM, pp 292– 300
Wang F, Liu H (2020) Understanding the behaviour of contrastive loss. CoRR
Wu J, Wang X, Feng F, He X, Chen L, Lian J, Xie X (2021) Self-supervised graph learning for recommendation. In: SIGIR, pp 726–735
Wu Y, DuBois C, Zheng AX, Ester M (2016) Collaborative denoising auto-encoders for top-n recommender systems. In: WSDM, pp 153–162
Wu Y, Macdonald C, Ounis I (2020) A hybrid conditional variational autoencoder model for personalised top-n recommendation. In: ICTIR ’20, pp 89–96
Yao T, Yi X, Cheng DZ, Yu FX, Menon AK, Hong L, Chi EH, Tjoa S, Kang J, Ettinger E (2020) Self-supervised learning for deep models in recommendations. CoRR
Zhang X, Luo H, Chen B, Guo G (2020a) Multi-view visual bayesian personalized ranking for restaurant recommendation. Appl Intell 50(9):2901–2915
Zhang Y, Zhu Z, He Y, Caverlee J (2020b) Content-collaborative disentanglement representation learning for enhanced recommendation. In: Recsys 2020: fourteenth ACM conference on recommender systems, virtual event, Brazil, 22-26 Sept 2020, pp 43–52
Zhang Z, Liu H (2015) Social recommendation model combining trust propagation and sequential behaviors. Appl Intell 43(3):695–706
Zhao G, Qian X, Xie X (2016) User-service rating prediction by exploring social users’ rating behaviors. IEEE Trans Multimed:496–506
Zhu Y, Min MR, Kadav A, Graf HP (2020) S3VAE: self-supervised sequential VAE for representation disentanglement and data generation. In: CVPR, pp 6537–6546
Acknowledgements
This work was supported in part by the Innovation Found from the Engineering Research Center of Integration and Application of Digital Learning Technology, Ministry of Education, under Grant 1221035, and in part by the Open Research Fund from the Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen, under Grant B10120210117-OF01.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no known competing financial interest or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, J., Wu, J., Jia, C. et al. Self-supervised variational autoencoder towards recommendation by nested contrastive learning. Appl Intell 53, 18887–18897 (2023). https://doi.org/10.1007/s10489-023-04488-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04488-6