Abstract
Recent years have witnessed the great success of self-supervised learning (SSL) in recommendation systems. However, SSL recommender models are likely to suffer from spurious correlations, leading to poor generalization. To mitigate spurious correlations, existing work usually pursues ID-based SSL recommendation or utilizes feature engineering to identify spurious features. Nevertheless, ID-based SSL approaches sacrifice the positive impact of invariant features, while feature engineering methods require high-cost human labeling. To address the problems, we aim to automatically mitigate the effect of spurious correlations. This objective requires to 1) automatically mask spurious features without supervision, and 2) block the negative effect transmission from spurious features to other features during SSL. To handle the two challenges, we propose an invariant feature learning framework, which first divides user-item interactions into multiple environments with distribution shifts and then learns a feature mask mechanism to capture invariant features across environments. Based on the mask mechanism, we can remove the spurious features for robust predictions and block the negative effect transmission via mask-guided feature augmentation. Extensive experiments on two datasets demonstrate the effectiveness of the proposed framework in mitigating spurious correlations and improving the generalization abilities of SSL models.
References
J. C. Wu, X. Wang, F. L. Feng, X. N. He, L. Chen, J. X. Lian, X. Xie. Self-supervised graph learning for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Canada, pp. 726–735, 2021. DOI: https://doi.org/10.1145/3404835.3462862.
T. S. Yao, X. Y. Yi, D. Z. Cheng, F. Yu, T. Chen, A. Menon, L. C. Hong, E. H. Chi, S. Tjoa, J. Kang, E. Ettinger. Self-supervised learning for large-scale item recommendations. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, ACM, Australia, pp.4321–4330, 2021. DOI: https://doi.org/10.1145/3459637.3481952.
K. Zhou, H. Wang, W. X. Zhao, Y. T. Zhu, S. R. Wang, F. Z. Zhang, Z. Y. Wang, J. R. Wen. S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Ireland, pp. 1893–1902, 2020. DOI: https://doi.org/10.1145/3340531.3411954.
Y. W. Wei, X. Wang, Q. Li, L. Q. Nie, Y. Li, X. P. Li, T. S. Chua. Contrastive learning for cold-start recommendation. In Proceedings of the 29th ACM International Conference on Multimedia, China, pp. 5382–5390, 2021. DOI: https://doi.org/10.1145/3474085.3475665.
T. Y. Qian, Y. L. Liang, Q. Li, X. Ma, K. Sun, Z. Y. Peng. Intent disentanglement and feature self-supervision for novel recommendation. IEEE Transactions on Knowledge and Data Engineering, to be published. DOI: https://doi.org/10.1109/TKDE.2022.3175536.
X. Xia, H. Z. Yin, J. L. Yu, Q. Y. Wang, L. Z. Cui, X. L. Zhang. Self-supervised hypergraph convolutional networks for session-based recommendation. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, Palo Alto, USA, pp.4503–4511, 2021. DOI: https://doi.org/10.1609/aaai.v35i5.16578.
J. Pearl. Causality, New York, USA: Cambridge University Press, 2009.
B. Nushi, E. Kamar, E. Horvitz. Towards accountable AI: Hybrid human-machine analyses for characterizing system failure. In Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing, Zürich, Switzerland, pp. 126–135, 2018.
Y. Chung, T. Kraska, N. Polyzotis, K. H. Tae, S. E. Whang. Slice finder: Automated data slicing for model validation. In Proceedings of the 35th International Conference on Data Engineering, IEEE, Macao, China, pp. 1550–1553, 2019. DOI: https://doi.org/10.1109/ICDE.2019.00139.
W. Y. Cheng, Y. Y. Shen, L. P. Huang. Adaptive factorization network: Learning adaptive-order feature interactions. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, USA, pp. 3609–3616, 2020. DOI: https://doi.org/10.1609/aaai.v34i04.5768.
B. Liu, C. X. Zhu, G. L. Li, W. N. Zhang, J. C. Lai, R. M. Tang, X. Q. He, Z. G. Li, Y. Yu. AutoFIS: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, USA., pp.2636–2645, 2020. DOI: https://doi.org/10.1145/3394486.3403314.
A. J. Baruah, S. Baruah. Data augmentation and deep neuro-fuzzy network for student performance prediction with MapReduce framework. International Journal of Automation and Computing, vol. 18, no. 6, pp. 981–992, 2021. DOI: https://doi.org/10.1007/s11633-021-1312-1.
B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio. Toward causal representation learning. Proceedings of IEEE, vol. 109, no. 5, pp. 612–634, 2021. DOI: https://doi.org/10.1109/JPROC.2021.3058954.
S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada, pp. 452–461, 2009.
Y. Yamada, O. Lindenbaum, S. Negahban, Y. Kluger. Feature selection using stochastic gates. In Proceedings of the 37th International Conference on Machine Learning, pp. 10648–10659, 2020.
J. S. Liu, Z. Y. Hu, P. Cui, B. Li, Z. Y. Shen. Heterogeneous risk minimization. In Proceedings of the 38th International Conference on Machine Learning, pp. 6804–6814, 2021.
M. Koyama, S. Yamaguchi. When is invariance useful in an Out-of-Distribution Generalization problem? [Online], Available: https://arxiv.org/abs/2008.01883, 2021.
M. Arjovsky, L. Bottou, I. Gulrajani, D. Lopez-Paz. Invariant risk minimization, [Online], Available: https://arxiv.org/abs/1907.02893, 2020.
X. Wang, X. N. He, M. Wang, F. L. Feng, T. S. Chua. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, pp. 165–174, 2019. DOI: https://doi.org/10.1145/3331184.3331267.
S. Rendle. Factorization machines. In Proceedings of IEEE International Conference on Data Mining, Sydney, Australia, pp. 995–1000, 2010. DOI: https://doi.org/10.1109/ICDM.2010.127.
X. N. He, T. S. Chua. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Japan, pp. 355–364, 2017. DOI: https://doi.org/10.1145/3077136.3080777.
H. F. Guo, R. M. Tang, Y. M. Ye, Z. G. Li, X. Q. He. DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia, pp. 1725–1731, 2017.
F. Liu, Z. Y. Cheng, L. Zhu, Z. Gao, L. Q. Nie. Interest-aware message-passing GCN for recommendation. In Proceedings of the Web Conference, ACM, Ljubljana, Slovenia, pp. 1296–1305, 2021. DOI: https://doi.org/10.1145/3442381.3449986.
L. Wu, X. N. He, X. Wang, K. Zhang, M. Wang. A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering, to be published. DOI: https://doi.org/10.1109/TKDE.2022.3145690.
A. Alqwadri, M. Azzeh, F. Almasalha. Application of machine learning for online reputation systems. International Journal of Automation and Computing, vol. 18, no. 3, pp. 492–502, 2021. DOI: https://doi.org/10.1007/s11633-020-1275-7.
F. Liu, Z. Y. Cheng, H. L. Chen, Y. W. Wei, L. Q. Nie, M. Kankanhalli. Privacy-preserving synthetic data generation for recommendation systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, pp. 1379–1389, 2022. DOI: https://doi.org/10.1145/3477495.3532044.
K. Zhou, H. Yu, W. X. Zhao, J. R. Wen. Filter-enhanced MLP is all you need for sequential recommendation. In Proceedings of ACM Web Conference, Lyon, France, pp. 2388–2399, 2022. DOI: https://doi.org/10.1145/3485447.3512111.
X. N. He, L. Z. Liao, H. W. Zhang, L. Q. Nie, X. Hu, T. S. Chua. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, Perth, Australia, pp. 173–182, 2017. DOI: https://doi.org/10.1145/3038912.3052569.
J. X. Tang, K. Wang. Personalized Top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining, Marina Del Rey, USA, pp. 565–573, 2018. DOI: https://doi.org/10.1145/3159652.3159656.
F. Sun, J. Liu, J. Wu, C. H. Pei, X. Lin, W. W. Ou, P. Jiang. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China, pp. 1441–1450, 2019. DOI: https://doi.org/10.1145/3357384.3357895.
X. N. He, K. Deng, X. Wang, Y. Li, Y. D. Zhang, M. Wang. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, China, pp. 639–648, 2020. DOI: https://doi.org/10.1145/3397271.3401063.
X. Wang, T. L. Huang, D. X. Wang, Y. C. Yuan, Z. G. Liu, X. N. He, T. S. Chua. Learning intents behind interactions with knowledge graph for recommendation. In Proceedings of the Web Conference, ACM, Ljubljana, Slovenia, pp. 878–887, 2021. DOI: https://doi.org/10.1145/3442381.3450133.
L. P. Wang, F. Y. Hu, S. Wu, L. Wang. Fully hyperbolic graph convolution network for recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Australia, pp. 3483–3487, 2021. DOI: https://doi.org/10.1145/3459637.3482109.
R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, Y. Bengio. Learning deep representations by mutual information estimation and maximization. In Proceedings of the 7th International Conference on Learning Representations, New Orleans, USA, 2019.
D. Y. She, K. Xu. Contrastive self-supervised representation learning using synthetic data. International Journal of Automation and Computing, vol. 18, no. 4, pp. 556–567, 2021. DOI: https://doi.org/10.1007/s11633-021-1297-9.
T. Chen, S. Kornblith, M. Norouzi, G. Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, Article number 149, 2020.
J. L. Yu, H. Z. Yin, X. Xia, T. Chen, J. D. Li, Z. Huang. Self-supervised learning for recommender systems: A survey, [Online], Available: https://arxiv.org/abs/2203.15876, 2022.
Y. Zhang, F. L. Feng, X. N. He, T. X. Wei, C. G. Song, G. H. Ling, Y. D. Zhang. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Canada, pp. 11–20, 2021. DOI: https://doi.org/10.1145/3404835.3462875.
Y. Saito, S. Yaginuma, Y. Nishino, H. Sakata, K. Nakata. Unbiased recommender learning from missing-not-at-random implicit feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining, ACM, Houston, USA, pp. 501–509, 2020. DOI: https://doi.org/10.1145/3336191.3371783.
J. Li, Y. L. Ren, K. Deng. FairGAN: GANs-based fairness-aware learning for recommendations with implicit feedback. In Proceedings of the ACM Web Conference, France, pp. 297–307, 2022. DOI: https://doi.org/10.1145/3485447.3511958.
W. J. Wang, X. Y. Lin, F. L. Feng, X. N. He, M. Lin, T. S. Chua. Causal representation learning for out-of-distribution recommendation. In Proceedings of ACM Web Conference, Lyon, France, pp. 3562–3571, 2022. DOI: https://doi.org/10.1145/3485447.3512251.
D. B. Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, vol. 100, no. 469, pp. 322–331, 2005. DOI: https://doi.org/10.1198/016214504000001880.
X. J. Wang, R. Zhang, Y. Sun, J. Z. Qi. Doubly robust joint learning for recommendation on data missing not at random. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, USA, pp. 6638–6647, 2019.
W. J. Wang, F. L. Feng, X. N. He, H. W. Zhang, T. S. Chua. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Canada, pp. 1288–1297, 2021. DOI: https://doi.org/10.1145/3404835.3462962.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Xin-Yu Lin received the B. Eng. degree in artificial intelligence & robotics from School of Control Science and Engineering, Shandong University, China in 2021. She is currently a master student in School of Science, National University of Singapore, Singapore.
Her research interests include causal recommendation, causal representation learning, and multimedia analysis.
Yi-Yan Xu received the B. Sc. degree in mathematics and applied mathematics from Qianweichang College, Shanghai University, China in 2022. She is currently a master student in electronic information engineering at School of Data Science, University of Science and Technology of China, China.
Her research interest include recommender system, graph neural networks and causal inference.
Wen-Jie Wang received the B. Eng. degree in computer science and engineering from School of Computer Science and Technology, Shandong University, China in 2019. He is currently a Ph. D. degree candidate in computer science and engineering at School of Computing, National University of Singapore, Singapore. His publications have appeared in top conferences and journals such as SIGIR, KDD, WWW, and TIP. Moreover, he has served as the PC member and reviewer for the top conferences and journals including TKDE, TOIS, SIGIR, AAAI, ACMMM, and WSDM.
His research interests include causal recommendation, data mining, and multimedia.
Yang Zhang received the B. Eng. degree in electronic information engineering from University of Science and Technology of China (USTC), China in 2019. He is currently a Ph. D. degree candidate in information and communication engineering at School of Information Science and Technology, USTC, China. He has two publications in the top conference SIGIR. His work on the causal recommendation has received the Best Paper Honorable Mention in SIGIR 2021. He has served as the PC member and reviewer for the top conferences and journals including TOIS, TIST, ICML-PKDD, AAAI and WSDM.
His research interest include recommender system and causal inference.
Fu-Li Feng received Ph. D. degree in computer science from National University of Singapore, Singapore in 2019. He is a professor in University of Science and Technology of China, China. He has over 60 publications appeared in several top conferences such as SIGIR, WWW, and SIGKDD, and journals including TKDE and TOIS. He has received the Best Paper Honourable Mention of SIGIR 2021 and Best Poster Award of WWW 2018. Moreover, he has been served as the PC member for several top conferences including SIGIR, WWW, SIGKDD, NeurIPS, ICML, ICLR, ACL and invited reviewer for prestigious journals such as TOIS, TKDE, TNNLS, TPAMI.
His research interests include information retrieval, data mining, causal inference and multi-media processing.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit https://doi.org/creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Lin, XY., Xu, YY., Wang, WJ. et al. Mitigating Spurious Correlations for Self-supervised Recommendation. Mach. Intell. Res. (2023). https://doi.org/10.1007/s11633-022-1374-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11633-022-1374-8
Keywords
- Self-supervised recommendation
- spurious correlations
- spurious features
- invariant feature learning
- contrastive learning