Abstract
Recommender systems are deployed heavily by many online platforms for better user engagement and providing recommendations. Despite being so popular, several works have shown the existence of popularity bias due to the non-random nature of missing data. Popularity bias leads to the recommendation of only a few popular items causing starvation of many non-popular items. This paper considers an easy-to-understand metric to evaluate the popularity bias as the difference between mean squared error on popular and non-popular items. Then, we propose EqBal-RS, a novel re-weighting technique that updates the weights of popular and non-popular items. Re-weighting ensures that both item sets are equally balanced during training using a trade-off function between overall loss and popularity bias. Our experiments on real-world datasets show that EqBal-RS outperforms the existing state-of-art algorithms in terms of accuracy, quality, and fairness. EqBal-RS works well on the proposed and existing popularity bias metrics and has significantly reduced runtime. The code is publicly available at https://github.com/eqbalrs/EqBalRS
Similar content being viewed by others
Availability of supporting data and material
All datasets used in the experiments are publicly available and cited.
Code Availability
The code has been made publicly available at https://github.com/eqbalrs/EqBalRS
References
Abdollahpouri, H., & Burke, R. (2019). Reducing popularity bias in recommendation over time. arXiv preprint. arXiv:1906.11711. https://doi.org/10.48550/arXiv.1906.11711
Abdollahpouri, H., Burke, R., & Mobasher, B. (2017). Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the Eleventh ACM conference on recommender systems (pp. 42–46). https://doi.org/10.1145/3109859.3109912
Abdollahpouri, H., Adomavicius, G., Burke, R. et al. (2019a). Beyond personalization: Research directions in multistakeholder recommendation. arXiv:1905.01986. https://doi.org/10.1007/s11257-019-09256-1
Abdollahpouri, H., Burke, R., & Mobasher, B. (2019b). Managing popularity bias in recommender systems with personalized re-ranking. arXiv:1901.07555. https://doi.org/10.48550/arXiv.1901.07555
Abdollahpouri, H., Mansoury, M., Burke, R. et al. (2019c). The impact of popularity bias on fairness and calibration in recommendation. arXiv:1910.05755. https://doi.org/10.48550/arXiv.1910.05755
Abdollahpouri, H., Mansoury, M., Burke, R. et al. (2019d). The unfairness of popularity bias in recommendation. arXiv:1907.13286. https://doi.org/10.48550/arXiv.1907.13286
Abdollahpouri, H., Mansoury, M., Burke, R. et al. (2020). The connection between popularity bias, calibration, and fairness in recommendation. In: Proceedings of the 14th ACM conference on recommender systems (pp. 726–731). https://doi.org/10.1145/3383313.3418487
Abdollahpouri, H., Mansoury, M., Burke, R. et al. (2021). User-centered evaluation of popularity bias in recommender systems. In: Proceedings of the 29th ACM conference on user modeling, adaptation and personalization (pp. 119–129). https://doi.org/10.1145/3450613.3456821
Aljunid, M. F., & Dh, M. (2020). An efficient deep learning approach for collaborative filtering recommender system. Procedia Computer Science, 171, 829–836. https://doi.org/10.1016/j.procs.2020.04.090
Amatriain, X., Pujol, J. M., Oliver, N. (2009). I like it... i like it not: Evaluating user ratings noise in recommender systems. In: User modeling, adaptation, and personalization: 17th international conference, UMAP 2009, formerly UM and AH, Trento, Italy, Proceedings 17. Springer (pp. 247–258). 22-26 June 2009. https://doi.org/10.1007/978-3-642-02247-0_24
Anelli, V. W., Deldjoo, Y., Di Noia, T., et al. (2022). User-controlled federated matrix factorization for recommender systems. Journal of Intelligent Information Systems, 58(2), 287–309. https://doi.org/10.1007/s10844-021-00688-z
Antikacioglu, A., & Ravi, R. (2017). Post processing recommender systems for diversity. In: Proceedings of the 23rd ACM SIGKDD International conference on knowledge discovery and data mining (pp. 707–716). https://doi.org/10.1145/3097983.3098173
Behera, G., & Nain, N. (2022). DeepNNMF: deep nonlinear non-negative matrix factorization to address sparsity problem of collaborative recommender system. International Journal of Information Technology, 14(7), 3637–3645. https://doi.org/10.1007/s41870-022-00982-1
Bellogín, A., Castells, P., & Cantador, I. (2017). Statistical biases in information retrieval metrics for recommender systems. Information Retrieval Journal, 20, 606–634. https://doi.org/10.1007/s10791-017-9312-z
Boratto, L., Fenu, G., & Marras, M. (2021). Connecting user and item perspectives in popularity debiasing for collaborative recommendation. Information Processing & Management, 58(1), 102387. https://doi.org/10.1016/j.ipm.2020.102387
Borges, R., & Stefanidis, K. (2021). on mitigating popularity bias in recommendations via variational autoencoders. In: Proceedings of the 36th annual ACM symposium on applied computing (pp. 1383–1389). https://doi.org/10.1145/3412841.3442123
Carraro, D., & Bridge, D. (2022). A sampling approach to debiasing the offline evaluation of recommender systems. Journal of Intelligent Information Systems, 1–26. https://doi.org/10.1007/s10844-021-00651-y
Chen, L., De Gemmis, M., Felfernig, A., et al. (2013). Human decision making and recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 3(3), 1–7. https://doi.org/10.1145/2533670.2533675
Chen, L., Yang, W., Li, K., et al. (2021). Distributed matrix factorization based on fast optimization for implicit feedback recommendation. Journal of Intelligent Information Systems, 56, 49–72. https://doi.org/10.1007/s10844-020-00601-0
Chen, S. H., Sou, S. I., & Hsieh, H. P. (2023). Top-n music recommendation framework for precision and novelty under diversity group size and similarity. Journal of Intelligent Information Systems, 1–26. https://doi.org/10.1007/s10844-023-00784-2
Chen, Z., Wu, J., Li, C. et al. (2022). Co-training disentangled domain adaptation network for leveraging popularity bias in recommenders. In: Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval (pp. 60–69). https://doi.org/10.1145/3477495.3531952
Chouldechova, A., & Roth, A. (2020). A snapshot of the frontiers of fairness in machine learning. Communications of the ACM, 63(5), 82–89. https://doi.org/10.1145/3376898
D’Amico, E., Gabbolini, G., Bernardis, C., et al. (2022). Analyzing and improving stability of matrix factorization for recommender systems. Journal of Intelligent Information Systems, 58(2), 255–285. https://doi.org/10.1007/s10844-021-00686-1
Dara, S., Chowdary, C. R., & Kumar, C. (2020). A survey on group recommender systems. Journal of Intelligent Information Systems, 54(2), 271–295. https://doi.org/10.1007/s10844-018-0542-3
Elahi, M., Kholgh, D. K., Kiarostami, M. S., et al. (2021). Investigating the impact of recommender systems on user-based and item-based popularity bias. Information Processing & Management, 58(5), 102655. https://doi.org/10.1016/j.ipm.2021.102655
Eren, M. E., Richards, L. E., Bhattarai, M. et al. (2022). FedSPLIT: one-shot federated recommendation system based on non-negative joint matrix factorization and knowledge distillation. arXiv:2205.02359. https://doi.org/10.48550/arXiv.2205.02359
Ferwerda, B., Ingesson, E., Berndl, M. et al. (2023). I don’t care how popular you are! investigating popularity bias in music recommendations from a user’s perspective. In: Proceedings of the 2023 conference on human information interaction and retrieval (pp. 357–361). https://doi.org/10.1145/3576840.3578287
Gupta, P., Sharma, A., Malhotra, P. et al. (2021). Causer: Causal session-based recommendations for handling popularity bias. In: Proceedings of the 30th ACM international conference on information & knowledge management (pp. 3048–3052). https://doi.org/10.1145/3459637.3482071
Gupta, S., Ghalme, G., Krishnan, N. C., & Jain, S. (2023a) Efficient algorithms for fair clustering with a new notion of fairness. Data Mining and Knowledge Discovery. 1–39
Gupta, S., Ghalme, G., Krishnan, N. C., & Jain, S. (2023b). Group Fair Clustering Revisited–Notions and Efficient Algorithm. In Proceedings of the 2023 International Conference on AutonomousAgents and Multiagent Systems, (pp. 2854–2856).
He, M., Li, C., Hu, X. et al. (2022). Mitigating popularity bias in recommendation via counterfactual inference. In: International conference on database systems for advanced applications. Springer (pp. 377–388). https://doi.org/10.1007/978-3-031-00129-1_32
He, X., Liao, L., Zhang, H. et al. (2017). Neural collaborative filtering. In: Proceedings of the 26th international conference on world wide web (pp. 173–182). https://doi.org/10.1145/3038912.3052569
Hitt, M. A. (2007). The long tail: Why the future of business is selling less of more
Huang, J., Oosterhuis, H., & de Rijke, M. (2022). It is different when items are older: Debiasing recommendations when selection bias and user preferences are dynamic. In: Proceedings of the fifteenth ACM international conference on web search and data mining (pp. 381–389). https://doi.org/10.1145/3488560.3498375
Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422–446. https://doi.org/10.1145/582415.582418
Jawaheer, G., Szomszor, M., Kostkova, P. (2010). Comparison of implicit and explicit feedback from an online music recommendation service. In: Proceedings of the 1st international workshop on information heterogeneity and fusion in recommender systems (pp. 47–51). https://doi.org/10.1145/1869446.1869453
Karboua, S., Harrag, F., Meziane, F. et al. (2022). Mitigation of popularity bias in recommendation systems. In: Tunisian-algerian joint conference on applied computing. https://doi.org/10.48550/arXiv.2211.01154
Khenissi, S., & Nasraoui, O. (2020). Modeling and counteracting exposure bias in recommender systems. arXiv:2001.04832. https://doi.org/10.48550/arXiv.2001.04832
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980
Klimashevskaia, A., Elahi, M., Jannach, D. et al. (2022). Mitigating popularity bias in recommendation: Potential and limits of calibration approaches. In: International workshop on algorithmic bias in search and recommendation, Springer (pp. 82–90). https://doi.org/10.1007/978-3-031-09316-6_8
Konjengbam, A., Kumar, N., & Singh, M. (2020). Unsupervised tag recommendation for popular and cold products. Journal of Intelligent Information Systems, 54, 545–566. https://doi.org/10.1007/s10844-019-00574-9
Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37. https://doi.org/10.1109/MC.2009.263
Kowald, D., & Lacic, E. (2022). Popularity bias in collaborative filtering-based multimedia recommender systems. In: Advances in bias and fairness in information retrieval: third international workshop, BIAS 2022, Stavanger, Norway, Revised Selected Papers, Springer (pp. 1–11). 10 Apr 2022. https://doi.org/10.1007/978-3-031-09316-6_1
Kowald, D., Schedl, M., & Lex, E. (2020). The unfairness of popularity bias in music recommendation: A reproducibility study. In: Advances in information retrieval: 42nd European conference on IR research, ECIR 2020, Lisbon, Portugal, Proceedings, Part II 42, Springer (pp. 35–42). 14–17 Apr 2020. https://doi.org/10.1007/978-3-030-45442-5_5
Kowald, D., Mayr, G., Schedl, M. et al. (2023). A study on accuracy, miscalibration, and popularity bias in recommendations. arXiv:2303.00400. https://doi.org/10.1007/978-3-031-37249-0_1
Krishnan, A., Sharma, A., Sankar, A. et al. (2018). An adversarial approach to improve long-tail performance in neural collaborative filtering. In: Proceedings of the 27th ACM international conference on information and knowledge management (pp. 1491–1494). https://doi.org/10.1145/3269206.3269264
Lara-Cabrera, R., González-Prieto, Á., & Ortega, F. (2020). Deep matrix factorization approach for collaborative filtering recommender systems. Applied Sciences, 10(14), 4926. https://doi.org/10.3390/app10144926
Li, Y., Chen, H., Fu, Z., et al. (2021). User-oriented fairness in recommendation. Proceedings of the Web Conference, 2021, 624–632. https://doi.org/10.1145/3442381.3449866
Liu, H., Wang, W., Zhang, Y., et al. (2022). Neural matrix factorization recommendation for user preference prediction based on explicit and implicit feedback. Computational Intelligence and Neuroscience, 2022,. https://doi.org/10.1155/2022/9593957
Liu, Q., Tian, F., Zheng, Q., et al. (2023). Disentangling interest and conformity for eliminating popularity bias in session-based recommendation. Knowledge and Information Systems, 65(6), 2645–2664. https://doi.org/10.1007/s10115-023-01839-0
Liu, S., Ge, Y., Xu, S. et al. (2022b). Fairness-aware federated matrix factorization. In: Proceedings of the 16th ACM conference on recommender systems (pp. 168–178). https://doi.org/10.1145/3523227.3546771
Liu, Z., Fang, Y., & Wu, M. (2023). Mitigating popularity bias for users and items with fairness-centric adaptive recommendation. ACM Transactions on Information Systems, 41(3), 1–27. https://doi.org/10.1145/3564286
Mai, P., & Pang, Y. (2023). Privacy-preserving multi-view matrix factorization for recommender systems. IEEE Transactions on Artificial Intelligence. https://doi.org/10.1109/TAI.2023.3240700
Mansoury, M., Abdollahpouri, H., Smith, J. et al. (2020). Investigating potential factors associated with gender discrimination in collaborative recommender systems. In: Proceedings of the 33rd international florida artificial intelligence research society conference, FLAIRS 2020 (pp. 193–196). https://doi.org/10.48550/arXiv.2002.07786
Musto, C., Lops, P., Semeraro, G. et al. (2021). Fairness and popularity bias in recommender systems: an empirical evaluation. In: CEUR workshop PROCEEDINGS (pp. 77–91)
Naghiaei, M., Rahmani, H. A., Dehghan, M. (2022). The unfairness of popularity bias in book recommendation. In: Advances in bias and fairness in information retrieval: third international workshop, BIAS 2022, Stavanger, Norway, revised selected papers, Springer (pp. 69–81). 10 Apr 2022. https://doi.org/10.1007/978-3-030-45442-5_5
Nguyen, H., & Dinh, T. (2012). A modified regularized non-negative matrix factorization for movielens. In: 2012 IEEE RIVF International conference on computing & communication technologies, research, innovation, and vision for the Future, IEEE (pp. 1–5). https://doi.org/10.1109/rivf.2012.6169831
Nguyen, P. T., Rubei, R., Di Rocco, J. et al. (2023). Dealing with popularity bias in recommender systems for third-party libraries: How far are we? arXiv:2304.10409. https://doi.org/10.48550/arXiv.2304.10409
Nikolov, D., Lalmas, M., Flammini, A., et al. (2019). Quantifying biases in online information exposure. Journal of the Association for Information Science and Technology, 70(3), 218–229. https://doi.org/10.1002/asi.24121
Ovaisi, Z., Ahsan, R., Zhang, Y., et al. (2020). Correcting for selection bias in learning-to-rank systems. Proceedings of The Web Conference, 2020, 1863–1873. https://doi.org/10.1145/3366423.3380255
Rahmani, H. A., Deldjoo, Y., Tourani, A. et al. (2022). The unfairness of active users and popularity bias in point-of-interest recommendation. In: Advances in bias and fairness in information retrieval: third international workshop, BIAS 2022, Stavanger, Norway, revised selected papers, Springer (pp. 56–68). 10 Apr 2022. https://doi.org/10.1007/978-3-031-09316-6_6
Ren, W., Wang, L., Liu, K. et al. (2022). Mitigating popularity bias in recommendation with unbalanced interactions: a gradient perspective. In: 2022 IEEE International conference on data mining (ICDM), IEEE (pp. 438–447). https://doi.org/10.1109/ICDM54844.2022.00054
Saito, K., Ushiku, Y., & Harada, T. (2017). Asymmetric tri-training for unsupervised domain adaptation. In: International conference on machine learning, PMLR (pp. 2988–2997). https://doi.org/10.5555/3305890.3305990
Saito, Y. (2020). Asymmetric tri-training for debiasing missing-not-at-random explicit feedback. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in information retrieval (pp. 309–318). https://doi.org/10.1145/3397271.3401114
Saito, Y., Yaginuma, S., Nishino, Y. et al. (2020). Unbiased recommender learning from missing-not-at-random implicit feedback. In: Proceedings of the 13th international conference on web search and data mining (pp. 501–509). https://doi.org/10.1145/3336191.3371783
San Ramon, M. G. (2020). Ten states sue google for ‘anti-competitive’ online ad sales. https://brandequity.economictimes.indiatimes.com/news/digital/ten-states-sue-google-for-anti-competitive-online-ad-sales/79771479. Accessed 10 Jan 2023.
Schnabel, T., Swaminathan, A., Singh, A. et al. (2016). Recommendations as treatments: Debiasing learning and evaluation. In: International conference on machine learning, PMLR (pp. 1670–1679). https://doi.org/10.48550/arXiv.1602.05352
Sinha, B. B., & Dhanalakshmi, R. (2022). DNN-MF: Deep neural network matrix factorization approach for filtering information in multi-criteria recommender systems. Neural Computing and Applications, 34(13), 10807–10821. https://doi.org/10.1007/s00521-022-07012-y
Stinson, C. (2022). Algorithms are not neutral: Bias in collaborative filtering. AI and Ethics, 2(4), 763–770. https://doi.org/10.1007/s43681-022-00136-w
Tahmasbi, H., Jalali, M., & Shakeri, H. (2021). TSCMF: Temporal and social collective matrix factorization model for recommender systems. Journal of Intelligent Information Systems, 56, 169–187. https://doi.org/10.1007/s10844-020-00613-w
Takács, G., Pilászy, I., Németh, B. et al. (2008). Matrix factorization and neighbor based algorithms for the netflix prize problem. In: Proceedings of the 2008 ACM conference on recommender systems (pp. 267–274). https://doi.org/10.1145/1454008.1454049
Urbano, J., Schedl, M., & Serra, X. (2013). Evaluation in music information retrieval. Journal of Intelligent Information Systems, 41(3), 345–369. https://doi.org/10.1007/s10844-013-0249-4
Wan, Q., He, X., Wang, X., et al. (2022). Cross pairwise ranking for unbiased item recommendation. Proceedings of the ACM Web Conference, 2022, 2370–2378. https://doi.org/10.1145/3485447.3512010
Wang, Y., Gao, M., Ran, X., et al. (2023). An improved matrix factorization with local differential privacy based on piecewise mechanism for recommendation systems. Expert Systems with Applications, 216, 119457. https://doi.org/10.1016/j.eswa.2022.119457
Wei, T., Feng, F., Chen, J. et al. (2021). Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining (pp. 1791–1800). https://doi.org/10.1145/3447548.3467289
Xue, H. J., Dai, X., Zhang, J. et al. (2017). Deep matrix factorization models for recommender systems. In: Proceedings of the 26th international joint conference on artificial intelligence, Melbourne, Australia (pp. 3203–3209). https://doi.org/10.5555/3172077.3172336
Yalcin, E. (2021). Blockbuster: A new perspective on popularity-bias in recommender systems. In: 2021 6th International conference on computer science and engineering (UBMK), IEEE (pp. 107–112). https://doi.org/10.1109/UBMK52708.2021.9558877
Yalcin, E., & Bilge, A. (2021). Investigating and counteracting popularity bias in group recommendations. Information Processing & Management, 58(5), 102608. https://doi.org/10.1016/j.ipm.2021.102608
Yalcin, E., & Bilge, A. (2022). Evaluating unfairness of popularity bias in recommender systems: A comprehensive user-centric analysis. Information Processing & Management, 59(6), 103100. https://doi.org/10.1016/j.ipm.2022.103100
Yalcin, E., & Bilge, A. (2023). Popularity bias in personality perspective: An analysis of how personality traits expose individuals to the unfair recommendation. Concurrency and Computation: Practice and Experience e7647. https://doi.org/10.1002/cpe.7647
Yin, H., Cui, B., Li, J. et al. (2012). Challenging the long tail recommendation. Proceedings of the VLDB Endowment 5(9). https://doi.org/10.14778/2311906.2311916
Zehlike, M., Bonchi, F., Castillo, C. et al. (2017). Fa* ir: A fair top-k ranking algorithm. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1569–1578. https://doi.org/10.1145/3132847.3132938
Zhang, Y., Feng, F., He, X. et al. (2021). Causal intervention for leveraging popularity bias in recommendation. In: Proceedings of the 44th International ACM SIGIR conference on research and development in information retrieval (pp. 11–20). https://doi.org/10.1145/3404835.3462875
Zhang, Z., Liu, Y., Xu, G., et al. (2016). Recommendation using dmf-based fine tuning method. Journal of Intelligent Information Systems, 47, 233–246. https://doi.org/10.1007/s10844-016-0407-6
Zheng, Y., Gao, C., Li, X., et al. (2021). Disentangling user interest and conformity for recommendation with causal embedding. Proceedings of the Web Conference, 2021, 2980–2991. https://doi.org/10.1145/3442381.3449788
Zhu, Z., He, Y., Zhao, X. et al. (2021). Popularity-opportunity bias in collaborative filtering. In: Proceedings of the 14th ACM international conference on web search and data mining (pp. 85–93). https://doi.org/10.1145/3437963.3441820
Acknowledgements
We express our gratitude to the reviewers for their invaluable feedback, which has greatly enhanced the manuscript’s quality. We also wish to express our heartfelt thanks to the entire editorial and production team for their efforts.
Funding
The authors thank the Prime Minister Research Fellowship for generously funding Shivam Gupta (ID: 2901481) for this work. The research is further supported by the Department of Science & Technology, India, with grant number SRG/2020/001138.
Author information
Authors and Affiliations
Contributions
Shivam Gupta formulated the problem statement, proposed algorithm EqBal-RS and executed experiments. Kirandeep Kaur helped review the literature, providing useful insights and executing MF, MFR baselines. Dr. Shweta Jain provided valuable insights and feedback throughout the work. All authors wrote the manuscript text and reviewed the manuscript.
Corresponding author
Ethics declarations
Consent for publication
The paper is the authors’ own original work, which has not been previously published elsewhere. The paper is not currently being considered for publication elsewhere. The paper reflects the author’s own research and analysis truthfully and completely. The paper properly credits the meaningful contributions of co-authors and co-researchers.
Competing interests
No potential competing interest was reported by the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gupta, S., Kaur, K. & Jain, S. EqBal-RS: Mitigating popularity bias in recommender systems. J Intell Inf Syst (2023). https://doi.org/10.1007/s10844-023-00817-w
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10844-023-00817-w