Abstract
In today’s business marketplace, the great demand for developing intelligent interactive recommendation systems is growing rapidly, which sequentially suggest users proper items by accurately predicting their preferences, while receiving up-to-date feedback to promote the overall performance. Multi-armed bandit, which has been widely applied to various online systems, is quite capable of delivering such efficient recommendation services. To further enhance online recommendations, many works have introduced clustering techniques to fully utilize users’ information. These works consider symmetric relations between users, i.e., users in one cluster share equal weights. However, in practice, users usually have different interaction frequency (i.e., activeness) in one cluster, and their collaborative relations are unsymmetrical. This brings a challenge for bandit clustering since inactive users lack the capability of leveraging these interaction information to mitigate the cold-start problem, and further affect active ones belonging to one cluster. In this work, we explore user activeness and propose a frequency-dependent clustering of bandit model to deal with the aforementioned challenge. The model learns representation of each user’s cluster by sharing collaborative information weighed based on user activeness, i.e., inactive users can utilize the collaborative information from active ones in the same cluster to optimize the cold start process. Extensive studies have been carefully conducted on both synthetic data and two real-world datasets indicating the efficiency and effectiveness of our proposed model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Our code and data is available at https://github.com/holywoodys/FreqCB.
References
Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005). https://doi.org/10.1109/TKDE.2005.99
Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32(1), 48–77 (2002). https://doi.org/10.1137/S0097539701398375
Ban, Y., He, J.: Local clustering in contextual multi-armed bandits. In: Proceedings of the Web Conference 2021, pp. 2335–2346, WWW 2021. Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3442381.3450058
Cesa-Bianchi, N., Gentile, C., Zappella, G.: A gang of bandits (2013)
Gangan, E., Kudus, M., Ilyushin, E.: Survey of multiarmed bandit algorithms applied to recommendation systems. Int. J. Open Inf. Technol. 9, 12–27 (2021)
Gentile, C., Li, S., Kar, P., Karatzoglou, A., Zappella, G., Etrue, E.: On context-dependent clustering of bandits. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1253–1262. PMLR, 6–11 August 2017. https://proceedings.mlr.press/v70/gentile17a.html
Gentile, C., Li, S., Zappella, G.: Online clustering of bandits. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 757–765. PMLR, Beijing, China, 22–24 June 2014. https://proceedings.mlr.press/v32/gentile14.html
Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 1–19 (2015). https://doi.org/10.1145/2827872
Korda, N., Szorenyi, B., Li, S.: Distributed clustering of linear bandits in peer to peer networks. In: Balcan, M.F., Weinberger, K.Q. (eds.) Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 48, pp. 1301–1309. PMLR, New York, New York, USA, 20–22 June 2016. https://proceedings.mlr.press/v48/korda16.html
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). https://doi.org/10.1109/MC.2009.263
Lattimore, T., Szepesvári, C.: Bandit Algorithms. Cambridge University Press, Cambridge (2020). https://doi.org/10.1017/9781108571401
Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 661–670. Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1772690.1772758, https://doi.org/10.1145/1772690.1772758
Li, S., Chen, W., Li, S., Leung, K.S.: Improved algorithm on online clustering of bandits (2019)
Li, S., Karatzoglou, A., Gentile, C.: Collaborative filtering bandits. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 539–548. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2911451.2911548
Lika, B., Kolomvatsos, K., Hadjiefthymiades, S.: Facing the cold start problem in recommender systems. Expert Syst. Appl. 41(4, Part 2), 2065–2073 (2014). https://doi.org/10.1016/j.eswa.2013.09.005, https://www.sciencedirect.com/science/article/pii/S0957417413007240
MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297, Oakland, CA, USA (1967)
Nguyen, T.T., Lauw, H.W.: Dynamic clustering of contextual multi-armed bandits. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM 2014, pp. 1959–1962. Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2661829.2662063
Pandey, S., Chakrabarti, D., Agarwal, D.: Multi-armed bandit problems with dependent arms. In: Proceedings of the 24th International Conference on Machine Learning, ICML 2007, pp. 721–728. Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1273496.1273587
Pazzani, M.J., Billsus, D.: Content-based recommendation systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web. LNCS, vol. 4321, pp. 325–341. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72079-9_10
Wang, Q., et al.: Online interactive collaborative filtering using multi-armed bandit with dependent arms. IEEE Trans. Knowl. Data Eng. 31(8), 1569–1580 (2019). https://doi.org/10.1109/TKDE.2018.2866041
Xu, X., Dong, F., Li, Y., He, S., Li, X.: Contextual-bandit based personalized recommendation with time-varying user interests. Proc. AAAI Conf. Artif. Intell. 34(04), 6518–6525 (2020). https://doi.org/10.1609/aaai.v34i04.6125, https://ojs.aaai.org/index.php/AAAI/article/view/6125
Yang, L., Liu, B., Lin, L., Xia, F., Chen, K., Yang, Q.: Exploring clustering of bandits for online recommendation system. In: Fourteenth ACM Conference on Recommender Systems, RecSys 2020, pp. 120–129. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3383313.3412250
Acknowledgement
This work is partially supported by China Natural Science Foundation under grant (No. 62171391) and the Natural Science Foundation of Fujian Province of China under grant (No. 2020J01053).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, S., Zhou, Q., Wang, Q. (2023). Clustering of Bandit with Frequency-Dependent Information Sharing. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-28238-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28237-9
Online ISBN: 978-3-031-28238-6
eBook Packages: Computer ScienceComputer Science (R0)