A Contextual Bandit Approach to Personalized Online Recommendation via Sparse Interactions

  • Chenyu Zhang
  • Hao WangEmail author
  • Shangdong Yang
  • Yang Gao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11440)


Online recommendation is an important feature in many applications. In practice, the interaction between the users and the recommender system might be sparse, i.e., the users are not always interacting with the recommender system. For example, some users prefer to sweep around the recommendation instead of clicking into the details. Therefore, a response of 0 may not necessarily be a negative response, but a non-response. It comes worse to distinguish these two situations when only one item is recommended to the user each time and few further information is reachable. Most existing recommendation strategies ignore the difference between non-responses and negative responses. In this paper, we propose a novel approach, named SAOR, to make online recommendations via sparse interactions. SAOR uses positive and negative responses to build the user preference model, ignoring all non-responses. Regret analysis of SAOR is provided, experiments on both real and synthetic datasets also show that SAOR outperforms competing methods.


Online recommendation Sparse interaction Contextual bandit 



This work was supported by the National Key R&D Program of China (2017YFB0702600, 2017YFB0702601) and the National Natural Science Foundation of China (61432008, U1435214, 61503178).


  1. 1.
    Abbasi-Yadkori, Y., Pál, D., Szepesvári, C.: Improved algorithms for linear stochastic bandits. In: Advances in Neural Information Processing Systems, pp. 2312–2320 (2011)Google Scholar
  2. 2.
    Agrawal, S., Goyal, N.: Thompson sampling for contextual bandits with linear payoffs. In: Proceedings 30th International Conference on Machine Learning (ICML 2013), pp. 127–135 (2013)Google Scholar
  3. 3.
    Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3(Nov), 397–422 (2002)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)CrossRefGoogle Scholar
  5. 5.
    Chu, W., Li, L., Reyzin, L., Schapire, R.E.: Contextual bandits with linear payoff functions. In: Proceedings 14th International Conference on Artificial Intelligence and Statistics (AISTTS 2011), pp. 208–214 (2011)Google Scholar
  6. 6.
    Chuklin, A., Markov, I., Rijke, M.d.: Click models for web search. In: Synthesis Lectures on Information Concepts, Retrieval, and Services, vol. 7, no. 3, pp. 1–115 (2015)Google Scholar
  7. 7.
    Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19(90), 297–301 (1965)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Craswell, N., Zoeter, O., Taylor, M.J., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of International Conference on Web Search Data Min (WSDM 2008), pp. 87–94 (2008)Google Scholar
  9. 9.
    Gentile, C., Li, S., Kar, P., Karatzoglou, A., Zappella, G., Etrue, E.: On context-dependent clustering of bandits. In: Proceedings of 34th International Conference on Machine Learning (ICML 2013), pp. 1253–1262 (2017)Google Scholar
  10. 10.
    Koren, Y.: Collaborative filtering with temporal dynamics. ACM Commun. 53(4), 89–97 (2010)CrossRefGoogle Scholar
  11. 11.
    Kveton, B., Szepesvari, C., Wen, Z., Ashkan, A.: Cascading bandits: learning to rank in the cascade model. In: Proceedings of 32nd International Conference on Machine Learning (ICML 2015), pp. 767–776 (2015)Google Scholar
  12. 12.
    Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of 19th International Conference on World Wide Web (WWW 2010), pp. 661–670 (2010)Google Scholar
  13. 13.
    Li, L., Chu, W., Langford, J., Wang, X.: Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In: Proceedings of 4th ACM International Conference on Web Search Data Mining (WSDM 2011), pp. 297–306. ACM (2011)Google Scholar
  14. 14.
    Li, L., Lu, Y., Zhou, D.: Provable optimal algorithms for generalized linear contextual bandits. arXiv preprint arXiv:1703.00048 (2017)
  15. 15.
    Li, S., Karatzoglou, A., Gentile, C.: Collaborative filtering bandits. In: Proceedings of 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 539–548. ACM (2016)Google Scholar
  16. 16.
    Liu, J., Dolan, P., Pedersen, E.R.: Personalized news recommendation based on click behavior. In: Proceedings of 15th International Conference on Intelligent User Interfaces (IUI 2010), pp. 31–40 (2010)Google Scholar
  17. 17.
    Ren, L., Gu, J., Xia, W.: A temporal item-based collaborative filtering approach. In: Signal Processing, Image Processing and Pattern Recognition (SIP 2011), pp. 414–421 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Chenyu Zhang
    • 1
  • Hao Wang
    • 2
    Email author
  • Shangdong Yang
    • 1
  • Yang Gao
    • 1
  1. 1.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.Inception Institute of Artificial IntelligenceAbu DhabiUAE

Personalised recommendations