Modeling uncertainty to improve personalized recommendations via Bayesian deep learning


Modeling uncertainty has been a major challenge in developing Machine Learning solutions to solve real world problems in various domains. In Recommender Systems, a typical usage of uncertainty is to balance exploration and exploitation, where the uncertainty helps to guide the selection of new options in exploration. Recent advances in combining Bayesian methods with deep learning enable us to express uncertain status in deep learning models. In this paper, we investigate an approach based on Bayesian deep learning to improve personalized recommendations. We first build deep learning architectures to learn useful representation of user and item inputs for predicting their interactions. We then explore multiple embedding components to accommodate different types of user and item inputs. Based on Bayesian deep learning techniques, a key novelty of our approach is to capture the uncertainty associated with the model output and further utilize it to boost exploration in the context of Recommender Systems. We test the proposed approach in both a Collaborative Filtering and a simulated online recommendation setting. Experimental results on publicly available benchmarks demonstrate the benefits of our approach in improving the recommendation performance.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Agrawal, S., Goyal, N.: Thompson sampling for contextual bandits with linear payoffs. In: Proceedings of the 30th International Conference on Machine Learning, vol. 28, pp. 127–135. PMLR (2013)

  2. 2.

    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)

    Article  Google Scholar 

  3. 3.

    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: CoRR (2014)

  4. 4.

    Bell, R., Koren, Y., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42, 30–37 (2009)

    Google Scholar 

  5. 5.

    Bishop, C.M.: Bayesian methods for neural networks. Technical report (1995)

  6. 6.

    Collier, M., Llorens, H.U.: Deep contextual multi-armed bandits. CoRR arXiv:1807.09809 (2018)

  7. 7.

    Ekstrand, M.D.: The lkpy package for recommender systems experiments. In: Computer Science Faculty Publications and Presentations, vol. 147. Boise State University. (2018). Accessed 16 July 2019

  8. 8.

    Funk, S.: Funksvd proposal (2006).

  9. 9.

    Féraud, R., Allesiardo, R., Urvoy, T., Clérot, F.: Random forest for the contextual bandit problem. In: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 51, pp. 93–101 (2016)

  10. 10.

    Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of The 33rd International Conference on Machine Learning, vol. 48, pp. 1050–1059 (2016)

  11. 11.

    Gomez-Uribe, C.A., Hunt, N.: The netflix recommender system: algorithms, business value, and innovation. ACM Trans. Manag. Inf. Syst. 6(4), 13:1–13:19 (2015)

  12. 12. (2019).

  13. 13.

    He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web, pp. 173–182. Republic and Canton of Geneva, Switzerland (2017)

  14. 14.

    Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263–272 (2008)

  15. 15.

    Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002)

    Article  Google Scholar 

  16. 16.

    Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? Adv. Neural Inf. Process. Syst. 30, 5574–5584 (2017)

    Google Scholar 

  17. 17.

    Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 426–434 (2008)

  18. 18.

    Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp. 661–670. New York, NY, USA (2010)

  19. 19.

    Liu, B., Wei, Y., Zhang, Y., Yan, Z., Yang, Q.: Transferable contextual bandit for cross-domain recommendation (2018)

  20. 20.

    Lops, P., de Gemmis, M., Semeraro, G.: Content-Based Recommender Systems: State of the Art and Trends, pp. 73–105. Springer, Boston (2011)

  21. 21.

    MacKay, D.J.C.: A practical Bayesian framework for backpropagation networks. Neural Comput. 4(3), 448–472 (1992)

    Article  Google Scholar 

  22. 22.

    McAuley, J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, pp. 43–52 (2015)

  23. 23.

    Neal, R.: Bayesian learning for neural networks. Ph.D. thesis, University of Toronto (1995)

  24. 24.

    Riquelme, C., Tucker, G., Snoek, J.: Deep Bayesian bandits showdown: an empirical comparison of Bayesian deep networks for Thompson sampling. In: International Conference on Learning Representations (2018)

  25. 25.

    Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Proceedings of the 20th International Conference on Neural Information Processing Systems, NIPS’07, pp. 1257–1264 (2007)

  26. 26.

    Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295. New York, NY, USA (2001)

  27. 27.

    Schwab, I., Koychev, I., Kobsa, A.: Learning about users from observation. In: In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI) Spring Symposium: Adaptive User Interfaces, pp. 241–247 (2000)

  28. 28.

    Shardanand, U., Maes, P.: Social information filtering: algorithms for automating“word of mouth”. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 210–217 (1995)

  29. 29.

    Slivkins, A.: Contextual bandits with similarity information. J. Mach. Learn. Res. 15(1), 2533–2568 (2014)

    MathSciNet  MATH  Google Scholar 

  30. 30.

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  31. 31.

    Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep Boltzmann machines. J. Mach. Learn. Res. 15(1), 2949–2980 (2014)

    MathSciNet  MATH  Google Scholar 

  32. 32.

    Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25, 285–294 (1933)

    Article  Google Scholar 

  33. 33.

    Tuan, T.X., Phuong, T.M.: 3d convolutional networks for session-based recommendation with content features. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys ’17, pp. 138–146 (2017)

  34. 34.

    Valko, M., Korda, N., Munos, R., Flaounas, I., Cristianini, N.: Finite-time analysis of kernelised contextual bandits. In: Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pp. 654–663 (2013)

  35. 35.

    van den Oord, A., Dieleman, S., Schrauwen, B.: Deep content-based music recommendation. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 26, pp. 2643–2651. Curran Associates Inc., Red Hook (2013)

    Google Scholar 

  36. 36.

    Wang, H., Wang, N., Yeung, D.Y.: Collaborative deep learning for recommender systems. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1235–1244 (2015)

  37. 37.

    Wang, H., Wu, Q., Wang, H.: Learning hidden features for contextual bandits. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 1633–1642 (2016)

  38. 38.

    Wang, H., Yeung, D.Y.: Towards Bayesian deep learning: a framework and some existing methods. IEEE Trans. Knowl. Data Eng. 28(12), 3395–3408 (2016)

    Article  Google Scholar 

  39. 39.

    Wang, X., Kadioglu, S.: Bayesian deep learning based exploration–exploitation for personalized recommendations. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 1715–1719 (2019)

  40. 40.

    Wu, C.Y., Ahmed, A., Beutel, A., Smola, A.J., Jing, H.: Recurrent recommender networks. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 495–503. New York, NY, USA (2017)

  41. 41.

    Yue, Y., Hong, S.A., Guestrin, C.: Hierarchical exploration for accelerating contextual bandits. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, pp. 979–986 (2012)

  42. 42.

    Zhao, Q., Chen, J., Chen, M., Jain, S., Beutel, A., Belletti, F., Chi, E.H.: Categorical-attributes-based item classification for recommender systems. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, pp. 320–328. New York, NY, USA (2018)

  43. 43.

    Zhao, X., Xia, L., Zhang, L., Ding, Z., Yin, D., Tang, J.: Deep reinforcement learning for page-wise recommendations. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, pp. 95–103 (2018)

  44. 44.

    Zheng, J., Liu, J., Shi, C., Zhuang, F., Li, J., Wu, B.: Recommendation in heterogeneous information network via dual similarity regularization. Int. J. Data Sci, Anal. 3(1), 35–48 (2017)

Download references


We would like to thank our anonymous reviewers whose feedback helped us improve the paper.

Author information



Corresponding author

Correspondence to Xin Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extended version of the short paper published in the proceedings of ICTAI 2019, “Bayesian Deep Learning based Exploration–Exploitation for Personalized Recommendations” [39]

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Kadıoğlu, S. Modeling uncertainty to improve personalized recommendations via Bayesian deep learning. Int J Data Sci Anal (2021).

Download citation


  • Bayesian deep learning
  • Exploration–exploitation
  • Personalized recommendation