Markov Chain Monte Carlo for Effective Personalized Recommendations

  • Michail-Angelos Papilaris
  • Georgios ChalkiadakisEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11450)


This paper adopts a Bayesian approach for finding top recommendations. The approach is entirely personalized, and consists of learning a utility function over user preferences via employing a sampling-based, non-intrusive preference elicitation framework. We explicitly model the uncertainty over the utility function and learn it through passive user feedback, provided in the form of clicks on previously recommended items. The utility function is a linear combination of weighted features, and beliefs are maintained using a Markov Chain Monte Carlo algorithm. Our approach overcomes the problem of having conflicting user constraints by identifying a convex region within a user’s preferences model. Additionally, it handles situations where not enough data about the user is available, by exploiting the information from clusters of (feature) weight vectors created by observing other users’ behavior. We evaluate our system’s performance by applying it in the online hotel booking recommendations domain using a real-world dataset, with very encouraging results.


Adaptation and learning Recommender systems 



The authors would like to thank Professor Michail Lagoudakis for extremely useful suggestions for improving an earlier version of this work.


  1. 1.
    Andrieu, C., de Freitas, N., Doucet, A., Jordan, M.I.: An introduction to MCMC for machine learning. Mach. Learn. 50(1), 5–43 (2003). Scholar
  2. 2.
    Applegate, D., Kannan, R.: Sampling and integration of near log-concave functions. In: Proceedings of the Twenty-Third Annual ACM Symposium on Theory of Computing, pp. 156–163 (1991)Google Scholar
  3. 3.
    Babas, K., Chalkiadakis, G., Tripolitakis, E.: You are what you consume: a Bayesian method for personalized recommendations. In: Proceedings of the 7th ACM Conference on Recommender Systems (ACM RecSys 2013), Hong Kong, China (2013)Google Scholar
  4. 4.
    Bishop, C.M.: Neural networks for pattern recognition (1995)Google Scholar
  5. 5.
    Boutilier, C.: A POMDP formulation of preference elicitation problems. In: Proceedings of the 18th AAAI Conference, AAAI 2002 (2002)Google Scholar
  6. 6.
    Bowling, M.H., Veloso, M.M.: Multiagent learning using a variable learning rate. Artif. Intell. (AIJ) 136(2), 215–250 (2002)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chajewska, U., Koller, D., Parr, R.: Making rational decisions using adaptive utility elicitation. In: Proceedings of the 17th AAAI Conference, AAAI 2000, pp. 363–369 (2000)Google Scholar
  8. 8.
    Chajewska, U., Koller, D., Ormoneit, D.: Learning an agent’s utility function by observing behavior. In: Proceedings of the International Conference on Machine Learning (ICML) (2001)Google Scholar
  9. 9.
    Chib, S., Greenberg, E.: Understanding the Metropolis-Hastings algorithm, pp. 327–335 (2012)Google Scholar
  10. 10.
    Dantzig, G.B., Orden, A., Wolfe, P.: The generalized simplex method for minimizing a linear form under linear inequality restraints. Pacific J. Math. 5(2), 183–195 (1955)MathSciNetCrossRefGoogle Scholar
  11. 11.
    DeGroot, M.: Probability and Statistics. Addison-Wesley Series in Behavioral Science, Addison-Wesley Publishing Company, Boston (1975).
  12. 12.
    Dong, R., Smyth, B.: From more-like-this to better-than-this: hotel recommendations from user generated reviews. In: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (UMAP 2016), pp. 309–310 (2016)Google Scholar
  13. 13.
    Gilks, W.R., Wild, P.: Adaptive rejection sampling for Gibbs sampling. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 41(2), 337–348 (1992)CrossRefGoogle Scholar
  14. 14.
    Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970). Scholar
  15. 15.
    Keeney, R.L., Raiffa, H.: Decisions with Multiple Objectives: Decisions with Preferences and Value Tradeoffs. Cambridge University Press, Cambridge (1993)Google Scholar
  16. 16.
    MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability (1967)Google Scholar
  17. 17.
    Nasery, M., Braunhofer, M., Ricci, F.: Recommendations with optimal combination of feature-based and item-based preferences. In: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (UMAP 2016), pp. 269–273 (2016)Google Scholar
  18. 18.
    Ng, A.Y., Russell, S.J.: Algorithms for inverse reinforcement learning. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 663–670. ICML (2000)Google Scholar
  19. 19.
    Ramachandran, D., Amir, E.: Bayesian inverse reinforcement learning. In: Proceedings of IJCAI-2007, pp. 2586–2591 (2007)Google Scholar
  20. 20.
    Ricci, F., Rokach, L., Shapira, B., Kantor, P.B.: Recommender Systems Handbook, 1st edn. Springer, New York (2010). Scholar
  21. 21.
    Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer, Heidelberg (2004). Scholar
  22. 22.
    Roy, S.B., Das, G., Amer-Yahia, S., Yu, C.: Interactive itinerary planning. In: Proceedings of the IEEE International Conference on Data Engineering (ICDE) (2011)Google Scholar
  23. 23.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice Hall, Upper Saddle River (2009)zbMATHGoogle Scholar
  24. 24.
    Tripolitakis, E., Chalkiadakis, G.: Probabilistic topic modeling, reinforcement learning, and crowdsourcing for personalized recommendations. In: Criado Pacheco, N., Carrascosa, C., Osman, N., Julián Inglada, V. (eds.) EUMAS/AT -2016. LNCS (LNAI), vol. 10207, pp. 157–171. Springer, Cham (2017). Scholar
  25. 25.
    Xie, M., Lakshmanan, L.V., Wood, P.T.: Generating top-k packages via preference elicitation. Proc. VLDB Endow. 7(14), 1941–1952 (2014)CrossRefGoogle Scholar
  26. 26.
    Yanxiang, L., Deke, G., Fei, C., Honghui, C.: User-based clustering with top-n recommendation on cold-start problem. In: 2013 Third International Conference on Intelligent System Design and Engineering Applications (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michail-Angelos Papilaris
    • 1
  • Georgios Chalkiadakis
    • 1
    Email author
  1. 1.School of Electrical and Computer EngineeringTechnical University of CreteChaniaGreece

Personalised recommendations