Knowledge and Information Systems

, Volume 44, Issue 3, pp 609–627 | Cite as

A probabilistic model to resolve diversity–accuracy challenge of recommendation systems

  • Amin Javari
  • Mahdi JaliliEmail author
Regular Paper


Recommendation systems have wide-spread applications in both academia and industry. Traditionally, performance of recommendation systems has been measured by their precision. By introducing novelty and diversity as key qualities in recommender systems, recently increasing attention has been focused on this topic. Precision and novelty of recommendation are not in the same direction, and practical systems should make a trade-off between these two quantities. Thus, it is an important feature of a recommender system to make it possible to adjust diversity and accuracy of the recommendations by tuning the model. In this paper, we introduce a probabilistic structure to resolve the diversity–accuracy dilemma in recommender systems. We propose a hybrid model with adjustable level of diversity and precision such that one can perform this by tuning a single parameter. The proposed recommendation model consists of two models: one for maximization of the accuracy and the other one for specification of the recommendation list to tastes of users. Our experiments on two real datasets show the functionality of the model in resolving accuracy–diversity dilemma and outperformance of the model over other classic models. The proposed method could be extensively applied to real commercial systems due to its low computational complexity and significant performance.


Social networks analysis and mining Markov chain  Maximum likelihood estimation Collaborative filtering Diversity 


  1. 1.
    Adomavicius G, Kwon Y (2012) Improving aggregate recommendation diversity using ranking-based techniques. IEEE Trans Knowl Data Eng 24:896–911CrossRefGoogle Scholar
  2. 2.
    Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Audio Electroacoust Knowl Data Eng 17:734–749CrossRefGoogle Scholar
  3. 3.
    Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. In: Web search and data mining. ACM, Barcelona, Spain, 5–14Google Scholar
  4. 4.
    Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 43–52Google Scholar
  5. 5.
    Burke R (2002) Hybrid recommender systems: survey and experiments. User Model User-Adapt Interact 12:331–370CrossRefzbMATHGoogle Scholar
  6. 6.
    Celma Ò, Herrera P (2008) A new approach to evaluating novel recommendations. In: Recommendation systems. ACM, Lausanne, Switzerland, pp 179–186Google Scholar
  7. 7.
    Deshpande M, Karypis G (2004a) Item-based top-N recommendation algorithms. ACM Trans Inf Syst 22:143–177Google Scholar
  8. 8.
    Deshpande M, Karypis G (2004b) Selective Markov models for predicting web page accesses. ACM Trans Internet Technol (TOIT) 4:163–184CrossRefGoogle Scholar
  9. 9.
    Garcin F, Dimitrakakis C, Faltings B (2013) Personalized news recommendation with context trees. arXiv preprint arXiv:1303.0665
  10. 10.
    He Q, Jiang D, Liao Z, Hoi S, Chang K, Lim E-P, Li H (2009) Web query recommendation via sequential query prediction. In: IEEE 25th international conference on data engineering, 2009. ICDE’09. IEEE, pp 1443–1454Google Scholar
  11. 11.
    Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst (TOIS) 22:5–53CrossRefGoogle Scholar
  12. 12.
    Hofmann T (2004) Latent semantic models for collaborative filtering. ACM Trans Inf Syst (TOIS) 22:89–115Google Scholar
  13. 13.
    Ben Schafer J, Frankowski D, Herlocker J, Sen S (2007) Collaborative filtering recommender systems. In: LNCS. Lecture Notes in Computer Science, vol 4321, pp 291–324Google Scholar
  14. 14.
    Javari A, Gharibshah J, Jalili M (2014) Recommender systems based on collaborative filtering and resource allocation. Soc Netw Anal Min (to appear)Google Scholar
  15. 15.
    Javari A, Jalili M (2014) Accurate and novel recommendations: an algorithm based on popularity forecasting. ACM Trans Intell Syst Technol (to appear)Google Scholar
  16. 16.
    Konstan J, Miller B, Maltz D, Herlocker J, Gordon L, Riedl J (1997) GroupLens: applying collaborative filtering in Usenet news. Commun ACM 40:77–87CrossRefGoogle Scholar
  17. 17.
    Maslov S, Zhang Y-C (2001) Extracting hidden information from knowledge networks. Phys Rev Lett 87:248701CrossRefGoogle Scholar
  18. 18.
    McNee SM, Riedl J, Konstan JA (2006) Being accurate is not enough: how accuracy metrics have hurt recommender systems. In: CHI’06 extended abstracts on human factors in computing systems. ACM, pp 1097–1101Google Scholar
  19. 19.
    Mobasher B (2007) Data mining for web personalization. In: The adaptive web. Springer, pp 90–135Google Scholar
  20. 20.
    Mobasher B, Dai H, Luo T, Nakagawa M, (2002) Using sequential and non-sequential patterns in predictive web usage mining tasks. In: 2002 IEEE International Conference on data mining, 2002. ICDM, 2003. Proceedings. IEEE, pp 669–672Google Scholar
  21. 21.
    Montgomery AL, Li S, Srinivasan K, Liechty JC (2004) Modeling online browsing and path analysis using clickstream data. Market Sci 23:579–595CrossRefGoogle Scholar
  22. 22.
    Nikulin V (2011) OpenStudy: recommendations of the following ten lectures after viewing a set of three given lectures. In: Proceedings of ECML-PKDD 2011 discovery challenge workshop, pp 59–69Google Scholar
  23. 23.
    Pazzani M, Billsus D (2007) Content-based recommendation systems. In: The adaptive web, pp 325–341Google Scholar
  24. 24.
    Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World Wide Web. ACM, pp 811–820Google Scholar
  25. 25.
    Resnick P, Varian HR (1997) Recommender systems. Commun ACM 40:56–58CrossRefGoogle Scholar
  26. 26.
    Sarukkai RR (2000) Link prediction and path analysis using Markov chains. Comput Netw 33:377–386CrossRefGoogle Scholar
  27. 27.
    Sarwar B, Karypis G, Konstan J, Riedl J (2000) Application of dimensionality reduction in recommender system—a case study. DTIC DocumentGoogle Scholar
  28. 28.
    Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, 2001. ACMGoogle Scholar
  29. 29.
    Shani G, Brafman RI, Heckerman D (2002) An MDP-based recommender system. In: Proceedings of the eighteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 453–460Google Scholar
  30. 30.
    Shani G, Gunawardana A (2011) Evaluating recommendation systems. In: Recommender systems handbook. Springer, pp 257–297Google Scholar
  31. 31.
    Shani G, Heckerman D, Brafman RI (2006) An MDP-based recommender system. J Mach Learn Res 6:1265MathSciNetGoogle Scholar
  32. 32.
    Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv Artif Intell 2009:2Google Scholar
  33. 33.
    Vargas S, Castells P (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: Recommendation systems. ACM, Chicago, IL, USA, pp 109–116Google Scholar
  34. 34.
    Vargas S, Castells P (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: Proceedings of the fifth ACM conference on recommender systems. ACM, pp 109–116Google Scholar
  35. 35.
    Yu C, Lakshmanan L, Amer-Yahia S (2009a) It takes variety to make a world: diversification in recommender systems. In: Proceedings of the 12th international conference on extending database technology: advances in database technology. ACM, pp 368–378Google Scholar
  36. 36.
    Yu C, Lakshmanan LVS, Amer-Yahia S (2009b) Recommendation diversification using explanations. In: Proceedings of the IEEE international conference on data engineering, 2009. IEEE, pp 1299–1302Google Scholar
  37. 37.
    Yu H, Riedl MO (2012) A sequential recommendation approach for interactive personalized story generation. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems, volume 1 international foundation for autonomous agents and multiagent systems, pp 71–78Google Scholar
  38. 38.
    Zhang Z-K, Zhou T, Zhang Y-C (2010) Personalized recommendation via integrated diffusion on user-item-tag tripartite graphs. Phys A 389:179–186CrossRefGoogle Scholar
  39. 39.
    Zhou T, Kuscsik Z, Liu J-G, Medo M, Wakeling JR, Zhang Y-C (2010) Solving the apparent diversity–accuracy dilemma of recommender systems. Proc Natl Acad Sci USA 107:4511–4515CrossRefGoogle Scholar
  40. 40.
    Ziegle C-N, Mcnee SM, Konstan JA, Lausen G (2005) Improving recommendation lists through topic diversification. In: World Wide Web. ACM, Chiba, Japan, pp 22–32Google Scholar
  41. 41.
    Zimdars A, Chickering DM, Meek C (2001) Using temporal data for making recommendations. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 580–588Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  1. 1.Department of Computer EngineeringSharif University of TechnologyTehranIran
  2. 2.School of Electrical and Computer Engineering, RMIT UniversityMelbourneAustralia

Personalised recommendations