Skip to main content
Log in

A probabilistic model to resolve diversity–accuracy challenge of recommendation systems

Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Recommendation systems have wide-spread applications in both academia and industry. Traditionally, performance of recommendation systems has been measured by their precision. By introducing novelty and diversity as key qualities in recommender systems, recently increasing attention has been focused on this topic. Precision and novelty of recommendation are not in the same direction, and practical systems should make a trade-off between these two quantities. Thus, it is an important feature of a recommender system to make it possible to adjust diversity and accuracy of the recommendations by tuning the model. In this paper, we introduce a probabilistic structure to resolve the diversity–accuracy dilemma in recommender systems. We propose a hybrid model with adjustable level of diversity and precision such that one can perform this by tuning a single parameter. The proposed recommendation model consists of two models: one for maximization of the accuracy and the other one for specification of the recommendation list to tastes of users. Our experiments on two real datasets show the functionality of the model in resolving accuracy–diversity dilemma and outperformance of the model over other classic models. The proposed method could be extensively applied to real commercial systems due to its low computational complexity and significant performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Adomavicius G, Kwon Y (2012) Improving aggregate recommendation diversity using ranking-based techniques. IEEE Trans Knowl Data Eng 24:896–911

    Article  Google Scholar 

  2. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Audio Electroacoust Knowl Data Eng 17:734–749

    Article  Google Scholar 

  3. Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. In: Web search and data mining. ACM, Barcelona, Spain, 5–14

  4. Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 43–52

  5. Burke R (2002) Hybrid recommender systems: survey and experiments. User Model User-Adapt Interact 12:331–370

    Article  MATH  Google Scholar 

  6. Celma Ò, Herrera P (2008) A new approach to evaluating novel recommendations. In: Recommendation systems. ACM, Lausanne, Switzerland, pp 179–186

  7. Deshpande M, Karypis G (2004a) Item-based top-N recommendation algorithms. ACM Trans Inf Syst 22:143–177

  8. Deshpande M, Karypis G (2004b) Selective Markov models for predicting web page accesses. ACM Trans Internet Technol (TOIT) 4:163–184

    Article  Google Scholar 

  9. Garcin F, Dimitrakakis C, Faltings B (2013) Personalized news recommendation with context trees. arXiv preprint arXiv:1303.0665

  10. He Q, Jiang D, Liao Z, Hoi S, Chang K, Lim E-P, Li H (2009) Web query recommendation via sequential query prediction. In: IEEE 25th international conference on data engineering, 2009. ICDE’09. IEEE, pp 1443–1454

  11. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst (TOIS) 22:5–53

    Article  Google Scholar 

  12. Hofmann T (2004) Latent semantic models for collaborative filtering. ACM Trans Inf Syst (TOIS) 22:89–115

  13. Ben Schafer J, Frankowski D, Herlocker J, Sen S (2007) Collaborative filtering recommender systems. In: LNCS. Lecture Notes in Computer Science, vol 4321, pp 291–324

  14. Javari A, Gharibshah J, Jalili M (2014) Recommender systems based on collaborative filtering and resource allocation. Soc Netw Anal Min (to appear)

  15. Javari A, Jalili M (2014) Accurate and novel recommendations: an algorithm based on popularity forecasting. ACM Trans Intell Syst Technol (to appear)

  16. Konstan J, Miller B, Maltz D, Herlocker J, Gordon L, Riedl J (1997) GroupLens: applying collaborative filtering in Usenet news. Commun ACM 40:77–87

    Article  Google Scholar 

  17. Maslov S, Zhang Y-C (2001) Extracting hidden information from knowledge networks. Phys Rev Lett 87:248701

    Article  Google Scholar 

  18. McNee SM, Riedl J, Konstan JA (2006) Being accurate is not enough: how accuracy metrics have hurt recommender systems. In: CHI’06 extended abstracts on human factors in computing systems. ACM, pp 1097–1101

  19. Mobasher B (2007) Data mining for web personalization. In: The adaptive web. Springer, pp 90–135

  20. Mobasher B, Dai H, Luo T, Nakagawa M, (2002) Using sequential and non-sequential patterns in predictive web usage mining tasks. In: 2002 IEEE International Conference on data mining, 2002. ICDM, 2003. Proceedings. IEEE, pp 669–672

  21. Montgomery AL, Li S, Srinivasan K, Liechty JC (2004) Modeling online browsing and path analysis using clickstream data. Market Sci 23:579–595

    Article  Google Scholar 

  22. Nikulin V (2011) OpenStudy: recommendations of the following ten lectures after viewing a set of three given lectures. In: Proceedings of ECML-PKDD 2011 discovery challenge workshop, pp 59–69

  23. Pazzani M, Billsus D (2007) Content-based recommendation systems. In: The adaptive web, pp 325–341

  24. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World Wide Web. ACM, pp 811–820

  25. Resnick P, Varian HR (1997) Recommender systems. Commun ACM 40:56–58

    Article  Google Scholar 

  26. Sarukkai RR (2000) Link prediction and path analysis using Markov chains. Comput Netw 33:377–386

    Article  Google Scholar 

  27. Sarwar B, Karypis G, Konstan J, Riedl J (2000) Application of dimensionality reduction in recommender system—a case study. DTIC Document

  28. Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, 2001. ACM

  29. Shani G, Brafman RI, Heckerman D (2002) An MDP-based recommender system. In: Proceedings of the eighteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 453–460

  30. Shani G, Gunawardana A (2011) Evaluating recommendation systems. In: Recommender systems handbook. Springer, pp 257–297

  31. Shani G, Heckerman D, Brafman RI (2006) An MDP-based recommender system. J Mach Learn Res 6:1265

    MathSciNet  Google Scholar 

  32. Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv Artif Intell 2009:2

  33. Vargas S, Castells P (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: Recommendation systems. ACM, Chicago, IL, USA, pp 109–116

  34. Vargas S, Castells P (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: Proceedings of the fifth ACM conference on recommender systems. ACM, pp 109–116

  35. Yu C, Lakshmanan L, Amer-Yahia S (2009a) It takes variety to make a world: diversification in recommender systems. In: Proceedings of the 12th international conference on extending database technology: advances in database technology. ACM, pp 368–378

  36. Yu C, Lakshmanan LVS, Amer-Yahia S (2009b) Recommendation diversification using explanations. In: Proceedings of the IEEE international conference on data engineering, 2009. IEEE, pp 1299–1302

  37. Yu H, Riedl MO (2012) A sequential recommendation approach for interactive personalized story generation. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems, volume 1 international foundation for autonomous agents and multiagent systems, pp 71–78

  38. Zhang Z-K, Zhou T, Zhang Y-C (2010) Personalized recommendation via integrated diffusion on user-item-tag tripartite graphs. Phys A 389:179–186

    Article  Google Scholar 

  39. Zhou T, Kuscsik Z, Liu J-G, Medo M, Wakeling JR, Zhang Y-C (2010) Solving the apparent diversity–accuracy dilemma of recommender systems. Proc Natl Acad Sci USA 107:4511–4515

    Article  Google Scholar 

  40. Ziegle C-N, Mcnee SM, Konstan JA, Lausen G (2005) Improving recommendation lists through topic diversification. In: World Wide Web. ACM, Chiba, Japan, pp 22–32

  41. Zimdars A, Chickering DM, Meek C (2001) Using temporal data for making recommendations. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, pp 580–588

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mahdi Jalili.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Javari, A., Jalili, M. A probabilistic model to resolve diversity–accuracy challenge of recommendation systems. Knowl Inf Syst 44, 609–627 (2015). https://doi.org/10.1007/s10115-014-0779-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-014-0779-2

Keywords

Navigation