Prediction with Confidence in Item Based Collaborative Filtering

  • Tadiparthi V. R. Himabindu
  • Vineet PadmanabhanEmail author
  • Arun K. Pujari
  • Abdul Sattar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9810)


Recommender systems can be viewed as prediction systems where we can predict the ratings which represent users’ interest in the corresponding item. Typically, items having the highest predicted ratings will be recommended to the users. But users do not know how certain these predictions are. Therefore, it is important to associate a confidence measure to the predictions which tells users how certain the system is in making the predictions. Many different approaches have been proposed to estimate confidence of predictions made by recommender systems. But none of them provide guarantee on the error rate of these predictions. Conformal Prediction is a framework that produces predictions with a guaranteed error rate. In this paper, we propose a conformal prediction algorithm with item-based collaborative filtering as the underlying algorithm which is a simple and widely used algorithm in commercial applications. We propose different nonconformity measures and empirically determine the best nonconformity measure. We empirically prove validity and efficiency of proposed algorithm. Experimental results demonstrate that the predictive performance of conformal prediction algorithm is very close to its underlying algorithm with little uncertainty along with the measures of confidence and credibility.


Recommender systems Conformal prediction Confidence Nonconformity measure 


  1. 1.
    Adomavicius, G., Kamireddy, S., Kwon, Y.: Towards more confident recommendations:improving recommender systems using filtering approach based onrating variance. In: Proceedings of (WITS 2007) (2007)Google Scholar
  2. 2.
    Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of WWW, pp. 285–295 (2001)Google Scholar
  3. 3.
    Mazurowski, M.A.: Estimating confidence of individual rating predictions in collaborative filtering recommender systems. Expert Syst. Appl. 40(10), 3847–3857 (2013)CrossRefGoogle Scholar
  4. 4.
    Shafer, G., Vovk, V.: A tutorial on conformal prediction. J. Mach. Learn. Res. 9, 371–421 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005)zbMATHGoogle Scholar
  6. 6.
    Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.J.: Transductive confidence machines for pattern recognition. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, p. 381. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    McNee, S.M., Lam, S.K., Guetzlaff, C., Konstan, J.A., Riedl, J.: Confidence displays and training in recommender systems. In: International Conference on Human-Computer Interaction (2003)Google Scholar
  8. 8.
    McLaughlin, M.R., Herlocker, J.L.: A collaborative filtering algorithm and evaluation metric that accurately model the user experience. SIGIR 2004, 329–336 (2004)Google Scholar
  9. 9.
    Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, New York (2011)CrossRefGoogle Scholar
  10. 10.
    Koren, Y., Sill, J.: Ordrec: an ordinal model for predicting personalized item rating distributions. In: RecSys, pp. 117–124 (2011)Google Scholar
  11. 11.
    Vovk, V., Fedorova, V., Nouretdinov, I., Gammerman, A.: Criteria of Efficiency for Conformal Prediction. Technical report (2014)Google Scholar
  12. 12.
    Linden, G., Smith, B., York, J.: recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003)CrossRefGoogle Scholar
  13. 13.
    Lemire, D., Maclachlan, A.: Slope One Predictors for Online Rating-Based Collaborative Filtering. CoRR abs/cs/0702144 (2007)Google Scholar
  14. 14.
    Hill, W., Stead, L., Rosenstein, M., Furnas, G.W.: Recommending and evaluating choices in a virtual community of use. In: Proceedings of ACM CHI95 Conference on Human Factors in Computing Systems, pp. 194–201 (1995)Google Scholar
  15. 15.
    Gunawardana, A., Shani, G.: A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 2935–2962 (2009)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Johansson, U., Bostrom, H., Lofstrom, T.: Conformal prediction using decisiontrees. In: IEEE 13th International Conference on Data Mining, pp. 330–339 (2013)Google Scholar
  17. 17.
    Koren, Y., Bell, R.M., Volinsky, C.: Matrix factorization techniques for recommender systems. IEEE Comput. 42(8), 30–37 (2009)CrossRefGoogle Scholar
  18. 18.
    Saunders, C., Gammerman, A., Vovk, V.: Transduction with confidence and credibility. In: Proceedings of IJCAI 1999, pp. 722–726 (1999)Google Scholar
  19. 19.
    Johansson, U., Bostrom, H., Lofstrom, T.: Conformal prediction using decision trees. In: IEEE 13th International Conference on Data Mining, pp. 330–339 (2013)Google Scholar
  20. 20.
    Papadopoulos, H., Vovk, V., Gammerman, A.: Conformal prediction with neural networks. In: 19th IEEE ICTAI 2007, pp. 388–395 (2007)Google Scholar
  21. 21.
    Breese, J. S., Heckerman, D., and Kadie, C. Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 14th Conference on UAI 1998, pp. 43–52 (1998)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Tadiparthi V. R. Himabindu
    • 1
  • Vineet Padmanabhan
    • 1
    Email author
  • Arun K. Pujari
    • 1
  • Abdul Sattar
    • 2
  1. 1.School of Computer and Information SciencesUniversity of HyderabadHyderabadIndia
  2. 2.Griffith UniversityBrisbaneAustralia

Personalised recommendations