Advertisement

Performance Prediction for Quality Recommendations

  • Josephine GriffithEmail author
  • Colm O’Riordan
  • Humphrey Sorensen
Chapter
  • 669 Downloads
Part of the Intelligent Systems Reference Library book series (ISRL, volume 50)

Abstract

Work in the area of collaborative filtering continues to predominantly focus on prediction accuracy as a measure of the quality of the systems. Other measures of quality of these systems have been explored but not to the same extent. The work described in this chapter considers quality from the perspective of performance prediction. Per user, the performance of a collaborative filtering system is predicted based on rules learned by a machine learning approach. The experiments outlined aim, using three different datasets, to firstly learn the rules for performance prediction and to secondly test the accuracy of the rules produced. Results show good performance prediction accuracy can be found for all three datasets. The work does not step too far from the idea of prediction accuracy as a measure of quality but it does consider prediction accuracy from a different perspective, that of predicting the performance of a collaborative filtering system, per user, in advance of recommendation.

Keywords

Information Retrieval Recommender System Performance Prediction Test User Rating Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agichtein, E., Brill, E., Dumais, S.: Improving web search ranking by incorporating user behavior information. In: Proceedings of the 29th Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 19–26 (2006)Google Scholar
  2. 2.
    Amati, G., Carpineto, C., Romano, G.: Query difficulty, robustness, and selective application of query expansion. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 127–137. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Beenen, G., Ling, K., Wang, X., Chang, K., Frankowski, D., Resnick, P., Kraut, R.: Using social psychology to motivate contributions to online communities. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 212–221 (2004)Google Scholar
  4. 4.
    Bellogín, A., Castells, P.: Predicting neighbor goodness in collaborative filtering. In: Andreasen, T., Yager, R.R., Bulskov, H., Christiansen, H., Larsen, H.L. (eds.) FQAS 2009. LNCS, vol. 5822, pp. 605–616. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Bellogín, A., Castells, P.: A performance prediction approach to enhance collaborative filtering performance. In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., van Rijsbergen, K. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 382–393. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Breese, J., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann (1998)Google Scholar
  7. 7.
    Cheung, K., Tian, L.: Learning user similarity and rating style for collaborative recommendation. Information Retrieval 7, 395–410 (2004)CrossRefGoogle Scholar
  8. 8.
    Cremonesi, P., Garzotto, F., Negro, S., Papadopoulos, A., Turrin, R.: Comparative evaluation of recommender system quality. In: Proceedings of the 2011 International Conference on Human Factors in Computing Systems: Extended Abstracts, CHI EA 2011, pp. 1927–1932 (2011)Google Scholar
  9. 9.
    Cronen-Townsend, S., Zhou, Y., Croft, W.: Predicting query performance. In: 25th Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR (2002)Google Scholar
  10. 10.
    Cummins, R., Jose, J., O’Riordan, C.: Improved query performance prediction using standard deviation. In: Proceedings of the 34th Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR (2011)Google Scholar
  11. 11.
    De Bruyn, A., Lee Giles, C., Pennock, D.M.: Offering collaborative-like recommendations when data is sparse: The case of attraction-weighted information filtering. In: De Bra, P.M.E., Nejdl, W. (eds.) AH 2004. LNCS, vol. 3137, pp. 393–396. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Griffith, J., O’Riordan, C., Sorensen, H.: Using user model information to support collaborative filtering recommendations. In: Proceedings of the 18th Irish Conference on Artificial Intelligence and Cognitive Science, pp. 71–80 (2007)Google Scholar
  13. 13.
    He, B., Ounis, I.: Inferring query performance using pre-retrieval predictors. In: Apostolico, A., Melucci, M. (eds.) SPIRE 2004. LNCS, vol. 3246, pp. 43–54. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Herlocker, J., Konstan, J., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 230–237 (1999)Google Scholar
  15. 15.
    Herlocker, J., Konstan, J., Terveen, L., Riedl, J.: Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22(1), 5–53 (2004)CrossRefGoogle Scholar
  16. 16.
    Jin, R., Chai, J., Si, L.: An automatic weighting scheme for collaborative filtering. In: Proceedings of the 27th Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 337–344 (2004)Google Scholar
  17. 17.
    Karypis, G.: Evaluation of item-based top-n recommendation algorithms. In: Proceedings of the 10th International Conference on Information and Knowledge Management (2001)Google Scholar
  18. 18.
    Kleinberg, J.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Konstas, I., Stathopoulos, V., Jose, J.: On social networks and collaborative recommendation. In: Proceedings of the 32nd Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 195–202 (2009)Google Scholar
  20. 20.
    Koren, Y., Bell, R., Volinsksy, C.: Matrix factorization techniques for recommender systems. IEEE Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  21. 21.
    McNee, S., Lam, S., Konstan, J., Riedl, J.: Interfaces for eliciting new user preferences in recommender systems. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 178–187. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  22. 22.
    McNee, S., Riedl, J., Konstan, J.: Being accurate is not enough: how accuracy metrics have hurt recommender systems. In: CHI 2006 Extended Abstracts on Human Factors in Computing Systems, pp. 1097–1101 (2006)Google Scholar
  23. 23.
    O’Donovan, J., Smyth, B.: Trust in recommender systems. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 167–174 (2005)Google Scholar
  24. 24.
    Pérez-Iglesias, J., Araujo, L.: Standard deviation as a query hardness estimator. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 207–212. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  25. 25.
    Pu, P., Chen, L., Hu, R.: A user-centric evaluation framework for recommender systems. In: Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 2011, pp. 157–164 (2011)Google Scholar
  26. 26.
    Quinlan, J.: Learning with continuous classes. In: Artificial Intelligence, pp. 343–348. World Scientific (1992)Google Scholar
  27. 27.
    Rashid, A., Karypis, G., Riedl, J.: Influence in ratings-based recommender systems: An algorithm-independent approach. In: SIAM International Conference on Data Mining (2005)Google Scholar
  28. 28.
    Robertson, S.: Evaluation in information retrieval. In: Agosti, M., Crestani, F., Pasi, G. (eds.) ESSIR 2000. LNCS, vol. 1980, pp. 81–92. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  29. 29.
    Shardanand, U., Maes, P.: Social information filtering: Algorithms for automating word of mouth. In: Proceedings of the Annual ACM SIGCHI on Human Factors in Computing Systems (CHI 1995), pp. 210–217 (1995)Google Scholar
  30. 30.
    Swearingen, K., Sinha, R.: Beyond algorithms: An HCI perspective on recommender systems. In: SIGIR 2001 Workshop on Recommender Systems (2001)Google Scholar
  31. 31.
    Tintarev, N., Masthoff, J.: A survey of explanations in recommender systems. In: IEEE 23rd International Conference on Data Engineering: Workshop, pp. 801–810 (2007)Google Scholar
  32. 32.
    Wang, J., de Vries, A., Reinders, M.: Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: Proceedings of the 29th International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 501–508 (2006)Google Scholar
  33. 33.
    Weng, L.-T., Xu, Y., Li, Y., Nayak, R.: Improve recommendation quality with item taxonomic information. In: Filipe, J., Cordeiro, J. (eds.) ICEIS 2008. LNBIP, vol. 19, pp. 265–279. Springer, Heidelberg (2009)Google Scholar
  34. 34.
    Witten, I., Frank, E., Hall, M.: Data Mining Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann (2011)Google Scholar
  35. 35.
    Yom-Tov, E., Fine, S., Carmel, D., Darlow, A.: Learning to estimate query difficulty. In: 28th Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR (2005)Google Scholar
  36. 36.
    Yu, K., Xu, X., Tao, J., Kri, M., Kriegel, H.: Feature weighting and instance selection for collaborative filtering: An information-theoretic approach. Knowledge and Information Systems 5(2) (2003)Google Scholar
  37. 37.
    Zhou, Y., Croft, B.: Ranking robustness: a novel framework to predict query performance. In: 15th ACM conference on Information and Knowledge Management, CIKM (2006)Google Scholar
  38. 38.
    Zhu, X., Gauch, S.: Incorporating quality metrics in centralized/distributed information retrieval on the world wide web. In: Proceedings of the 23rd Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR, pp. 288–295 (2000)Google Scholar
  39. 39.
    Ziegler, C., McNee, S., Konstan, J., Lausen, G.: Improving recommendation lists through topic diversification. In: Proceedings of the 14th International Conference on World Wide Web, WWW 2005, pp. 22–32 (2005)Google Scholar

Copyright information

© Springer- Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Josephine Griffith
    • 1
    Email author
  • Colm O’Riordan
    • 1
  • Humphrey Sorensen
    • 2
  1. 1.College of Engineering and InformaticsNational University of IrelandGalwayIreland
  2. 2.Dept. of Computer ScienceUniversity College CorkCorkIreland

Personalised recommendations