A New Cross-Validation Technique to Evaluate Quality of Recommender Systems

  • Dmitry I. Ignatov
  • Jonas Poelmans
  • Guido Dedene
  • Stijn Viaene
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7143)


The topic of recommender systems is rapidly gaining interest in the user-behaviour modeling research domain. Over the years, various recommender algorithms based on different mathematical models have been introduced in the literature. Researchers interested in proposing a new recommender model or modifying an existing algorithm should take into account a variety of key performance indicators, such as execution time, recall and precision. Till date and to the best of our knowledge, no general cross-validation scheme to evaluate the performance of recommender algorithms has been developed. To fill this gap we propose an extension of conventional cross-validation. Besides splitting the initial data into training and test subsets, we also split the attribute description of the dataset into a hidden and visible part. We then discuss how such a splitting scheme can be applied in practice. Empirical validation is performed on traditional user-based and item-based recommender algorithms which were applied to the MovieLens dataset.


recommender systems quality of recommendations user-behavior modeling applied combinatorics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Deshpande, M., Karypis, G.: Item-based top-n recommendation algorithms. J. ACM Transactions on Information Systems 22(1), 143–177 (2004)CrossRefGoogle Scholar
  2. 2.
    Sarwar, B.M., Karypis, J., Konstan, J.A., Riedl, J.: Analysis of recommendation algorithms for e-commerce. In: ACM Conference on Electronic Commerce, pp. 158–167 (2001)Google Scholar
  3. 3.
    Goldberg, D., Nichols, D.A., Oki, B.M., Terry, D.B.: Using Collaborative Filtering to Weave an Information Tapestry. Communications of the ACM 35(12), 61–70 (1992)CrossRefGoogle Scholar
  4. 4.
    Ignatov, D.I., Kuznetsov, S.O.: Concept-based Recommendations for Internet Advertisement. In: Sixth International Conference on Concept Lattices and Their Applications, pp. 157–166. Palacky University, Olomouc (2008)Google Scholar
  5. 5.
    Ignatov, D.I., Kuznetsov, S.O.: Data Mining techniques for Internet Advertisement Recommender System. In: Proc. of 11th National Conference on Artificial Intelligence, Lenand, Moscow, vol. 2, pp. 34–42 (2008) (in Russian)Google Scholar
  6. 6.
    del Olmo, F.H., Gaudioso, E.: Evaluation of recommender systems: A new approach. J. Expert Systems with Applications 35, 790–804 (2008)CrossRefGoogle Scholar
  7. 7.
    Segaran, T.: Programming Collective Intelligence. O’Reilly Media, Sebastopol (2007)Google Scholar
  8. 8.
    Symeonidis, P., Nanopoulos, A., Papadopoulos, A.N., Manolopoulos, Y.: Nearest-biclusters collaborative filtering based on constant and coherent values. J. Information Retrieval 11, 51–75 (2007)CrossRefGoogle Scholar
  9. 9.
    Poelmans, J., Elzinga, P., Viaene, S., Dedene, G.: Curbing domestic violence: instantiating C-K theory with formal concept analysis and emergent self-organizing maps. Int. Syst. in Accounting, Finance and Management 17(3-4), 167–191 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dmitry I. Ignatov
    • 1
  • Jonas Poelmans
    • 2
  • Guido Dedene
    • 2
    • 4
  • Stijn Viaene
    • 2
    • 3
  1. 1.National Research University Higher School of EconomicsRussia
  2. 2.Katholieke Universiteit LeuvenBelgium
  3. 3.Vlerick Leuven Management SchoolBelgium
  4. 4.Amsterdam Business SchoolNetherlands

Personalised recommendations