Chapter

Recommendation Systems in Software Engineering

pp 245-273

Date:

Dimensions and Metrics for Evaluating Recommendation Systems

  • Iman AvazpourAffiliated withFaculty of ICT, Centre for Computing and Engineering Software and Systems (SUCCESS), Swinburne University of Technology Email author 
  • , Teerat PitakratAffiliated withInstitute of Software Technology, Universität Stuttgart Email author 
  • , Lars GrunskeAffiliated withInstitute of Software Technology, Universität Stuttgart Email author 
  • , John GrundyAffiliated withFaculty of ICT, Centre for Computing and Engineering Software and Systems (SUCCESS), Swinburne University of Technology Email author 

Abstract

Recommendation systems support users and developers of various computer and software systems to overcome information overload, perform information discovery tasks, and approximate computation, among others. They have recently become popular and have attracted a wide variety of application scenarios ranging from business process modeling to source code manipulation. Due to this wide variety of application domains, different approaches and metrics have been adopted for their evaluation. In this chapter, we review a range of evaluation metrics and measures as well as some approaches used for evaluating recommendation systems. The metrics presented in this chapter are grouped under sixteen different dimensions, e.g., correctness, novelty, coverage. We review these metrics according to the dimensions to which they correspond. A brief overview of approaches to comprehensive evaluation using collections of recommendation system dimensions and associated metrics is presented. We also provide suggestions for key future research and practice directions.