Chapter

Advances in Information Retrieval

Volume 7814 of the series Lecture Notes in Computer Science pp 760-763

Optimizing nDCG Gains by Minimizing Effect of Label Inconsistency

  • Pavel MetrikovAffiliated withNortheastern University
  • , Virgil PavluAffiliated withNortheastern University
  • , Javed A. AslamAffiliated withNortheastern University

* Final gross prices may vary according to local VAT.

Get Access

Abstract

We focus on nDCG choice of gains, and in particular on the fracture between large differences in exponential gains of high relevance labels and the not-so-small confusion, or inconsistency, between these labels in data. We show that better gains can be derived from data by measuring the label inconsistency, to the point that virtually indistinguishable labels correspond to equal gains. Our derived optimal gains make a better nDCG objective for training Learning to Rank algorithms.