Machine Learning

, Volume 90, Issue 1, pp 1-28

First online:

Open Access This content is freely available online to anyone, anywhere at any time.

Efficiently learning the preferences of people

  • Adriana BirlutiuAffiliated withInstitute for Computing and Information Sciences, Radboud University NijmegenRomanian Institute of Science and Technology Email author 
  • , Perry GrootAffiliated withInstitute for Computing and Information Sciences, Radboud University NijmegenDept. of Electrical Engineering, Control Systems, Technical University Eindhoven
  • , Tom HeskesAffiliated withInstitute for Computing and Information Sciences, Radboud University Nijmegen


This paper presents a framework for optimizing the preference learning process. In many real-world applications in which preference learning is involved the available training data is scarce and obtaining labeled training data is expensive. Fortunately in many of the preference learning situations data is available from multiple subjects. We use the multi-task formalism to enhance the individual training data by making use of the preference information learned from other subjects. Furthermore, since obtaining labels is expensive, we optimally choose which data to ask a subject for labelling to obtain the most of information about her/his preferences. This paradigm—called active learning—has hardly been studied in a multi-task formalism. We propose an alternative for the standard criteria in active learning which actively chooses queries by making use of the available preference data from other subjects. The advantage of this alternative is the reduced computation costs and reduced time subjects are involved. We validate empirically our approach on three real-world data sets involving the preferences of people.


Learning preferences Active learning Experimental design Multi-task learning Hierarchical modeling