Quadratic voting in the wild: real people, real votes

Abstract

Since their introduction in 1932, Likert and other continuous, independent rating scales have become the de facto toolset for survey research. Scholars have raised significant reliability and validity problems with these types of scales, and alternative methods for capturing perceptions and preferences have gained traction within specific domains. In this paper, we evaluate a new, broadly applicable approach to opinion measurement based on quadratic voting (QV), a method in which respondents express preferences by ‘buying’ votes for options using a fixed budget from which they pay quadratic prices for votes. Comparable QV-based and Likert-based survey instruments designed by Collective Decision Engines LLC were evaluated experimentally by assigning potential respondents randomly to one or the other method. Using a host of metrics, including respondent engagement and process-based metrics, we provide some initial evidence that the QV-based instrument provides a clearer measure of the preferences of the most intensely motivated respondents than the Likert-based instrument does. We consider the implications for survey satisficing, a key threat to the continued value of survey research, and discuss the mechanisms by which QV differentiates itself from Likert-based scales, thus establishing QV as a promising alternative survey tool for political and commercial research. We also explore key design issues within QV-based surveys to extend these promising results.