Empirical Evaluation of Voting Rules with Strictly Ordered Preference Data

  • Nicholas Mattei
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6992)


The study of voting systems often takes place in the theoretical domain due to a lack of large samples of sincere, strictly ordered voting data. We derive several million elections (more than all the existing studies combined) from a publicly available data, the Netflix Prize dataset. The Netflix data is derived from millions of Netflix users, who have an incentive to report sincere preferences, unlike random survey takers. We evaluate each of these elections under the Plurality, Borda, k-Approval, and Repeated Alternative Vote (RAV) voting rules. We examine the Condorcet Efficiency of each of the rules and the probability of occurrence of Condorcet’s Paradox. We compare our votes to existing theories of domain restriction (e.g., single-peakedness) and statistical models used to generate election data for testing (e.g., Impartial Culture). We find a high consensus among the different voting rules; almost no instances of Condorcet’s Paradox; almost no support for restricted preference profiles, and very little support for many of the statistical models currently used to generate election data for testing.


Social Choice Vote Rule Condorcet Winner Strict Preference Election Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arrow, K., Sen, A., Suzumura, K. (eds.): Handbook of Social Choice and Welfare, vol. 1. North-Holland, Amsterdam (2002)zbMATHGoogle Scholar
  2. 2.
    Arrow, K.: Social choice and individual values. Yale Univ. Press, New Haven (1963)zbMATHGoogle Scholar
  3. 3.
    Bennett, J., Lanning, S.: The Netflix Prize. In: Proceedings of KDD Cup and Workshop (2007),
  4. 4.
    Berg, S.: Paradox of voting under an urn model: The effect of homogeneity. Public Choice 47(2), 377–387 (1985)CrossRefGoogle Scholar
  5. 5.
    Black, D.: On the rationale of group decision-making. The Journal of Political Economy 56(1) (1948)Google Scholar
  6. 6.
    Brandt, F., Brill, M., Hemaspaandra, E., Hemaspaandra, L.A.: Bypassing combinatorial protections: Polynomial-time algorithms for single-peaked electorates. In: Proc. of the 24th AAAI Conf. on Artificial Intelligence, pp. 715–722 (2010)Google Scholar
  7. 7.
    Chamberlin, J.R., Cohen, J.L., Coombs, C.H.: Social choice observed: Five presidential elections of the American Psychological Association. The Journal of Politics 46(2), 479–502 (1984)CrossRefGoogle Scholar
  8. 8.
    Condorcet, M.: Essay sur l’application de l’analyse de la probabilit des decisions: Redues et pluralit des voix, Paris (1785)Google Scholar
  9. 9.
    Conitzer, V., Sandholm, T.: Common voting rules as maximum likelihood estimators. In: Proc. of the 21st Annual Conf. on Uncertainty in AI (UAI), pp. 145–152 (2005)Google Scholar
  10. 10.
    Conitzer, V., Sandholm, T., Lang, J.: When are elections with few candidates hard to manipulate? Journal of the ACM 54(3), 1–33 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Faliszewski, P., Hemaspaandra, E., Hemaspaandra, L.A., Rothe, J.: A richer understanding of the complexity of election systems. In: Ravi, S., Shukla, S. (eds.) Fundamental Problems in Computing: Essays in Honor of Professor D.J. Rosenkrantz, pp. 375–406. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Felsenthal, D.S., Maoz, Z., Rapoport, A.: An empirical evaluation of six voting procedures: Do they really make any difference? British Journal of Political Science 23, 1–27 (1993)CrossRefGoogle Scholar
  13. 13.
    Gehrlein, W.V.: Condorcet’s paradox and the likelihood of its occurance: Different perspectives on balanced preferences. Theory and Decisions 52(2), 171–199 (2002)CrossRefzbMATHGoogle Scholar
  14. 14.
    Han, J., Kamber, M. (eds.): Data Mining. Morgan Kaufmann, San Francisco (2006)zbMATHGoogle Scholar
  15. 15.
    Merrill III, S.: A comparison of efficiency of multicandidate electoral systems. American Journal of Politial Science 28(1), 23–48 (1984)CrossRefGoogle Scholar
  16. 16.
    Niemi, R.G.: The occurrence of the paradox of voting in university elections. Public Choice 8(1), 91–100 (1970)CrossRefGoogle Scholar
  17. 17.
    Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V. (eds.): Algorithmic Game Theory. Cambridge Univ. Press, Cambridge (2007)Google Scholar
  18. 18.
    Nurmi, H.: Voting procedures: A summary analysis. British Journal of Political Science 13, 181–208 (1983)CrossRefGoogle Scholar
  19. 19.
    Regenwetter, M., Grogman, B., Marley, A.A.J., Testlin, I.M.: Behavioral Social Choice: Probabilistic Models, Statistical Inference, and Applications. Cambridge Univ. Press, Cambridge (2006)Google Scholar
  20. 20.
    Regenwetter, M., Kim, A., Kantor, A., Ho, M.R.: The unexpected empirical consensus among consensus methods. Psychological Science 18(7), 629–635 (2007)CrossRefGoogle Scholar
  21. 21.
    Rivest, R.L., Shen, E.: An optimal single-winner preferential voting system based on game theory. In: Conitzer, V., Rothe, J. (eds.) Proc. of the 3rd Intl. Workshop on Computational Social Choice (COMSOC), pp. 399–410 (2010)Google Scholar
  22. 22.
    Sen, A.K.: A possibility theorem on majority decisions. Econometrica 34(2), 491–499 (1966)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Tideman, N., Plassmann, F.: Modeling the outcomes of vote-casting in actual elections. To appear in Springer publshed book,
  24. 24.
    Walsh, T.: An empirical study of the manipulability of single transferable voting. In: Proc. of the 19th European Conf. on AI (ECAI 2010), pp. 257–262. IOS Press, Amsterdam (2010)Google Scholar
  25. 25.
    Walsh, T.: Where are the hard manipulation problems? In: Conitzer, V., Rothe, J. (eds.) Proc. of the 3rd Intl. Workshop on Computational Social Choice (COMSOC), pp. 9–11 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Nicholas Mattei
    • 1
  1. 1.Department of Computer ScienceUniversity of KentuckyLexingtonUSA

Personalised recommendations