Public Choice

, Volume 176, Issue 1–2, pp 79–106 | Cite as

Saying versus doing: a new donation method for measuring ideal points

  • Nicholas Haas
  • Rebecca B. MortonEmail author


Scaling methods pioneered by Poole and Rosenthal (Am J Polit Sci 29(2):357–384, 1985) redefined how scholars think about and estimate the ideologies of representatives seated in the US Congress. Those methods also have been used to estimate citizens’ ideologies. Whereas studies evaluating Congress typically use a behavioral measure, roll call votes, to estimate where representatives stand on the left-right ideological spectrum, those of the public most often have relied on survey data of stated, rather than revealed, preferences. However, measures of individuals’ preferences and, accordingly, estimates of their ideal points, may differ in important ways based on how preferences are elicited. In this paper, we elicit the same individuals’ preferences on the same 10 issues using two different methods: standard survey responses measured on a Likert scale and a donation exercise wherein individuals are forced to divide $1.50 between interest groups with diametrically opposed policy preferences. Importantly, expressing extreme views is costless under the former, but not the latter, method. We find that the type of elicitation method used is a significant predictor of individuals’ ideal points, and that the elicitation effect is driven primarily by Democratic respondents. Under the donation method, the ideal points of Democrats in the aggregate shift left, particularly for those Democrats who are politically engaged. In contrast, wealthy Democrats’ ideal points shift to the right. We also document effects for Republicans and Independents and find that overall polarization is similar under both elicitation methods. We conclude with a discussion of our results, and the consequences and tradeoffs of each elicitation method.


Survey methods Polarization Preference elicitation Ideal point estimation Bayesian estimation 

JEL Classification

C1 C83 D7 



We acknowledge the helpful research assistance of Arusyak Hakhnazaryan. We thank Douglas von Kohorn for helpful comments. The authors take credit for all errors. This research was approved by NYU’s Institutional Review Board under protocol IRB # 1304.

Supplementary material

11127_2018_558_MOESM1_ESM.pdf (181 kb)
Supplementary material 1 (pdf 180 KB)
11127_2018_558_MOESM2_ESM.pdf (149 kb)
Supplementary material 2 (pdf 148 KB)


  1. Aldrich, J. H., & McKelvey, R. D. (1977). A method of scaling with applications to the 1968 and 1972 presidential elections. American Political Science Review, 71(1), 111–130.CrossRefGoogle Scholar
  2. Alford, J., & Hibbing, J. R. (2007). Personal, interpersonal, and political temperaments. The ANNALS of the American Academy of Political and Social Science, 614, 196–212.CrossRefGoogle Scholar
  3. Anderson, L. R., Mellor, J. M., & Milyo, J. (2005). Do liberals play nice? The effects of party and political ideology in public goods and trust games. In J. Morgan (Ed.), Advances in Applied Microeconomics: Experimental and Behavioral Economics (pp. 107–131). Bingley: Emerald Group Publishing.Google Scholar
  4. Ansolabehere, S., Rodden, J., & Snyder, J. M. (2008). The strength of issues: Using multiple measures to gauge preference stability, ideological constraint, and issue voting. American Political Science Review, 102(2), 215–232.CrossRefGoogle Scholar
  5. Arechar, A. A., Gachter, S., & Molleman, L. (2017). Conducting interactive experiments online. Experimental Economics, 21(1), 99–131.CrossRefGoogle Scholar
  6. Armstrong, J. S., Denniston, W. B., & Gordon, M. M. (1975). The use of the decomposition principle in making judgments. Organizational Behavior and Human Performance, 14(2), 257–263.CrossRefGoogle Scholar
  7. Bafumi, J., & Herron, M. C. (2010). Leapfrog representation and extremism: A study of American voters and their members in Congress. American Political Science Review, 104(3), 519–542.CrossRefGoogle Scholar
  8. Barbera, P. (2015). Birds of the same feather tweet together. Bayesian ideal point estimation using Twitter data. Political Analysis, 23(1), 76–91.CrossRefGoogle Scholar
  9. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological), 57(1), 289–300.Google Scholar
  10. Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research:’s Mechanical Turk. Political Analysis, 20(3), 351–368.CrossRefGoogle Scholar
  11. Bodner, R., & Prelec, D. (2003). Self-signaling in a neo-Calvinist model of everyday decision making. In I. Brocas & J. Carrillo (Eds.), Psychology of economic decisions (pp. 105–126). London: Oxford University Press.Google Scholar
  12. Brooks, A. (2007). Who really cares. New York: Basic Books.Google Scholar
  13. Chandar, B., & Weyl, G. (2017). Quadratic voting in finite populations. Working Paper.Google Scholar
  14. Cliffiord, S., Jewell, R. M., & Waggoner, P. D. (2015). Are samples drawn from Mechanical Turk valid for research on political ideology? Research and Politics, 1–9.Google Scholar
  15. Coppock, A. (2018). Generalizing from survey experiments conducted on Mechanical Turk: A replication approach. Political Science Research and Methods, 1-16.Google Scholar
  16. Farc, M. M., & Sagarin, B. J. (2009). Using attitude strength to predict registration and voting behavior in the 2004 U.S. presidential elections. Basic and Applied Social Psychology, 31(2), 160–73.CrossRefGoogle Scholar
  17. Farwell, L., & Weiner, B. (2000). Bleeding hearts and the heartless: Popular perceptions of liberal and conservative ideologies. Personality and Social Psychology Bulletin, 26(7), 845–852.CrossRefGoogle Scholar
  18. Fiorina, M. P., Abrams, S. A., & Pope, J. C. (2008). Polarization in the American public: Misconceptions and misreadings. Journal of Politics, 70(2), 556–560.CrossRefGoogle Scholar
  19. Fisman, R., Jakiela, P., Kariv, S., & Markovits, D. (2015). The distributional preferences of an elite. Science, 349(6254), 1300–1307.CrossRefGoogle Scholar
  20. Fowler, J. H., & Kam, C. D. (2007). Beyond the self: Social identity, altruism, and political participation. The Journal of Politics, 69(3), 813–827.CrossRefGoogle Scholar
  21. Graham, J., Nosek, B. A., & Haidt, J. (2012). The moral stereotypes of liberals and conservatives: Exaggeration of differences across the political spectrum. PLoS ONE, 7(12), 1–13.CrossRefGoogle Scholar
  22. Haas, N., Hassan, M., Mansour, S., & Morton, R.B. (2017). Polarizing information and support for reform. Working Paper.Google Scholar
  23. Hare, C., Armstrong, D. A, I. I., Bakker, R., Carroll, R., & Poole, K. T. (2015). Using bayesian Aldrich-McKelvey scaling to study citizens’ Ideological preferences and perceptions. American Journal of Political Science, 59(3), 759–774.CrossRefGoogle Scholar
  24. Hauser, D. J., & Schwartz, N. (2016). Attentive turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407.CrossRefGoogle Scholar
  25. Hill, S. J., & Tausanovitch, C. (2015). A disconnect in representation? Comparison of trends in congressional and public polarization. Journal of Politics, 77(4), 1058–1075.CrossRefGoogle Scholar
  26. Holbrook, A. A., Sterrett, D., Johnson, T. P., & Krysan, M. (2016). Racial disparities in political participation across issues: The role of issue-specific motivators. Political Behavior, 38(1), 1–32.CrossRefGoogle Scholar
  27. Horton, J. J., Rand, D. G., & Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425.CrossRefGoogle Scholar
  28. Kristof, N. (2008). Bleeding heart tightwads. The New York Times.Google Scholar
  29. Krosnick, J. A. (1988). Attitude importance and attitude change. Journal of Experimental Social Psychology, 24, 240–255.CrossRefGoogle Scholar
  30. Krosnick, J.A., Presser, S., Fealing, K.H., & Ruggles, S. (2015). The future of survey research: Challenges and opportunities. A Report to the National Science Foundation Presented by The National Science Foundation Advisory Committee for the Social, Behavioral and Economic Sciences Subcommittee on Advancing SBE Survey Research. Google Scholar
  31. Lalley, S.P., & Weyl, G. (2016). Quadratic voting. Working Paper.Google Scholar
  32. Levay, K. E., Freese, J., & Druckman, J. N. (2016). The demographic and political composition of Mechanical Turk samples. SAGE Open, 6(1), 1–17.CrossRefGoogle Scholar
  33. Malhotra, N., Krosnick, J. A., & Thomas, R. K. (2009). Optimal design of branching questions to measure bipolar constructs. The Public Opinion Quarterly, 73(2), 304–324.CrossRefGoogle Scholar
  34. Miller, J. M., Krosnick, J. A., Holbrook, A., Tahk, A., & Dionne, L. (2016). The impact of policy change threat on financial contributions to interest groups. In J. A. Krosnick, I. C. Chiang, & T. Stark (Eds.), Explorations in Political Psychology. New York: Psychology Press.Google Scholar
  35. Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Freese, J. (2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2(2), 109–138.CrossRefGoogle Scholar
  36. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5),Google Scholar
  37. Poole, K. T., & Rosenthal, H. (1985). A spatial model for legislative roll call analysis. American Journal of Political Science, 29(2), 357–384.CrossRefGoogle Scholar
  38. Poole, K. T. (1998). Recovering a basic space from a set of issue scales. American Journal of Political Science, 42(3), 954–993.CrossRefGoogle Scholar
  39. Poole, K. T., & Rosenthal, H. (2000). Congress: A political-economic history of roll call voting. New York: Oxford University Press.Google Scholar
  40. Posner, E. A., & Weyl, G. (2015). Voting squared: Quadratic voting in democratic politics. Vanderbilt Law Review, 68(2), 441–500.Google Scholar
  41. Quarfoot, D., Kohorn, D. V., Slavin, K., Sutherland, R., Goldstein, D., & Konar, E. (2017). Quadratic voting in the wild: Real people, real votes. Public Choice, 172(1–2), 283–303.CrossRefGoogle Scholar
  42. Visser, P. S., Krosnick, J. A., & Simmons, J. P. (2003). Distinguishing the cognitive and behavioral consequences of attitude importance and certainty: A new approach to testing the common-factor hypothesis. Journal of Experimental Social Psychology, 39(2), 118–141.CrossRefGoogle Scholar
  43. Visser, P. S., Krosnick, J. A., & Norris, C. J. (2016). Attitude importance and attitude-relevant knowledge: Motivator and enabler. In J. A. Krosnick, I. C. Chiang, & T. Stark (Eds.), Explorations in Political Psychology. New York: Psychology Press.Google Scholar
  44. Treier, S., & Hillygus, D. S. (2009). The nature of political ideology in the contemporary electorate. The Public Opinion Quarterly, 73(4), 679–703.CrossRefGoogle Scholar
  45. Weyl, Glen. (2017). The robustness of quadratic voting. Public Choice, 172(1–2), 75–107.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of PoliticsNew York UniversityNew YorkUSA
  2. 2.Social Science Experimental LaboratoryNYU Abu DhabiAbu DhabiUnited Arab Emirates

Personalised recommendations