Skip to main content

Randomized experiments by government institutions and American political development

Public Choice volume 185, pages 401–413 (2020)Cite this article

Abstract

Are the methods of causal inference and, in particular, randomized controlled trials, compatible with the study of political history? While many important questions regarding political institutions and American political development cannot be answered with randomized controlled trials, scholars can and should be using the many instances of randomized experiments conducted by and within government institutions to further our understanding of institutions and political behavior. We argue that a surprising abundance of opportunities are available for scholars to utilize methods of random audits as natural experiments. Public and administrative officials have engaged in randomized interventions or audits to test for policy effects, to encourage compliance with the law, or to distribute government resources or personal risk to citizens fairly. With rare exceptions, such audits have not been leveraged by scholars interested in American political development or political history. Examples of randomized controlled trials conducted by agencies or institutions throughout US history are offered, and a historical random audit of members of the US Congress by the Federal Election Commission is highlighted. We conclude with limitations and advice on how to analyze the effects of randomized controlled trials conducted by governments. Scholars can use historical randomization to enhance causal inference and test theoretical implications, though deep knowledge of descriptive historical data and events are required to discover historical randomizations within political and legal institutions.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Notes

  1. Other causal inference methods have been utilized, but we emphasize randomization in our discussion that follows. Examples of other approaches include matching across subjects to estimate the effects of legal changes (e.g., Gordon and Huber 2007; Jensenius 2017; Mattes and Vonnahme 2010); regression discontinuity designs (e.g., Lerman 2014); difference-in-differences with plausibly exogenous institutional changes (O’Brien and Rickne 2016); within-subjects reforms of electoral institutions (e.g., Baskaran and Lopes Da Fonseca 2016; Carson and Sievert 2015; Yoshinaka and Grose 2008); and marginal structural modeling (Torres 2019); and synthetic controls (e.g. Michalski and Wood 2017).

  2. There is evidence that the IRS engages in political targeting (Young, Reksulak, and Shughart 2001), which raises questions as to whether IRS audits are random.

  3. Personal communication with a former FEC Commissioner.

References

  • Andersen, S. C., & Moynihan, D. P. (2016). Bureaucratic investments in expertise: Evidence from a randomized controlled field trial. Journal of Politics, 78(4), 1032–1044.

    Google Scholar 

  • Andersen, S. C., & Moynihan, D. P. (2018). How do socially distinctive newcomers fare? Evidence from a field experiment. Public Administration Review, 78(6), 874–882.

    Google Scholar 

  • Avellaneda, C. N. (2013). Mayoral decision-making: Issue salience, decision context, and choice constraint? An experimental study with 120 Latin American mayors. Journal of Public Administration Research and Theory, 23(3), 631–661.

    Google Scholar 

  • Ban, P., Fouirnaies, A., Hall, A. B., & Snyder, J. M. (2019). How newspapers reveal political power. Political Science Research and Methods. https://doi.org/10.1017/psrm.2017.43.

    Article  Google Scholar 

  • Baskaran, T., & Lopes da Fonseca, M. (2016). Electoral thresholds and political representation. Public Choice, 169(1–2), 117–136.

    Google Scholar 

  • Benston, G. J. (1973). Required disclosure and the stock market: An evaluation of the Securities Exchange Act of 1934. American Economic Review, 63(1), 132–155.

    Google Scholar 

  • Berinsky, A. J., & Chatfield, S. (2015). An empirical justification for the use of draft lottery numbers as a random treatment in political science research. Political Analysis, 23(3), 449–454.

    Google Scholar 

  • Broockman, D. E. (2014). Distorted communication, unequal representation: Constituents communicate less to representatives not of their race. American Journal of Political Science, 58(2), 307–321.

    Google Scholar 

  • Broockman, D. E., & Butler, D. M. (2015). Do better committee assignments meaningfully benefit legislators? Evidence from a randomized experiment in the Arkansas state legislature. Journal of Experimental Political Science, 2(2), 152–163.

    Google Scholar 

  • Butler, D. M. (2014). Representing the advantaged: How politicians reinforce inequality. New York: Cambridge University Press.

    Google Scholar 

  • Butler, D. M. (2019). Facilitating field experiments at the subnational level. Journal of Politics, 81(1), 371–376.

    Google Scholar 

  • Byrnes, P. E., Al-Awadhi, A., Gullvist, B., Brown-Liburd, H., Teeter, R., Warren, J. D., et al. (2012). Evolution of auditing: From the traditional approach to the future audit. In D. Y. Chan, V. Chiu, & M. A. Vasarhelyi (Eds.), Continuous auditing: Theory and application (pp. 285–297). Bingley, UK: Emerald Publishing Limited.

    Google Scholar 

  • Carson, J. L., & Sievert, J. (2015). Electoral reform and changes in legislative behavior: Adoption of the secret ballot in congressional elections. Legislative Studies Quarterly, 40(1), 83–110.

    Google Scholar 

  • Chin, M. L., Bond, J. R., & Geva, N. (2000). A foot in the door: An experimental study of PAC and constituency effects on access. Journal of Politics, 62(2), 534–549.

    Google Scholar 

  • Cirone, A., & Van Coppenolle, B. (2019). Bridging the gap: Lottery-based procedures in early parliamentarization. World Politics, 71(2), 197–235.

    Google Scholar 

  • Clark, W. R., & Golder, M. (2015). Big data, causal inference, and formal theory: Contradictory trends in political science? PS: Political Science and Politics, 48(1), 65–70.

    Google Scholar 

  • Costa, M. (2017). How responsive are political elites? A meta-analysis of experiments on public officials. Journal of Experimental Political Science, 4(3), 241–254.

    Google Scholar 

  • Diamond, J., & Robinson, J. A. (2010). Natural experiments of history. Cambridge, MA: The Belknap Press of Harvard University Press.

    Google Scholar 

  • Eber, L. (2005). Waiting for Watergate: The long road to FEC reform. Southern California Law Review, 79(5), 1155–1202.

    Google Scholar 

  • Eldersveld, S. J. (1956). Experimental propaganda techniques and voting behavior. American Political Science Review, 50(1), 154–165.

    Google Scholar 

  • Erikson, R. S., & Stoker, L. (2011). Caught in the draft: The effects of Vietnam draft lottery status on political attitudes. American Political Science Review, 105(2), 221–237.

    Google Scholar 

  • Fiorina, M. P., & Plott, C. R. (1978). Committee decisions under majority rule: An experimental study. American Political Science Review, 72(2), 575–598.

    Google Scholar 

  • Friedman, J. (1996). The rational choice controversy: Economic models of politics reconsidered. New Haven: Yale University Press.

    Google Scholar 

  • Gerber, A., & Green, D. P. (2000). The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review, 94(3), 653–663.

    Google Scholar 

  • Gordon, S. C., & Huber, G. A. (2007). The effect of electoral competitiveness on incumbent behavior. Quarterly Journal of Political Science, 2(2), 107–138.

    Google Scholar 

  • Gosnell, H. (1927). Getting-out-the-vote: An experiment in the stimulation of voting. Chicago: University of Chicago Press.

    Google Scholar 

  • Grimmer, J. (2015). We are all social scientists now: How big data, machine learning, and causal inference work together. PS: Political Science and Politics, 48(1), 80–83.

    Google Scholar 

  • Grose, C. R. (2011). Congress in black and white: Race and representation in Washington and at home. New York: Cambridge University Press.

    Google Scholar 

  • Grose, C. R. (2014a). Field experimental work on political institutions. Annual Review of Political Science, 17(1), 355–370.

    Google Scholar 

  • Grose, C. R. (2014b). The adoption of electoral reforms and ideological change in the California state legislature. USC Schwarzenegger Institute Report. http://www.schwarzeneggerinstitute.com/images/SI-Adoption%20of%20Electoral%20Reforms%20Report.pdf. Accessed 5 July 2019.

  • Grose, C. R. (2019). Experiments, political elites, and political institutions. In J. N. Druckman & D. P. Green (Eds.), Cambridge handbook of experiments. New York: Cambridge University Press.

    Google Scholar 

  • Grose, C. R., Malhotra, N., & Van Houweling, R. P. (2015). Explaining explanations: How legislators explain their positions and how citizens react. American Journal of Political Science, 59(3), 724–743.

    Google Scholar 

  • Hall, M. E. K. (2009). Experimental justice: Random judicial assignment and the partisan process of Supreme Court review. American Politics Research, 37(2), 195–226.

    Google Scholar 

  • Hall, M. E. K. (2010). Randomness reconsidered: Modeling random judicial assignment in the U.S. courts of appeals. Journal of Empirical Legal Studies, 7(3), 574–589.

    Google Scholar 

  • Hall, A. B., Huff, C., & Kuriwaki, S. (2019). Wealth, slaveownership, and fighting for the confederacy: An empirical study of the American civil war. American Political Science Review, 113(3), 658–673.

    Google Scholar 

  • Hartman, E., & Hidalgo, F. D. (2018). An equivalence approach to balance and placebo tests. American Journal of Political Science, 62(4), 1000–1013.

    Google Scholar 

  • Ho, D. E., Handen-Nader, C., Ames, D. & Marcus, D. (2018). Quality review of mass adjudication: A randomized natural experiment at the board of veterans appeals, 2003–2016. Working Paper, Stanford Institute for Economic Policy Research.

  • Hug, S. (2014). Further twenty years of pathologies? Is rational choice better than it used to be? Swiss Political Science Review, 20(3), 486–497.

    Google Scholar 

  • Jenkins, J. A. (2012). Studying Congress historically. In J. L. Carson (Ed.), New directions in congressional politics. Routledge/Taylor and Francis Group: New York and London.

    Google Scholar 

  • Jensenius, F. (2017). Social justice through inclusion: The consequences of electoral quotas in India. New York: Oxford University Press.

    Google Scholar 

  • Kalla, J. L., & Broockman, D. E. (2016). Campaign contributions facilitate access to congressional officials: a randomized field experiment. American Journal of Political Science, 60(3), 545–558.

    Google Scholar 

  • Kanthak, K., & Woon, J. (2015). Women don’t run? Election aversion and candidate entry. American Journal of Political Science, 59(3), 595–612.

    Google Scholar 

  • Karpowitz, C., Preece, J. R., & Monson, J. Q. (2017). How to elect more women: Gender and candidate success in a field experiment. American Journal of Political Science, 61(4), 927–943.

    Google Scholar 

  • Kastellec, J. P. (2011). Panel composition and voting on the U.S. courts of appeals over time. Political Research Quarterly, 64(2), 377–391.

    Google Scholar 

  • Kinder, D. R., & Palfrey, T. R. (1993). Experimental foundations of political science. Ann Arbor: University of Michigan Press.

    Google Scholar 

  • Kriner, D. L., & Schickler, E. (2014). Investigating the president: Committee probes and presidential approval, 1953-2006. Journal of Politics, 76(2), 521–534.

    Google Scholar 

  • Lajevardi, N. (2018). Access denied: Exploring Muslim American representation and exclusion by state legislators. Politics, Groups, and Identities. https://doi.org/10.1080/21565503.2018.1528161.

    Article  Google Scholar 

  • Larimer, C. (2018). Voter turnout field experiments. Oxford Bibliographies. https://www.oxfordbibliographies.com/view/document/obo-9780199756223/obo-9780199756223-0243.xml. Accessed 5 July 2019.

  • Lerman, A. (2014). The modern prison paradox: Politics, punishment, and social community. New York: Cambridge University Press.

    Google Scholar 

  • Levine, A. S. (2019). Research impact through matchmaking: How and why to connect researchers and practitioners. PS: Political Science and Politics. https://www.r4impact.org/sites/default/files/paper7_r4i_0.pdf. Accessed 5 July 2019.

  • Levine, D., Toffel, M. W., & Johnson, M. S. (2012). Randomized government safety inspections reduce worker injuries with no detectable job loss. Science, 336(6083), 907–911.

    Google Scholar 

  • Levy, M. K. (2017). Panel assignment in the federal courts of appeals. Cornell Law Review, 103(1), 65–116.

    Google Scholar 

  • Mattes, M., & Vonnahme, G. (2010). Contracting for peace: Do nonaggression pacts reduce conflict? Journal of Politics, 72(4), 925–938.

    Google Scholar 

  • Matthews, D. (2006). A history of auditing: The changing audit process in Britain from the nineteenth century to the present day. New York: Routledge.

    Google Scholar 

  • Mendez, M. S., & Grose, C. R. (2018). Doubling down: Inequality in responsiveness and the policy preferences of elected officials. Legislative Studies Quarterly, 43(3), 457–491.

    Google Scholar 

  • Merkel, A., & Vanberg, C. (2019). Legislative bargaining with costly communication. Public Choice. https://doi.org/10.1007/s11127-019-00682-8.

  • Michalski, R. M., & Wood, A. K. (2017). Twombly and Iqbal at the state level. Journal of Empirical Legal Studies, 14(2), 424–469.

    Google Scholar 

  • Morton, R. B., & Williams, K. C. (2010). Experimental political science and the study of causality. New York: Cambridge University Press.

    Google Scholar 

  • O’Brien, D. Z., & Rickne, J. (2016). Gender quotas and women’s political leadership. American Political Science Review, 110(1), 112–126.

    Google Scholar 

  • Orren, K., & Skowronek, S. (2016). Pathways to the present: Political development in America. In R. Valelly, S. Mettler, & R. Lieberman (Eds.), The Oxford handbook of American political development. New York: Oxford University Press.

    Google Scholar 

  • Palfrey, T. R. (2008). Laboratory experiments. In D. A. Wittman & B. R. Weingast (Eds.), The Oxford handbook of political economy. New York: Oxford University Press.

    Google Scholar 

  • Plott, C. R. (2014). Public choice and the development of modern laboratory experimental methods in economics and political science. Constitutional Political Economy, 25(4), 331–353.

    Google Scholar 

  • Poulos, J. (2019). Land lotteries, long-term wealth, and political selection. Public Choice, 178(1–2), 217–230.

    Google Scholar 

  • Preece, J. R., & Stoddard, O. B. (2015a). Why women don’t run: Experimental evidence on the role of competition aversion. Journal of Economic Behavior & Organization, 117(1), 296–308.

    Google Scholar 

  • Preece, J. R., & Stoddard, O. B. (2015b). Does the message matter? A field experiment on political party recruitment. Journal of Experimental Political Science, 2(1), 1–10.

    Google Scholar 

  • Riker, W. H. (1967). Bargaining in three person games. American Political Science Review, 61(3), 342–356.

    Google Scholar 

  • Riker, W. H., & Zavoina, W. J. (1970). Rational behavior in politics: Evidence from a three-person game. American Political Science Review, 64(1), 48–60.

    Google Scholar 

  • Rogowski, J. C., & Sinclair, B. (2017). Estimating the causal effects of social interactions with endogenous networks. Political Analysis, 20(3), 316–328.

    Google Scholar 

  • Rudolph, S. H. (2005). Perestroika and its other. In K. R. Monroe (Ed.), Perestroika! The raucous rebellion in political science. New Haven: Yale University Press.

    Google Scholar 

  • Sanders, E. (2005). Work that counts. In K. R. Monroe (Ed.), Perestroika! The raucous rebellion in political science. New Haven, CT: Yale University Press.

    Google Scholar 

  • Sen, M. (2017). How political signals affect public support for judicial nominations: Evidence from a conjoint experiment. Political Research Quarterly, 70(2), 374–393.

    Google Scholar 

  • Sunstein, C. R., Schkade, D., Ellman, L. M., & Sawicki, A. (2006). Are judges political? An empirical analysis of the federal judiciary. Washington, DC: Brookings Institution Press.

    Google Scholar 

  • Titiunik, R. (2016). Drawing your senator from a jar: Term length and legislative behavior. Political Science Research and Methods, 4(2), 293–316.

    Google Scholar 

  • Titiunik, R., & Feher, A. (2018). Legislative behaviour absent re-election incentives: Findings from a natural experiment in the Arkansas senate. Journal of the Royal Statistical Society, 181(2), 351–378.

    Google Scholar 

  • Torres, M. (2019). Estimating controlled direct effects through marginal structural models. Political Science Research and Methods. https://www.dropbox.com/s/872jdkg3dgo8ics/MSM_PSRM_2.pdf?dl=0. Accessed 5 July 2019.

  • Tóth, M., & Chytilek, R. (2018). Fast, frugal, and correct? An experimental study on the influence of time scarcity and quantity of information on the voter decision making process. Public Choice, 177(1–2), 67–86.

    Google Scholar 

  • Watts, R. L., & Zimmerman, J. L. (1983). Agency problems, auditing, and the theory of the firm: Some evidence. Journal of Law and Economics, 26(3), 613–633.

    Google Scholar 

  • Weiman, D. F. (1991). Peopling the land by lottery? The market in public lands and the regional differentiation of territory on the Georgia frontier. Journal of Economic History, 51(4), 835–860.

    Google Scholar 

  • White, A. R., Nathan, N. L., & Faller, J. K. (2015). What do I need to vote? Bureaucratic discretion and discrimination by local election officials. American Political Science Review, 109(1), 129–142.

    Google Scholar 

  • Wilson, R. K. (2011). The contribution of behavioral economics to political science. Annual Review of Political Science, 14(1), 201–223.

    Google Scholar 

  • Wood, A. K. (2018). Campaign finance disclosure. Annual Review of Law and Social Science, 14(1), 1–17.

    Google Scholar 

  • Wood, A. K., Elmendorf, C. S., & Spencer, D. (2019). Mind the (participation) gap: Vouchers, voting, and visibility. USC Law Legal Studies Paper No. 19-9; USC CLASS Research Paper No. CLASS 19-9, SSRN. https://ssrn.com/abstract=3354826. Accessed 5 July 2019.

  • Wood, A. K., & Grose, C. R. (2019). Campaign finance transparency affects legislators’ election outcomes and behaviors. SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3236939. Accessed 5 July 2019.

  • Wood, A. K., & Lewis, D. E. (2017). Agency performance challenges and agency politicization. Journal of Public Administration Research and Theory, 27(4), 581–595.

    Google Scholar 

  • Yoshinaka, A., & Grose, C. R. (2008). Ideological hedging in uncertain times: inconsistent legislative representation and voter enfranchisement. British Journal of Political Science, 41(4), 765–794.

    Google Scholar 

  • Young, M., Reksulak, M., & Shughart, W. F. (2001). The political economy of the IRS. Economics and Politics, 13(2), 201–220.

    Google Scholar 

  • Zvobgo, K. (2019). Human rights versus national interests: Shifting U.S. public attitudes on the international criminal court, International Studies Quarterly. https://docs.wixstatic.com/ugd/dc2664_541a78e9e6d24bd593be35bf5cb933d4.pdf. Accessed 5 July 2019.

Download references

Acknowledgements

We would like to thank Joel Sievert, Jeff Jenkins, and the participants in the Causal Inference and American Political Development conference for feedback on this manuscript.

Author information

Authors and Affiliations

  1. Political Science and Public Policy, University of Southern California, Los Angeles, USA

    Christian R. Grose

  2. Law, Political Science, and Public Policy, University of Southern California, Los Angeles, USA

    Abby K. Wood

Authors
  1. Christian R. Grose
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abby K. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian R. Grose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Grose, C.R., Wood, A.K. Randomized experiments by government institutions and American political development. Public Choice 185, 401–413 (2020). https://doi.org/10.1007/s11127-019-00704-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11127-019-00704-5

Keywords

  • Audits
  • Political institutions
  • Political history
  • Randomized controlled trials
  • Federal Election Commission
  • Congress
  • Natural experiments