Abstract
While few would advocate that policy decisions be based solely on interest group influence or political pandering, few would also agree that decisions be based solely on evidence from randomized trials devoid of context or attention to stakeholder concerns. Yet, this is the implicit tension that has emerged between scholars, who privilege rigorously established research evidence as the primary legitimate basis for policy decision-making, and their critics, who advocate for a broader evidence boundary. However, the policy literature has hitherto failed to suggest an appropriate means of processing various forms of evidence to inform the policy decision-making process. This challenge is especially apparent in public hearings, a frequently used participatory medium where a great variety of evidence is presented. In this paper, we aim to reevaluate the value of public hearings as a means of collecting evidence by exploring 189 testimonies across six public hearings on single-payer healthcare in New York State. At the same time, we evaluate and categorize the types of evidence invoked in public hearings and compare this against what might “count” as evidence from an EBP perspective. Results highlight nine types of “evidence”, along two dimensions: observation span and form of knowledge. We find that applying a narrow boundary of research evidence, only one of nine types of evidence fit that classification: problem-based research. We conclude by suggesting that policy scholars expand their consideration of what types of evidence claims are useful to policymakers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Even though the result section primarily focuses on qualitatively describing the findings from this stage, we also processed and summarized the coding result using R.
References
Adams, D. (2004). Usable knowledge in public policy. Australian Journal of Public Administration, 63(1), 29–42. https://doi.org/10.1111/j.1467-8500.2004.00357.x
Almer, H. L., & Koontz, T. M. (2004). Public hearings for EIAs in post-communist Bulgaria: Do they work? Environmental Impact Assessment Review, 24(5), 473–493. https://doi.org/10.1016/j.eiar.2003.12.004
Baron, J. (2018). A brief history of evidence-based policy. The Annals of the American Academy of Political and Social Science, 678(1), 40–50. https://doi.org/10.1177/0002716218763128
Bawole, J. N. (2013). Public hearing or ‘hearing public’? An evaluation of the participation of local stakeholders in environmental impact assessment of Ghana’s Jubilee Oil Fields. Environmental Management, 52(2), 385–397. https://doi.org/10.1007/s00267-013-0086-9
Beesley, M., Cometto, G., & Pavignani, E. (2011). From drought to deluge: How information overload saturated absorption capacity in a disrupted health sector. Health Policy and Planning, 26(6), 445–452. https://doi.org/10.1093/heapol/czq083
Behague, D., Tawiah, C., Rosato, M., Some, T., & Morrison, J. (2009). Evidence-based policy-making: The implications of globally-applicable research for context-specific problem-solving in developing countries. Social Science and Medicine, 69(10), 1539–1546. https://doi.org/10.1016/j.socscimed.2009.08.006
Brownson, R. C., Chriqui, J. F., & Stamatakis, K. A. (2009). Understanding evidence-based public health policy. American Journal of Public Health, 99(9), 1576–1583. https://doi.org/10.2105/AJPH.2008.156224
Brownson, R. C., Green, L. W., & Fielding, J. E. (2018). Building capacity for evidence-based public health: Reconciling the pulls of practice and the push of research. Ssrn. https://doi.org/10.1146/annurev-publhealth-040617-014746
Cairney, P. (2016). The politics of evidence-based policy making. Palgrave Macmillan.
Charmaz, K. (2006). Grounded theory. Sage Publications Ltd.
Checkoway, B. (1981). The politics of public hearings. The Journal of Applied Behavioral Science, 17(4), 566–582. https://doi.org/10.1177/002188638101700411
Church, J. (2002). Citizen participation in health decision-making: Past experience and future prospects. Journal of Public Health Policy, 23(1), 12–32.
Contandriopoulos, D. (2004). A sociological perspective on public participation in health care. Social Science and Medicine, 58(2), 321–330. https://doi.org/10.1016/S0277-9536(03)00164-3
Davies, H. T. O., Nutley, S. M., & Smith, P. C. (2000). What works? Evidence-based policy and practice in public services (p. 117). The Policy Press.
Eising, R., & Spohr, F. (2017). The more, the merrier? Interest groups and legislative change in the public hearings of the German Parliamentary Committees. German Politics, 26(2), 314–333. https://doi.org/10.1080/09644008.2016.1213244
Epstein, D., Farina, C., & Heidt, J. (2014). The value of words: Narrative as evidence in policy making. Evidence and Policy, 10(2), 243–258. https://doi.org/10.1332/174426514X13990325021128
Fox, A. M., & Choi, Y. (2019). Political economy of reform under US Federalism: Adopting single-payer health coverage in New York State. Health Systems & Reform, 5(3), 209–223. https://doi.org/10.1080/23288604.2019.1635414
Fox, A. M., & Poirier, R. (2018). How single-payer stacks up: Evaluating different models of universal health coverage on cost, access, and quality. International Journal of Health Services, 48(3), 568–585. https://doi.org/10.1177/0020731418779377
Fung, A. (2006). Varieties of participation in complex governance. Public Administration Review, 66, 66–75. https://doi.org/10.1111/j.1540-6210.2006.00667.x
Geddes, M. (2018). Committee hearings of the UK parliament: Who gives evidence and does this matter? Parliamentary Affairs, 71(2), 283–304. https://doi.org/10.1093/pa/gsx026
Geyman, J. P. (2005). Myths and memes about single-payer health insurance in the United States: A rebuttal to conservative claims. International Journal of Health Services, 35(1), 63–90. https://doi.org/10.2190/XK59-V3CC-1F4N-1C4X
Glied, S. (2009). Single payer as a financing mechanism. Journal of Health Politics, Policy and Law, 34(4), 593–615. https://doi.org/10.1215/03616878-2009-017
Gray, V., Lowery, D., Monogan, J., & Godwin, E. K. (2010). Incrementing toward nowhere: Universal health care coverage in the states. Publius, 40(1), 82–113. https://doi.org/10.1093/publius/pjp023
Green, J. (2000). The role of theory in evidence-based health promotion practice. Health Education Research, 15(2), 125–129. https://doi.org/10.1093/her/15.2.125
Guyatt, G. H., & Busse, J. W. (2006). The philosophy of evidence-based medicine. In V. M. Montori (Ed.), Evidence-based endocrinology (pp. 25–33). Humana Press.
Halpin, D., Baxter, G., & MacLeod, I. (2012). Multiple arenas, multiple populations: Counting organized interests in Scottish Public Policy. In D. Halpin & G. Jordan (Eds.), The scale of interest organization in democratic politics (pp. 118–140). Palgrave Macmillan. https://doi.org/10.1057/9780230359239_6
Hammersley, M. (2013). The myth of research-based policymaking and practice. In The myth of research-based policy & practice. Sage Publications Ltd. https://doi.org/10.4135/9781473957626.n3
Hansen, H. F. (2014). Organisation of evidence-based knowledge production: Evidence hierarchies and evidence typologies. Scandinavian Journal of Public Health, 42(13, Suppl), 11–17.
Head, B. W. (2008a). Three lenses of evidence-based policy. Australian Journal of Public Administration, 67(1), 1–11. https://doi.org/10.1111/j.1467-8500.2007.00564.x
Head, B. W. (2008b). Wicked problems in public policy. Public Policy, 3(2), 101–101.
Holmes, D., Murray, S. J., Perron, A., & Rail, G. (2006). Deconstructing the evidence-based discourse in health sciences: Truth, power and fascism. International Journal of Evidence-Based Healthcare, 4(3), 180–186. https://doi.org/10.1111/j.1479-6988.2006.00041.x
Hunter, D. J. (2009). Relationship between evidence and policy: A case of evidence-based policy or policy-based evidence? Public Health, 123(9), 583–586. https://doi.org/10.1016/j.puhe.2009.07.011
Hunter, D. J. (2016). Evidence-informed policy: In praise of politics and political science. Public Health Panorama, 02(03), 268–272.
Jasanoff, S. (2003). Technologies of humility: Citizen participation in governing science. Minerva, 41(3), 223–244. https://doi.org/10.1023/A:1025557512320
Johnston, R., Pattie, C., & Rossiter, D. (2013). Local inquiries or public hearings: Changes in public consultation over the redistribution of UK parliamentary constituency boundaries. Public Administration. https://doi.org/10.1111/padm.12020
King, C. S., Feltey, K. M., & Susel, B. O. (1998). The question of participation: Authentic public participation in public administration. Public Administration Review, 58(4), 317–326.
Kislov, R., Wilson, P., Cummings, G., Ehrenberg, A., Gifford, W., Kelly, J., Kitson, A., Pettersson, L., Wallin, L., & Harvey, G. (2019). From research evidence to “evidence by proxy”? Organizational enactment of evidence-based health care in four high-income countries. Public Administration Review, 79(5), 684–698. https://doi.org/10.1111/puar.13056
Kothari, A., Rudman, D., Dobbins, M., Rouse, M., Sibbald, S., & Edwards, N. (2010). The use of tacit and explicit knowledge in public health. In Proceedings of the European conference on knowledge management, ECKM (pp. 574–582).
Lando, T. (2003). The public hearing process: A tool for citizen participation, or a path toward citizen alienation? National Civic Review, 92(1), 73–73. https://doi.org/10.1002/ncr.7
Mosley, J. E., & Gibson, K. (2017). Strategic use of evidence in state-level policymaking: Matching evidence type to legislative stage. Policy Sciences, 50(4), 697–719. https://doi.org/10.1007/s11077-017-9289-x
New York State Assembly Standing Committee on Health. (2014a, December 4). Public hearing on the New York health bill to create single payer health coverage—Syracuse.
New York State Assembly Standing Committee on Health. (2014b, December 10). Public hearing on the New York health bill to create single payer health coverage—Buffalo.
New York State Assembly Standing Committee on Health. (2014c, December 16). Public hearing on the New York health bill to create single payer health coverage—New York City.
New York State Assembly Standing Committee on Health. (2015, January 13). Public hearing on the New York health bill to create single payer health coverage—Albany.
Newman, J. (2017). Deconstructing the debate over evidence-based policy. Critical Policy Studies, 11(2), 211–226. https://doi.org/10.1080/19460171.2016.1224724
Nutley, S., Powell, A., & Davies, H. (2013). What counts as good evidence? (pp. 1–40). http://www.alliance4usefulevidence.org/assets/What-Counts-as-Good-Evidence-WEB.pdf
Oliver, K., & Pearce, W. (2017). Three lessons from evidence-based medicine and policy: Increase transparency, balance inputs and understand power. Palgrave Communications, 3(1), 43. https://doi.org/10.1057/s41599-017-0045-9
Parkhurst, J. (2017). The politics of evidence: From evidence-based policy to the good governance of evidence (p. 182). Routledge.
Parry, R. (2020). Episteme and techne. In Zalta, E. N. (Ed.), The Stanford encyclopedia of philosophy (Fall 2020 Edition).
Pearce, W., & Raman, S. (2014). The new randomised controlled trials (RCT) movement in public policy: Challenges of epistemic governance. Policy Sciences, 47(4), 387–402. https://doi.org/10.1007/s11077-014-9208-3
Petticrew, M., & Roberts, H. (2003). Evidence, hierarchies, and typologies: Horses for courses. Journal of Epidemiology and Community Health, 57(7), 527–529. https://doi.org/10.1136/jech.57.7.527
Pew-MacArthur Results First Initiative. (2017). How states engage in evidence-based policymaking: A national assessment.
Plager, S. (1969). Participatory democracy and the public hearing: A functional approach. 21 Administrative Law Review 153.
Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. https://doi.org/10.1007/BF01405730
Rowe, G., & Frewer, L. J. (2000). Public participation methods: A framework for evaluation. Science Technology and Human Values, 25(1), 3–29. https://doi.org/10.1177/016224390002500101
Sainath, N. V., & Rajan, K. S. (2015). Meta-analysis of EIA public hearings in the State of Gujarat, India: Its role versus the goal of environmental management. Impact Assessment and Project Appraisal, 33(2), 148–153. https://doi.org/10.1080/14615517.2014.964085
Sanderson, I. (2002). Evaluation, policy learning and evidence-based policy making. Public Administration, 80(1), 1–22.
Smith, K. B., & Larimer, C. W. (2009). The public policy theory primer (1st ed.). Westview Press.
Smith-Walter, A., Peterson, H. L., Jones, M. D., & Nicole Reynolds Marshall, A. (2016). Gun stories: How evidence shapes firearm policy in the United States. Politics and Policy, 44(6), 1053–1088. https://doi.org/10.1111/polp.12187
Sur, R., & Dahm, P. (2011). History of evidence-based medicine. Indian Journal of Urology, 27(4), 487. https://doi.org/10.4103/0970-1591.91438
The Grade Working Group. (2004). Grading quality of evidence and strength of recommendations. BMJ, 328(7454), 1–8. https://doi.org/10.1136/bmj.328.7454.1490
Thomas, P. N. (2017). Contentious actions and communication for social change: The public hearing (Jan Sunwai) as process. Journal of Communication, 67(5), 719–732. https://doi.org/10.1111/jcom.12301
Toulmin, S. E. (2003). The uses of argument. Cambridge University Press. http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=120780
Walgrave, S., & Dejaeghere, Y. (2017). Surviving information overload: How elite politicians select information. Governance, 30(2), 229–244. https://doi.org/10.1111/gove.12209
Wesselink, A., Colebatch, H., & Pearce, W. (2014). Evidence and policy: Discourses, meanings and practices. Policy Sciences, 47(4), 339–344. https://doi.org/10.1007/s11077-014-9209-2
Whittle, A., & Mueller, F. (2012). Bankers in the dock: Moral storytelling in action. Human Relations, 65(1), 111–139. https://doi.org/10.1177/0018726711423442
Acknowledgements
This work was initiated as a term paper for Mitchel Y. Abolafia's Cultural Analysis of Organization course in the Spring 2019 and was developed for doctoral candidacy in the Department of Public Administration and Policy at University at Albany. We thank Erika G. Martin for her extensive efforts to review this paper several times and provide insightful comments. We thank the anonymous referees for their thorough reviews and constructive comments. Also, helpful comments from Mitchel Y. Abolafia, Luis Felipe Luna-Reyes, R. Karl Rethemeyer, Younhee Kim, Michael Ahn, Jinwoo Lee, Pirmin Bundi, Soohyun Park are gratefully acknowledged. This work was supported by the Department of Public Administration and Policy at University at Albany and the Public Management Research Association (PMRA) via travel grants to attend academic conferences. The views and errors are all the authors’ responsibility.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1: Coding example
Appendix 2: Coding scheme
Types | Operational definitions | Basic forms |
|---|---|---|
Personal opinions | A way of thinking** of an individual* about something | “Society should not ~ ” |
Public opinions | Statements with plural forms indicating a common pattern of positions** found across people on the issue** | “Many physicians favor single-payer ~ ” |
Opinion polls | Numbers, figures, and estimates based on a specific data source* indicating people’s positions on the issue** | “Gallup poll shows that ~ ” |
Anecdotes | Individual experiences** in which a single individual is identified* | “One of our students, shared a disturbing experience that ~ ” |
Practices | Ways of doing something** of people, systems, organizations, or countries* | “Patients/other countries do ~ ” “Every ER physician knows that ~ ” |
Indicators | Numbers, figures, and estimates based on data* indicating physical and behavioral patterns** | “Premiums increased by 00%” “Forty-five thousand lives are lost every year in the U.S. ~ ” |
Judgments | Presentations of a theory or a law** explaining the reason or cause of a single subject* | “By the letter of the law, any person can and will receive emergency care ~ ” |
Professional principles or practices | Patterns and mechanisms found across individuals or incidents* based on theoretical or legal principles** | “Millions of years of evolution had led us mammals to have stomachs which produce acid ~ ” |
Problem-based research | Systematic analyses or investigations* conducted by professionals** | “There are some researchers that used information ~ ” |
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Choi, Y., Fox, A.M. & Dodge, J. What counts? Policy evidence in public hearing testimonies: the case of single-payer healthcare in New York State. Policy Sci 55, 631–660 (2022). https://doi.org/10.1007/s11077-022-09475-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11077-022-09475-1