Skip to main content
SpringerLink
Log in

Using Quali-Quantitative Indicators for Assessing the Quality of Citizen Participation: A Study on Three Citizen Juries

  • Published:
Social Indicators Research Aims and scope Submit manuscript

Abstract

Over the last 3 decades, citizen involvement has become rather common in policymaking processes. Its rationale, as well as its potential benefits and limitations, are manifold. The literature on the evaluation of public participation is copious and it is crucial both to implement effective processes, and to achieve high-quality outcomes. Inspired by deliberative democracy theory, dialogue/fairness and knowledge/competence have been considered the two main criteria to assess the quality of deliberative processes. Based on the analysis of three citizen juries, the paper focuses on the process through which citizen deliberation occurs. Specifically, three properties related to dialogue, i.e., equity, cooperation, and cognitive openness, were treated as quality indicators of the deliberative process. Both quantitative and qualitative methods were used, and three sources of data utilized: (a) semi-structured interviews to jurors; (b) post-jury questionnaires; and (c) jurors’ conversational turns. Altogether, the analyses showed that despite the imbalance in participation, the deliberation process was perceived as fair. However, findings also suggested that the participatory setting did not promote the ability of participants to generate new collective knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The analysis went through the following steps: First the research group familiarized with the gathered data, reading and re-reading the transcribed interviews and noting down initial ideas. Then initial codes were generated aimed at capturing interesting features across the entire data set. Successively, themes were searched, and codes collated into potential themes; themes were then reviewed to check if they worked in relation to the entire data set, and finally they were defined and named through an ongoing analysis.

  2. J1 = Jury 1, J2 = Jury 2, J3 = Jury 3; the following number = Interview number; M = Male, F = Female; code(s) in square brackets.

References

  • Abdel-Monem, T., Bingham, S., Marincic, J., & Tomkins, A. (2010). Deliberation and diversity: Perceptions of small group discussions by race and ethnicity. Small Group Research, 41, 746–776.

    Article  Google Scholar 

  • Abelson, J., Forest, P. G., Eyles, J., Smith, P., Martin, E., & Gauvin, F. P. (2003). Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science and Medicine, 57(2), 239–251.

    Article  Google Scholar 

  • Asch, S. E. (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychological Monographs General and Applied, 70(9), 1–70.

    Article  Google Scholar 

  • Beauvais, E., & Baechtiger, A. (2016). Taking the goals of deliberation seriously: a differentiated view on equality and equity in deliberative designs and processes. Journal of Public Deliberation, 12(2), 1–18. http://www.publicdeliberation.net/jpd/vol12/iss2/art2/.

  • Beierle, T. C., & Cayford, J. (2002). Democracy in practice: Public participation in environmental decisions. Washington, D.C.: Resources for the Future.

    Google Scholar 

  • Beste, S. (2013). Contemporary trends of deliberative research: Synthesizing a new study agenda. Journal of Public Deliberation, 9(2). http://www.publicdeliberation.net/jpd/vol9/iss2/art1.

  • Bobbio, L. (2013). La qualità della deliberazione [The quality of deliberation]. Roma: Carocci.

    Google Scholar 

  • Botes, L., & van Rensburg, D. (2000). Community participation in development: Nine plagues and twelve commandments. Community Development Journal, 1, 41–58.

    Article  Google Scholar 

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.

    Article  Google Scholar 

  • Buchy, M., & Hoverman, S. (2000). Understanding public participation: A review. Forest Policy and Economics, 1, 15–25.

    Article  Google Scholar 

  • Burton, P. (2003). Community involvement in neighbourhood regeneration: Stairway to heaven or road to nowhere?. Paper n. 13, ESRC Centre for Neighbourhood Research.

  • Canary, D. J., & Spitzberg, B. H. (1987). Appropriateness and effectiveness perceptions of conflict strategies. Human Communication Research, 14, 93–120.

    Article  Google Scholar 

  • Carnes, S. A., Schweitzer, M., Peelle, E. B., Wolfe, A. K., & Munro, J. F. (1998). Measuring the success of public participation on environmental restoration and waste management activities in the US Department of Energy. Technology in Society, 20(4), 385–406.

    Article  Google Scholar 

  • Chambers, S. (2003). Deliberative democratic theory. Annual Review of Political Science, 6, 307–326.

    Article  Google Scholar 

  • Charnley, S., & Engelbert, B. (2005). Evaluating public participation in environmental decision-making: EPA’s superfund community involvement program. Journal of Environmental Management, 77, 165–182.

    Article  Google Scholar 

  • Chavis, D. M., & Wandersman, A. (1990). Sense of community in the urban environment: A catalyst for participation and community development. American Journal of Community Psychology, 18, 55–81.

    Article  Google Scholar 

  • Chess, C., & Purcell, K. (1999). Public participation and the environment: Do we know what works? Environmental Science and Technology, 33(16), 2685–2692.

    Article  Google Scholar 

  • Coelho, V. S. R. P., & Waisbich, L. (2016). Participatory mechanisms and inequality reduction: searching for plausible relations. Journal of Public Deliberation, 12(2). http://www.publicdeliberation.net/jpd/vol12/iss2/art13.

  • Cohen, J. (1996). Procedure and substance in deliberative democracy. In S. Benhabib (Ed.), Democracy and difference: Contesting the boundaries of the political (pp. 95–119). Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Cohen, J. (1997). Deliberation and democratic legitimacy. In J. F. Bohman & W. Rehg (Eds.), Deliberative democracy: Essays on reason and politics (pp. 67–91). Cambridge, MA: MIT Press.

    Google Scholar 

  • Crosby, N. (1995). Citizens juries: One solution for difficult environmental questions. In O. Renn, T. Webler, & P. Wiedemann (Eds.), Fairness and competence in citizen participation (pp. 157–174). London: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  • de Castro-Silva, C. R., & Cavichioli, S. (2013). La participación politíca en una ONG/SIDA y las posibilidades de emancipación en un contexto de exclusión social. Revista Interamericana de Psicología/Interamerican Journal of Psychology, 47, 9–16.

    Google Scholar 

  • De Vries, R., Stanczyk, A., Wall, I. F., Uhlmann, R., Damschroder, L. J., & Kim, S. Y. (2010). Assessing the quality of democratic deliberation: A case study of public deliberation on the ethics of surrogate consent for research. Social Science and Medicine, 70(12), 1896–1903.

    Article  Google Scholar 

  • Delli Carpini, M. X., Cook, F. L., & Jacobs, L. R. (2004). Public deliberation, discursive participation, and citizen engagement: A review of the empirical literature. Annual Review of Political Science, 7, 315–344.

    Article  Google Scholar 

  • Dunkerley, D., & Glasner, P. (1998). Empowering the public? Citizens juries and the new genetic technologies. Critical Public Health, 8, 181–192.

    Article  Google Scholar 

  • Edwards, P., Hindmarsh, R., Merer, H., Bond, M., & Rowland, A. (2008). A three-stage evaluation of a deliberative event on climate change and transforming energy. Journal of Public Deliberation, 4(1). http://services.bepress.com/jpd/vol4/iss1/art6.

  • Font, J., & Blanco, I. (2007). Procedural legitimacy and political trust: The case of citizen juries in Spain. European Journal of Political Research, 46, 557–589.

    Article  Google Scholar 

  • Gastil, J., & Black, L. (2008). Public deliberation as the organizing principle of political communication research. Journal of Public Deliberation, 4(1). http://services.bepress.com/jpd/vol4/iss1/art3/.

  • Goodin, R. E., & Dryzek, J. S. (2006). Deliberative impacts: The macro-political uptake of mini-publics. Politics and Society, 34, 219–244.

    Article  Google Scholar 

  • Grice, P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and semantics: Speech acts (Vol. 3, pp. 41–58). New York: Academic Press.

    Google Scholar 

  • Habermas, J. (1984). The theory of communicative action. Reason and the rationalization of society. Boston: Beacon Press.

    Google Scholar 

  • Habermas, J. (1989). The structural transformation of the public sphere. Cambridge, MA: MIT Press.

    Google Scholar 

  • Habermas, J. (1996). Between facts and norms: Contributions to a discourse theory of law and democracy. Cambridge, MA: MIT Press.

    Google Scholar 

  • Huntington, S. P. (1991). Democracy’s third wave. The Journal of Democracy, 2, 12–34.

    Article  Google Scholar 

  • Jackson, D. N. (1984). Personality research form. Manual. Port Huron, MI: Research Psychologists Press.

    Google Scholar 

  • Janis, I. L. (1982). Groupthink: Psychological studies of policy decisions and fiascos (2nd ed.). New York, NY: Houghton Mifflin.

    Google Scholar 

  • Kadlec, A., & Friedman, W. (2007). Deliberative democracy and the problem of power. Journal of Public Deliberation, 3(1). http://www.publicdeliberation.net/jpd/vol3/iss1/art8.

  • Karpowitz, C. F., & Mansbridge, J. (2005). Disagreement and consensus: The need for dynamic updating in public deliberation. Journal of Public deliberation, 1(1), 348–364. http://www.publicdeliberation.net/jpd/vol1/iss1/art2/.

  • Keyes, C. L. M. (1998). Social well-being. Social Psychological Quarterly, 61(2), 121–140.

    Article  Google Scholar 

  • King, R. C., Hartzel, K. S., Schilhavy, R. A., Melone, N. P., & McGuire, T. W. (2010). Social responsibility and stakeholder influence: Does technology matter during stakeholder deliberation with high-impact decisions? Decision Support Systems, 48(4), 536–547.

    Article  Google Scholar 

  • Klijn, E. H., & Koppenjan, J. F. M. (2000). Politicians and interactive decision making: Institutional spoilsports or playmakers. Public Administration, 2, 365–387.

    Article  Google Scholar 

  • Korfmacher, K. S. (2001). The politics of participation in watershed modeling. Environment Management, 27, 161–176.

    Article  Google Scholar 

  • Kroon, M., van Kreveld, D., & Rabbie, J. (1992). Group versus individual decision making. Effects of accountability and gender on groupthink. Small Group Research, 4, 427–458.

    Article  Google Scholar 

  • Kruglanski, A., Raviv, A., Bar-Tal, D., Raviv, A., Sharvit, K., Ellis, S., et al. (2005). Says who?: Epistemic authority effects in social judgment. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 37, pp. 345–392). New York: Academic Press.

    Google Scholar 

  • Kruglanski, A., & Webster, D. (1996). Motivated closing of mind: “Seizing” and “freezing”. Psychological Review, 2, 263–283.

    Article  Google Scholar 

  • Ledwith, M., & Springett, J. (2010). Participatory practice. Bristol, UK: Policy Press.

    Google Scholar 

  • Linnel, P., & Luckman, T. (1991). Asymmetries in dialogue: Some conceptual preliminaries. In I. Markova & K. Foppa (Eds.), Asymmetries in dialogue (pp. 1–20). Hemel Hempstead: Harvester Wheatsheaf.

    Google Scholar 

  • Linnell, P., Gustavsson, L., & Juvonen, P. (1988). Interactional dominance in dyadic communication: A presentation of initiative-response analysis. Linguistics, 26, 415–442.

    Google Scholar 

  • Mannarini, T. (2011). Public involvement and competent communities: Towards a social psychology of public participation. International Journal of Humanities and Social Sciences, 1(7), 66–72.

    Google Scholar 

  • Mannarini, T. (2014). Riding paradox: Lessons learned from Italian participatory policy-making experiences. Interamerican Journal of Psychology, 48, 71–81.

    Google Scholar 

  • Mendelberg, T. (2002). The deliberative citizen: Theory and evidence. In M. X. Delli Carpini, L. Huddy, & R. Y. Shapiro (Eds.), Political decision-making, deliberation and participation. Research in Micropolitics (Vol. 6, pp. 151–194). Amsterdam Boston: Emerald Group Publishing.

    Google Scholar 

  • Mendelberg, T. (2006). Small group deliberation. Paper presented at the Annual Meeting of the American Political Science Association, Philadelphia, PA.

  • Montero, M. (2004). Introducción a la Psicología comunitaria. Buenos Aires: Paidos.

    Google Scholar 

  • Moscovici, S., & Doise, W. (1991). Dissensus et consensus. Une theéorie générale des decisions collectives. Paris: Presses Universitaires de France.

    Google Scholar 

  • Moscovici, S., & Zavalloni, M. (1969). The group as a polarizer of attitudes. Journal of Personality and Social Psychology, 12(2), 125–135.

    Article  Google Scholar 

  • Moscrop, D. R., & Warren, M. E. (2016). When is Deliberation democratic?. Journal of Public Deliberation, 12(2). http://www.publicdeliberation.net/jpd/vol12/iss2/art4.

  • Nemeth, C. J. (1986). The differential contributions of majority and minority influence. Psychological Review, 93, 23–32.

    Article  Google Scholar 

  • Noelle-Neuman, E. (1984). The spiral of silence. A theory of public opinion. Chicago: University of Chicago Press.

    Google Scholar 

  • Nussbaum, M. (1999). Sex & social justice. New York: Oxford University Press.

    Google Scholar 

  • Papadopulos, Y., & Warin, P. (2007). Are innovative, participatory and deliberative procedures in policy-making democratic and effective? European Journal of Political Research, 46, 445–472.

    Article  Google Scholar 

  • Prestby, J., Wandersman, A., Florin, P., Rich, R., & Chavis, D. M. (1990). Benefits, costs, incentives management and participation in voluntary associations. American Journal of Community Psychology, 18, 117–149.

    Article  Google Scholar 

  • Putnam, R. D. (2000). Bowling alone: The collapse and revival of american community. New York: Simon & Schuster.

    Book  Google Scholar 

  • Radcliff, B., & Shufeldt, G. (2016). Direct democracy and subjective well-being: The initiative and life satisfaction in the American States. Social Indicators Research, 128(3), 1405–1423.

    Article  Google Scholar 

  • Rondinella, T., Segre, E., & Zola, D. (2017). Participative processes for measuring progress: Deliberation, consultation and the role of civil society. Social Indicators Research, 130(3), 959–982.

    Article  Google Scholar 

  • Rowe, G., & Frewer, L. J. (2000). Public participation methods: A framework for evaluation. Science, Technology and Human Values, 25(1), 3–29.

    Article  Google Scholar 

  • Rowe, G., & Frewer, L. J. (2004). Evaluating public-participation exercises: A research agenda. Science, Technology and Human Values, 29(4), 512–557.

    Article  Google Scholar 

  • Rowe, G., Marsh, R., & Frewer, L. (2004). Evaluation of a deliberative conference in science. Technology and Human Values, 29(1), 88–121.

    Article  Google Scholar 

  • Ryfe, D. M. (2005). Does deliberative democracy work? Annual Review of Political Science, 8(1), 49–71.

    Article  Google Scholar 

  • Sanders, L. (1997). Against deliberation. Political Theory, 25(3), 347–376.

    Article  Google Scholar 

  • Steiner, J., Steenbergen, M. R., Bachtiger, A., & Spörndli, M. (2003). Measuring political deliberation: A discourse quality index. Comparative European Politics, 1, 21–48.

    Article  Google Scholar 

  • Stephens, J. B., & Berner, M. (2011). Learning from your neighbor: The value of public participation evaluation for public policy dispute resolution. Journal of Public Deliberation, 7(1), art. 10, http://services.bepress.com/jpd/vol7/iss1/art10.

  • Stoner, J. A. F. (1968). Risky and cautious shifts in group decisions: The influence of widely held values. Journal of Experimental Social Psychology, 4(4), 442–459.

    Article  Google Scholar 

  • Street, J., Duszynski, K., Krawczyk, S., & Braunack-Mayer, A. (2013). Citizens’ juries in health policy decision-making: A systematic review. Social Science and Medicine, 109, 1–9.

    Article  Google Scholar 

  • Stromer-Galley, J. (2007). Measuring deliberation’s content: A coding scheme. Journal of Public Deliberation, 3(1). http://services.bepress.com/jpd/vol3/iss1/art12.

  • Sunstein, C. (2000). Deliberative Trouble? Why groups go to extremes. The Yale Law Journal, 1, 71–119.

    Article  Google Scholar 

  • Sunstein, C. (2002). The law of group polarization. The Journal of Political Philosophy, 2, 175–195.

    Article  Google Scholar 

  • Sunstein, C. (2005). Group judgments: Statistical means, deliberation and information markets. New York University Law Review, 80, 962–1049.

    Google Scholar 

  • Sunstein, C. & Hastie, R. (2008). Fur failures of deliberating groups. Public law and legal theory working paper series. Retrieved April 28, 2017 from http://www.law.uchicago.edu/academics/publiclaw/index.html.

  • The Jefferson Center (2004). Citizens jury handbook. Washington, DC. http://www.epfound.ge/files/citizens_jury_handbook.pdf. Accessed July 5, 2016.

  • Thomson, A. M., & Perry, J. L. (2006). Collaboration processes: Inside the black box. Public Administration Review, 66(s1), 20–32.

    Article  Google Scholar 

  • Wandersman, A., & Florin, P. (2000). Citizen participation and community organizations. In J. Rappaport & E. Seidman (Eds.), Handbook of community psychology (pp. 247–272). New York, NY: Kluwer Academic/Plenum Press.

    Chapter  Google Scholar 

  • Webler, T. (1995). ‘Right’ discourse in citizen participation: An evaluative yardstick. In O. Renn, T. Webler, & P. Wiedemann (Eds.), Fairness and competence in citizen participation: Evaluating models for environmental discourse (pp. 35–86). Boston; MA: Kluwer Academic.

    Chapter  Google Scholar 

  • Webler, T., & Tuler, S. (2001). Public participation in watershed management planning: Views on process from people in the field. Human Ecology Review, 8(2), 29–39.

    Google Scholar 

  • Wollabæk, D., & Selle, P. (2003). Participation and social capital formation: Norway in a comparative perspective. Scandinavian Political Studies, 26(1), 67–91.

    Article  Google Scholar 

  • Young, I. M. (2000). Democracy and inclusion. New York, NY: Oxford University Press.

    Google Scholar 

  • Zimmerman, M. A., & Rappaport, J. (1988). Citizen participation, perceived control, and psychological empowerment. American Journal of Community Psychology, 5, 725–750.

    Article  Google Scholar 

Download references

Acknowledgements

This work benefits from a research project “The quality of deliberation” (PRIN) funded by the Ministry of Education, University and Research of the Italian Government.

Author information

Authors and Affiliations

  1. Department of History, Society and Human Studies, University of Salento, Via di Valesio angolo V.le San Nicola, 73100, Lecce, Italy

    Terri Mannarini

  2. Department of Psychology, University of Torino, Via Verdi 10, 10124, Turin, Italy

    Angela Fedi

Authors
  1. Terri Mannarini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angela Fedi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angela Fedi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mannarini, T., Fedi, A. Using Quali-Quantitative Indicators for Assessing the Quality of Citizen Participation: A Study on Three Citizen Juries. Soc Indic Res 139, 473–490 (2018). https://doi.org/10.1007/s11205-017-1735-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-017-1735-9

Keywords

Search

Navigation