Skip to main content

Advertisement

SpringerLink
Log in

A Meta-study Investigating the Sources of Protest Behaviour in Stated Preference Surveys

  • Published:
Environmental and Resource Economics Aims and scope Submit manuscript

Abstract

It is a well-known empirical finding that some percentage of respondents participating in Stated Preference surveys will not give responses that reflect their true preferences. One reason is protest behaviour. If the distribution of protest responses is not independent of respondent or survey characteristics, then simply expelling protesters from surveys can lead to sample selection bias. Furthermore, WTP estimates will not be comparable across surveys. This paper seeks to explore potential causes of protest behaviour through a meta-study based on full datasets from 38 different surveys. The objective of the study is to examine the effect of respondent specific variables as well as survey specific variables on protest behaviour. Our results suggest that some of the differences in WTP typically observed between different demographic groups, different elicitation formats and different question formats might actually be attributed to inherent differences in the propensity to protest. Our results indicate that the propensity for respondents to exhibit protest behaviour when asked a stated preference type valuation question depends on a number of specific factors, respondent specific as well as survey specific—knowledge which could be used in order to reduce protest behaviour.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. This is also the reason why we have chosen to refer to our approach a meta-study rather than a meta-analysis since we are modeling on full datasets rather than meta-data.

  2. The indication of not being willing to pay naturally differed across question formats. In OE-CVM a zero WTP bid served as an indicator of not being willing to pay whereas a “No” response were used in DC-CVM. In CE, respondents consistently choosing the status quo in all choice tasks were treated as not being willing to pay, and thus being potential protesters.

  3. Typically, conformity effects in SP surveys are thought of as affecting positive bids in terms of respondents adjusting their own WTP statements relative to the WTP statements of others (Alpizar et al. 2008). The situation we describe here would be a sort of second best conformity effect: Due to budget constraints, the zero bidders have already decided not to conform to the social norm which would be to state a positive bid. However, they might think that stating budget constraints as the reason for a zero bid might make others think less of them, and, hence, they pick some other reason. Choosing a protest reason indicates that a zero bid might not be the respondent’s true WTP. In that sense, this could be a way of signaling that you might conform to the social norm of paying for the good, but you just do not conform to the premises of the survey.

  4. As noted by a reviewer, all the surveys used here are potentially prone to sample self-selection bias. In particular, it is likely that users of a good will be more inclined to participate in a survey than non-users. Our model assumes independence between the survey level random error and the respondent level random error. As such it does not take into account the potential endogeneity (for instance between use of the good and the type of good) that could emerge as a result of such self-selection bias.

  5. We consider this measure more precise than simply looking at the number of questions since a question can contain several items, and the more items, the higher the cognitive load.

  6. It should be noted that this is relative to a 0 % protest rate in the personal interviews.

References

  • Abildtrup J, Garcia S, Olsen SB, Stenger A (2012) Spatial preference heterogeneity in forest recreation. Ecol Econ (in press). doi:10.1016/j.ecolecon.2013.01.001

  • Alpizar F, Carlsson F, Johansson-Stenman O (2008) Anonymity, reciprocity, and conformity: evidence from voluntary contributions to a national park in Costa Rica. J Public Econ 92:1047–1060

    Article  Google Scholar 

  • Arrow K, Solow R, Portney PR, Leamer EE, Radner R, Schuman H (1993) Report of the NOAA panel on contingent valuation. The National Ocean and Atmospheric Association’s Damage Assessment and Restoration Program (DARP), Washington

    Google Scholar 

  • Bartczak A (2010) Wycena korzyści z poprawy jakości wody kranowej i owierzchniowej w Polsce. Ekonomia i Środowisko 38(2):123–141

    Google Scholar 

  • Bartczak A, Chilton S, Meyerhoff J (2012) Valuing improvements to threatened lynx populations in Poland. An application of choice experiments with an experimental measure of risk preferences of individuals, Manuscript, Warsaw

  • Bateman I, Carson RT, Day B, Hanemann M, Hanley N, Hett T, Jones-Lee M, Loomes G, Mourato S, Özdemiroglu E, Pearce DW, Sugden R, Swanson J (2002) Economic valuation with stated preference techniques: a manual. Edward Elgar, Cheltenham

    Book  Google Scholar 

  • Bateman I, Jones AP (2003) Contrasting conventional multi-level modeling approaches to meta-analysis: expectation consistency in U.K. woodland recreation values. Land Econ 79(2):235–258

    Article  Google Scholar 

  • Berrens RP, Bohara AK, Kerkvliet J (1997) A randomized response approach to dichotomous choice contingent valuation. Am J Agric Econ 79(1):252–266

    Article  Google Scholar 

  • Bonnichsen O (2011) Elicitation of ostomy pouch preferences: a discrete-choice experiment. Patient 4(3): 163–175

    Google Scholar 

  • Bonnichsen O, Ladenburg J (2009) Using an ex-ante entreatry to Reduce protest zero bias in stated preference surveys a health economic case. J Choice Model 2(2):83–98

    Article  Google Scholar 

  • Boyle KJ (2003) Contingent valuation in practice. In: Champ PA, Boyle KJ, Brown TC (eds) A primer on nonmarket valuation. Kluwer, Dordrecht

    Google Scholar 

  • Boyle KJ, Bergstrom JC (1999) Doubt, doubt, and doubters: the genesis of a new research agenda? In: Bateman IJ, Willis KG (eds) Valuing environmental preferences. Theory and practice of the contingent valuation method in the US, EU, and developing countries. Oxford University Press, Oxford, pp 183–206

    Google Scholar 

  • Brookshire DS, Thayer MA, Schulze WD, d’Arge RC (1982) Valuing public goods: a comparison of survey and hedonic approaches. Am Econ Rev 72(1):165–177

    Google Scholar 

  • Brown KM, Taylor LO (2000) Do as you say, say as you do: evidence on gender differences in actual and stated contributions to public goods. J Econ Behav Organ 43(1):127–139

    Article  Google Scholar 

  • Campbell D, Mørkbak MR, Olsen SB (2012) Assessing the impact of opt-out definitions in choice experiments: a case study on food safety. Paper manuscript

  • Cameron TA, Poe GL, Ethier RG, Schulze WD (2002) Alternative non-market value-elicitation methods: are the underlying preferences the same? J Environ Econ Manag 44:391–425

    Article  Google Scholar 

  • Carlsson F, Mørkbak MR, Olsen SB (2012) The first time is the hardest: a test of ordering effects in choice experiments. J Choice Model 5(2):19–37

    Google Scholar 

  • Carson RT, Flores NE, Meade NF (2001) Contingent valuation: controversies and evidence. Environ Resour Econ 19(2):173–210

    Google Scholar 

  • Cherry TL, Crocker TD, Shogren JF (2003) Rationality spillovers. J Environ Econ Manag 45(1):63–84

    Article  Google Scholar 

  • Christensen T, Mørkbak MR, Evald SST, Jensen JD (2011) Danish consumers’ perceptions of food additives and other technologies. FOI Commissioned work No. 2011/4

  • Christoffersen LB (2006) Juvre meadow—an economic analysis. Report from FOI, University of Copenhagen, Denmark

  • Cook C, Heath F, Thompson RL (2000) A meta-analysis of response rates in web- or internet-based surveys. Educ. Psychol. Meas. 60:821–836

    Article  Google Scholar 

  • Czajkowski M, Buszko-Briggs M, Hanley N (2009) Valuing changes in forest biodiversity. Ecol Econ 68(12): 2910–2917

    Google Scholar 

  • Daubert JT, Young RA (1981) Recreational demands for maintaining instream flows: a contingent valuation approach. Am J Agric Econ 63(4):666–676

    Article  Google Scholar 

  • Day BH, Bateman IJ, Carson RT, Dupont D, Louviere JJ, Morimoto S, Scarpa R, Wang P (2012) Ordering effects and choice set awareness in repeat-response stated preference studies. J Environ Econ Manag 63:73–91

    Article  Google Scholar 

  • Dillman DA (2007) Mail and internet surveys: the tailored design method—2007 update with new internet, visual, and mixed-mode guide, 2nd edn. Wiley, New York

    Google Scholar 

  • Dubgaard A, Olsen SB, Jespersen MML, Bonnichsen O, Klagenberg PA, Nielsen C-CM (2011) Opfattelse af risiko for oversvømmelse 2010 (Perception of the risk of flooding 2010—in Danish only). Joint report from Institute of Food and Resource Economics, University of Copenhagen, and the Danish Coastal Authority, Lemvig, Denmark

  • Dupont DP (2004) Do children matter? An examination of gender differences in environmental valuation. Ecol Econ 49(3):273–286

    Article  Google Scholar 

  • Glenk K, Colombo S (2011) Designing policies to mitigate the agricultural contribution to climate change: an assessment of soil based carbon sequestration and its ancillary effects. Clim Change 105:43–66

    Google Scholar 

  • Glenk K, Hall C, Liebe U, Meyerhoff J (2011a) Pesticide use in the Scotch malt whisky supply chain—analysing preference heterogeneity of whisky consumers. Manuscript, Edinburgh

  • Glenk K, Lago M, Moran D (2011b) Public preferences for water quality improvements: implications for the implementation of the EC Water Framework Directive in Scotland. Water Policy 13(5):645–662

    Google Scholar 

  • Goldstein H (1995) Multilevel statistical models, 2nd edn. Edward Arnold, London

    Google Scholar 

  • Hanemann M (1991) Willingness to pay and willingness to accept: How much can they differ? Am Econ Rev 81(3):635–647

    Google Scholar 

  • Hanley N, Shogren JF, White B (1997) Environmental economics in theory and practice. Palgrave Macmillan, England

    Google Scholar 

  • Hanley N, Macmillan DC, Wright RE, Bullock C, Simpson I, Parsisson D, Crabtree B (1998a) Contingent valuation versus choice experiments: estimating the benefits of environmentally sensitive areas in Scotland. J Agric Econ 49(1):1–15

    Article  Google Scholar 

  • Hanley N, Wright RE, Adamowicz WL (1998b) Using choice experiments to value the environment. Environ Resour Econ 11(3–4):413–428

    Article  Google Scholar 

  • Hartje V, Meyer I, Meyerhoff J (2002) Kosten einer möglichen Klimaveränderung auf Sylt. In: Daschkeit A, Schottes P (eds) Sylt- Klimafolgen für Mensch und Küste. Springer, Berlin, pp 181–218

  • Hasler B, Brodersen SL, Christensen LP, Christensen T, Dubgaard A, Hansen HE, Kataria M, Martinsen L, Nissen CJ, Wulff AF (2009) Denmark: assessing economic benefits of Good Ecological Status under the EU Water Framework Directive. Testing practical guidelines in Odense river basin. Case study report, AquaMoney

  • Hayes KM, Tyrrell TJ, Anderson G (1992) Estimating the benefits of water quality improvements in the upper Narragansett Bay. Marine Resour Econ 7:75–85

    Google Scholar 

  • Heberlein TA, Baumgartner R (1979) Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. Am Sociol Rev 43:447–462

    Article  Google Scholar 

  • Holms TP, Kramer RA (1995) An independent sample test of yea-saying and starting point bias in dichotomous-choice contingent valuation. J Environ Econ Manag 29:121–132

    Article  Google Scholar 

  • Hoyos D, Mariel P, Garmendia E (2011) The management of Natura 2000 Network sites: a discrete choice experiment approach. Working paper, Departamento de Economía Aplicada III (Econometría y Estadística), Universidad del País Vasco, Bilbao

  • Jensen CL, Nissen CJ, Olsen SB, Boesen M (2010) Analyse af udenlandske fisketurister i Danmark (An analysis of tourists angling in Denmark—in Danish only). Working paper from FOI, University of Copenhagen, Denmark. (forthcoming)

  • Jorgensen BS, Syme GJ (2000) Protest response and willingness to pay: attitude toward paying for Stormwater pollution abatment. Ecol Econ 33:251–265

    Article  Google Scholar 

  • Jorgensen BS, Syme GJ, Bishop BJ, Nancarrow BE (1999) Protest responses in contingent valuation. Environ Resour Econ 14(1):131–150

    Article  Google Scholar 

  • Jorgensen BS, Wilson MA, Heberlein TA (2001) Fairness in the contingent valuation of environmental goods: attitude toward paying for environmental improvements at two levels of scope. Ecol Econ 36:133–148

    Article  Google Scholar 

  • Källstrøm MN, Hasler B, Olsen SB, Brodersen SL, Viuf P, Levin G (2010) Testing different approaches to Benefit Transfers between two sites in the same country, valuing the improvement of water quality. In: Paper presented at the 11th ISEE conference. Oldenburg and Bremen, Germany, August 22–25 (2010)

  • Kamp A (2010) Værdisætning af faciliteter og services i Isfjordsområdet (Valuation of facilities and services in the Icefjord area—in Danish only). Unpublished report. University of Copenhagen

  • Kelman H (1958) Compliance, identification, and internalization: three processes of attitude change. J Confl Resolut 1:51–60

    Article  Google Scholar 

  • Klaphake A, Meyerhoff J (2004) Zur Zahlungsbereitschaft für Eintrittspreise der Besucher der Parkanlagen Schloss Sanssouci und Schloss Charlottenburg. Report. Berlin

  • Kontoleon A, Yabe M, Darby L (2005) Alternative payment vehicles in contingent valuation: the case of genetically modified foods. MPRA paper No. 1827, University of Cambridge, UK. http://mpra.ub.uni-muenchen.de/1827/

  • Ladenburg J, Olsen SB (2008) Gender-specific starting point bias in choice experiments: evidence from an empirical study. J Environ Econ Manag 56(3):275–285

    Article  Google Scholar 

  • Ladenburg J, Olsen SB (2009) Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys. In: Paper presented at the EAERE 2009 annual conference, Amsterdam, Holland 24–27 June, 2009

  • List JA (2003) Does market experience eliminate market anomalies? Q J Econ 118(1):41–71

    Article  Google Scholar 

  • Longo A, Hoyos D, Markandya A (2012) Willingness to pay for ancillary benefits of climate change mitigation. Environ Resour Econ 51(1):119–140

    Article  Google Scholar 

  • Marta-Pedroso C, Freitas H, Domingos T (2007) Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecol Econ 62:388–398

    Google Scholar 

  • Meyerhoff J, Liebe U (2008) Do protest responses to a contingent valuation question and a choice experiment differ? Environ Resour Econ 39(4):433–446

    Article  Google Scholar 

  • Meyerhoff J, Dehnhardt A (2007) The European water framework directive and economic valuation of wetlands: the restoration of floodplains along the River Elbe. Eur Environ 17(1):18–36

    Article  Google Scholar 

  • Meyerhoff J, Liebe U (2006) Protest beliefs in contingent valuation: explaining their motivation. Ecol Econ 57(4):583–594

    Article  Google Scholar 

  • Meyerhoff J, Liebe U (2009) Status quo effect in choice experiments: empirical evidence on attitudes and choice task complexity. Land Econ 85(3):515–528

    Google Scholar 

  • Meyerhoff J, Liebe U (2010) Determinants of protest responses in environmental valuation: a meta-study. Ecol Econ 70(2):366–374

    Article  Google Scholar 

  • Meyerhoff J, Angeli D (2011) Willingness to pay for reducing eutrophication in the Baltic Sea. Report for the Federal German Environmental Protection Agency. Berlin

  • Meyerhoff J, Angeli D, Hartje V (2010c) Social benefits of implementing a national strategy on biological diversity in Germany. Environ Sci Policy 23:109–119

    Google Scholar 

  • Meyerhoff J, Dehnhardt A, Hartje V (2010a) Take your swim suit along: the value of improving urban bathing sites in the metropolitan area of Berlin. J Environ Plan Manag 53(1):107–124

    Google Scholar 

  • Meyerhoff J, Ohl C, Hartje V (2008) Präferenzen für die Ausgestaltung der Windkraft in der Landschaft. Ergebnisse einer Online-Umfrage in Deutschland. Berlin

  • Meyerhoff J, Ohl C, Hartje V (2010b) Landscape externalities of onshore wind power generation. Energy Policy 38(1):82–92

    Google Scholar 

  • Meyers-Levy J (1989) Gender differences in information processing: a selective interpretation. In: Cafferata P, Tybout AM (eds) Cognitive and affective responses to advertising. Lexington, Books, Canada

  • Mitani Y, Flores N (2007) Does gender matter for demand revelation in threshold public goods experiments? Econ Bull 3(27):1–7

    Google Scholar 

  • Mitchell RC, Carson RT (1989) Using surveys to value public goods: the contingent valuation method. Resources for the future, Washington DC

    Google Scholar 

  • Mogas J, Riera P, Bennett J (2005) Accounting for afforestation externalities: a comparison of contingent valuation and choice modelling. Eur Environ 15(1):44–58

    Article  Google Scholar 

  • Mørkbak MR, Nordström J (2009) The impact of information on consumer preferences for different animal food production methods. J Consum Policy. doi:10.1007/s10603-009-9106-9

  • Mørkbak MR, Christensen T, Gyrd-Hansen D, Olsen SB (2011) Is embedding entailed in consumer valuation of food safety characteristics? Eur Rev Agric Econ. doi:10.1093/erae/jbr021

  • Mørkbak MR, Jensen JD (2012) Do consumers’ preferences change when on vacation? A willingness to pay study on apples and honey. The joint AAEA/EAAE conference on ‘Food Environment: The Effects of Context on Food Choice’, May 30–31, 2012, Tufts University, Boston, MA

  • Morrison MD, Blamey RK, Bennett JW (2000) Minimising payment vehicle bias in contingent valuation studies. Environ Resour Econ 16:407–422

    Article  Google Scholar 

  • Nielsen JS (2011) use of the internet for willingness-to-pay surveys: a comparison of face-to-face and web-based interviews. Resour Energy Econ 33:119–129

    Article  Google Scholar 

  • Nielsen AB, Olsen SB, Lundhede T (2007) An economic valuation of the recreational benefits associated with nature-based forest management practices. Landsc Urban Plan 80(1–2):63–71

    Article  Google Scholar 

  • Olsen SB (2009) Choosing between internet and mail survey modes for choice experiment surveys considering non-market goods. Environ Res Econ 44(4):591–610

    Google Scholar 

  • Olsen SB, Ladenburg J, Petersen ML, Lopdrup U, Hansen AS, Dubgaard A (2005) Motorways versus Nature—A Welfare Economic Valuation of Impacts. Report from FOI and IMV, Copenhagen

  • Rosenberger RS, Johnston RJ (2009) Selection effects in meta-analysis and benefit transfer: avoiding unintended consequences. Land Econ 85(3):410–428

    Google Scholar 

  • Sheehan K (2001) E-mail survey response rates: a review. J Comput Mediat Commun 6

  • Söderquist T (1998) Why give up money for the Baltic Sea? Environ Resour Econ 12:249–254

    Article  Google Scholar 

  • Strazzera E, Genius M, Scarpa R, Hutchinson WG (2003) The effect of protest votes on the estimates of WTP for use values of recreational sites. Environ Resour Econ 25:461–476

    Article  Google Scholar 

  • Teal G, Loomis JB (2000) Effects of gender and parental status on the economic valuation of increasing wetlands, reducing wildlife contamination and increasing salmon populations. Soc Nat Resour 13(1):1–14

    Article  Google Scholar 

  • Tranberg J, Christoffersen LB, Dubgaard A, Olsen SB, Lassen C (2005) Opfattelse af risiko for oversvømmelse (Perception of the risk of flooding—in Danish only). Report from the Danish Coastal Authority, Lemvig, Denmark

  • von Haefen R, Massey DM, Adamowicz WL (2005) Serial nonparticipation in repeated discrete choice models. Am J Agri Econo 87(4):1061–1076

    Google Scholar 

  • Whittington D, Adamowicz V (2011) The use of hypothetical baselines in stated preference surveys. Discussion Paper Series, December 2011 EfD DP 11-11

  • Wronka T (2004) Ökonomische Umweltbewertung: vergleichende Analyse und neuere Entwicklungen der kontingenten Bewertung am Beispiel der Artenvielfalt und Trinkwasserqualität. Kiel

  • Yammarino FJ, Skinner SJ, Childers TL (1991) Understanding mail survey response behavior. Public Opin Q 55(4):613–639

    Google Scholar 

Download references

Acknowledgments

We are indebted to the following who graciously supplied us with datasets: Anna Bartczak, Mikołaj Czajkowski, Klaus Glenk, David Hoyos, Tobias Wronka, Berit Hasler, Marianne Källstrøm, Andreas Kamp and Ole Bonnichsen. We further thank participants at the WCERE 2010 and EAERE 2012 conferences as well as the editor and reviewers for their helpful comments and suggestions.

Author information

Authors and Affiliations

  1. Institute for Landscape Architecture and Environmental Planning, Technische Universität Berlin, Berlin, Germany

    Jürgen Meyerhoff

  2. Department of Business and Economics, COHERE, University of Southern Denmark, Odense, Denmark

    Morten Raun Mørkbak

  3. Department of Food and Resource Economics, Science Faculty, University of Copenhagen, Rolighedsvej 25, 1958, Frederiksberg C, Denmark

    Søren Bøye Olsen

Authors
  1. Jürgen Meyerhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Morten Raun Mørkbak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Søren Bøye Olsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Søren Bøye Olsen.

Appendix A

Appendix A

See Table 4.

Table 4 Surveys included in the meta-study
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Meyerhoff, J., Mørkbak, M.R. & Olsen, S.B. A Meta-study Investigating the Sources of Protest Behaviour in Stated Preference Surveys. Environ Resource Econ 58, 35–57 (2014). https://doi.org/10.1007/s10640-013-9688-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10640-013-9688-1

Keywords

JEL Classification

Search

Navigation