Environmental and Resource Economics

, Volume 58, Issue 1, pp 35–57 | Cite as

A Meta-study Investigating the Sources of Protest Behaviour in Stated Preference Surveys

  • Jürgen Meyerhoff
  • Morten Raun Mørkbak
  • Søren Bøye Olsen


It is a well-known empirical finding that some percentage of respondents participating in Stated Preference surveys will not give responses that reflect their true preferences. One reason is protest behaviour. If the distribution of protest responses is not independent of respondent or survey characteristics, then simply expelling protesters from surveys can lead to sample selection bias. Furthermore, WTP estimates will not be comparable across surveys. This paper seeks to explore potential causes of protest behaviour through a meta-study based on full datasets from 38 different surveys. The objective of the study is to examine the effect of respondent specific variables as well as survey specific variables on protest behaviour. Our results suggest that some of the differences in WTP typically observed between different demographic groups, different elicitation formats and different question formats might actually be attributed to inherent differences in the propensity to protest. Our results indicate that the propensity for respondents to exhibit protest behaviour when asked a stated preference type valuation question depends on a number of specific factors, respondent specific as well as survey specific—knowledge which could be used in order to reduce protest behaviour.


Protest behaviour Stated preferences Survey design Willingness to pay Hierarchical logistic regression  Mixed effects 

JEL Classification

C93 D03 D60 Q51 



We are indebted to the following who graciously supplied us with datasets: Anna Bartczak, Mikołaj Czajkowski, Klaus Glenk, David Hoyos, Tobias Wronka, Berit Hasler, Marianne Källstrøm, Andreas Kamp and Ole Bonnichsen. We further thank participants at the WCERE 2010 and EAERE 2012 conferences as well as the editor and reviewers for their helpful comments and suggestions.


  1. Abildtrup J, Garcia S, Olsen SB, Stenger A (2012) Spatial preference heterogeneity in forest recreation. Ecol Econ (in press). doi: 10.1016/j.ecolecon.2013.01.001
  2. Alpizar F, Carlsson F, Johansson-Stenman O (2008) Anonymity, reciprocity, and conformity: evidence from voluntary contributions to a national park in Costa Rica. J Public Econ 92:1047–1060CrossRefGoogle Scholar
  3. Arrow K, Solow R, Portney PR, Leamer EE, Radner R, Schuman H (1993) Report of the NOAA panel on contingent valuation. The National Ocean and Atmospheric Association’s Damage Assessment and Restoration Program (DARP), WashingtonGoogle Scholar
  4. Bartczak A (2010) Wycena korzyści z poprawy jakości wody kranowej i owierzchniowej w Polsce. Ekonomia i Środowisko 38(2):123–141Google Scholar
  5. Bartczak A, Chilton S, Meyerhoff J (2012) Valuing improvements to threatened lynx populations in Poland. An application of choice experiments with an experimental measure of risk preferences of individuals, Manuscript, WarsawGoogle Scholar
  6. Bateman I, Carson RT, Day B, Hanemann M, Hanley N, Hett T, Jones-Lee M, Loomes G, Mourato S, Özdemiroglu E, Pearce DW, Sugden R, Swanson J (2002) Economic valuation with stated preference techniques: a manual. Edward Elgar, CheltenhamCrossRefGoogle Scholar
  7. Bateman I, Jones AP (2003) Contrasting conventional multi-level modeling approaches to meta-analysis: expectation consistency in U.K. woodland recreation values. Land Econ 79(2):235–258CrossRefGoogle Scholar
  8. Berrens RP, Bohara AK, Kerkvliet J (1997) A randomized response approach to dichotomous choice contingent valuation. Am J Agric Econ 79(1):252–266CrossRefGoogle Scholar
  9. Bonnichsen O (2011) Elicitation of ostomy pouch preferences: a discrete-choice experiment. Patient 4(3): 163–175Google Scholar
  10. Bonnichsen O, Ladenburg J (2009) Using an ex-ante entreatry to Reduce protest zero bias in stated preference surveys a health economic case. J Choice Model 2(2):83–98CrossRefGoogle Scholar
  11. Boyle KJ (2003) Contingent valuation in practice. In: Champ PA, Boyle KJ, Brown TC (eds) A primer on nonmarket valuation. Kluwer, DordrechtGoogle Scholar
  12. Boyle KJ, Bergstrom JC (1999) Doubt, doubt, and doubters: the genesis of a new research agenda? In: Bateman IJ, Willis KG (eds) Valuing environmental preferences. Theory and practice of the contingent valuation method in the US, EU, and developing countries. Oxford University Press, Oxford, pp 183–206Google Scholar
  13. Brookshire DS, Thayer MA, Schulze WD, d’Arge RC (1982) Valuing public goods: a comparison of survey and hedonic approaches. Am Econ Rev 72(1):165–177Google Scholar
  14. Brown KM, Taylor LO (2000) Do as you say, say as you do: evidence on gender differences in actual and stated contributions to public goods. J Econ Behav Organ 43(1):127–139CrossRefGoogle Scholar
  15. Campbell D, Mørkbak MR, Olsen SB (2012) Assessing the impact of opt-out definitions in choice experiments: a case study on food safety. Paper manuscriptGoogle Scholar
  16. Cameron TA, Poe GL, Ethier RG, Schulze WD (2002) Alternative non-market value-elicitation methods: are the underlying preferences the same? J Environ Econ Manag 44:391–425CrossRefGoogle Scholar
  17. Carlsson F, Mørkbak MR, Olsen SB (2012) The first time is the hardest: a test of ordering effects in choice experiments. J Choice Model 5(2):19–37Google Scholar
  18. Carson RT, Flores NE, Meade NF (2001) Contingent valuation: controversies and evidence. Environ Resour Econ 19(2):173–210Google Scholar
  19. Cherry TL, Crocker TD, Shogren JF (2003) Rationality spillovers. J Environ Econ Manag 45(1):63–84CrossRefGoogle Scholar
  20. Christensen T, Mørkbak MR, Evald SST, Jensen JD (2011) Danish consumers’ perceptions of food additives and other technologies. FOI Commissioned work No. 2011/4Google Scholar
  21. Christoffersen LB (2006) Juvre meadow—an economic analysis. Report from FOI, University of Copenhagen, DenmarkGoogle Scholar
  22. Cook C, Heath F, Thompson RL (2000) A meta-analysis of response rates in web- or internet-based surveys. Educ. Psychol. Meas. 60:821–836CrossRefGoogle Scholar
  23. Czajkowski M, Buszko-Briggs M, Hanley N (2009) Valuing changes in forest biodiversity. Ecol Econ 68(12): 2910–2917Google Scholar
  24. Daubert JT, Young RA (1981) Recreational demands for maintaining instream flows: a contingent valuation approach. Am J Agric Econ 63(4):666–676CrossRefGoogle Scholar
  25. Day BH, Bateman IJ, Carson RT, Dupont D, Louviere JJ, Morimoto S, Scarpa R, Wang P (2012) Ordering effects and choice set awareness in repeat-response stated preference studies. J Environ Econ Manag 63:73–91CrossRefGoogle Scholar
  26. Dillman DA (2007) Mail and internet surveys: the tailored design method—2007 update with new internet, visual, and mixed-mode guide, 2nd edn. Wiley, New YorkGoogle Scholar
  27. Dubgaard A, Olsen SB, Jespersen MML, Bonnichsen O, Klagenberg PA, Nielsen C-CM (2011) Opfattelse af risiko for oversvømmelse 2010 (Perception of the risk of flooding 2010—in Danish only). Joint report from Institute of Food and Resource Economics, University of Copenhagen, and the Danish Coastal Authority, Lemvig, DenmarkGoogle Scholar
  28. Dupont DP (2004) Do children matter? An examination of gender differences in environmental valuation. Ecol Econ 49(3):273–286CrossRefGoogle Scholar
  29. Glenk K, Colombo S (2011) Designing policies to mitigate the agricultural contribution to climate change: an assessment of soil based carbon sequestration and its ancillary effects. Clim Change 105:43–66Google Scholar
  30. Glenk K, Hall C, Liebe U, Meyerhoff J (2011a) Pesticide use in the Scotch malt whisky supply chain—analysing preference heterogeneity of whisky consumers. Manuscript, EdinburghGoogle Scholar
  31. Glenk K, Lago M, Moran D (2011b) Public preferences for water quality improvements: implications for the implementation of the EC Water Framework Directive in Scotland. Water Policy 13(5):645–662Google Scholar
  32. Goldstein H (1995) Multilevel statistical models, 2nd edn. Edward Arnold, LondonGoogle Scholar
  33. Hanemann M (1991) Willingness to pay and willingness to accept: How much can they differ? Am Econ Rev 81(3):635–647Google Scholar
  34. Hanley N, Shogren JF, White B (1997) Environmental economics in theory and practice. Palgrave Macmillan, EnglandGoogle Scholar
  35. Hanley N, Macmillan DC, Wright RE, Bullock C, Simpson I, Parsisson D, Crabtree B (1998a) Contingent valuation versus choice experiments: estimating the benefits of environmentally sensitive areas in Scotland. J Agric Econ 49(1):1–15CrossRefGoogle Scholar
  36. Hanley N, Wright RE, Adamowicz WL (1998b) Using choice experiments to value the environment. Environ Resour Econ 11(3–4):413–428CrossRefGoogle Scholar
  37. Hartje V, Meyer I, Meyerhoff J (2002) Kosten einer möglichen Klimaveränderung auf Sylt. In: Daschkeit A, Schottes P (eds) Sylt- Klimafolgen für Mensch und Küste. Springer, Berlin, pp 181–218Google Scholar
  38. Hasler B, Brodersen SL, Christensen LP, Christensen T, Dubgaard A, Hansen HE, Kataria M, Martinsen L, Nissen CJ, Wulff AF (2009) Denmark: assessing economic benefits of Good Ecological Status under the EU Water Framework Directive. Testing practical guidelines in Odense river basin. Case study report, AquaMoneyGoogle Scholar
  39. Hayes KM, Tyrrell TJ, Anderson G (1992) Estimating the benefits of water quality improvements in the upper Narragansett Bay. Marine Resour Econ 7:75–85Google Scholar
  40. Heberlein TA, Baumgartner R (1979) Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. Am Sociol Rev 43:447–462CrossRefGoogle Scholar
  41. Holms TP, Kramer RA (1995) An independent sample test of yea-saying and starting point bias in dichotomous-choice contingent valuation. J Environ Econ Manag 29:121–132CrossRefGoogle Scholar
  42. Hoyos D, Mariel P, Garmendia E (2011) The management of Natura 2000 Network sites: a discrete choice experiment approach. Working paper, Departamento de Economía Aplicada III (Econometría y Estadística), Universidad del País Vasco, BilbaoGoogle Scholar
  43. Jensen CL, Nissen CJ, Olsen SB, Boesen M (2010) Analyse af udenlandske fisketurister i Danmark (An analysis of tourists angling in Denmark—in Danish only). Working paper from FOI, University of Copenhagen, Denmark. (forthcoming)Google Scholar
  44. Jorgensen BS, Syme GJ (2000) Protest response and willingness to pay: attitude toward paying for Stormwater pollution abatment. Ecol Econ 33:251–265CrossRefGoogle Scholar
  45. Jorgensen BS, Syme GJ, Bishop BJ, Nancarrow BE (1999) Protest responses in contingent valuation. Environ Resour Econ 14(1):131–150CrossRefGoogle Scholar
  46. Jorgensen BS, Wilson MA, Heberlein TA (2001) Fairness in the contingent valuation of environmental goods: attitude toward paying for environmental improvements at two levels of scope. Ecol Econ 36:133–148CrossRefGoogle Scholar
  47. Källstrøm MN, Hasler B, Olsen SB, Brodersen SL, Viuf P, Levin G (2010) Testing different approaches to Benefit Transfers between two sites in the same country, valuing the improvement of water quality. In: Paper presented at the 11th ISEE conference. Oldenburg and Bremen, Germany, August 22–25 (2010)Google Scholar
  48. Kamp A (2010) Værdisætning af faciliteter og services i Isfjordsområdet (Valuation of facilities and services in the Icefjord area—in Danish only). Unpublished report. University of CopenhagenGoogle Scholar
  49. Kelman H (1958) Compliance, identification, and internalization: three processes of attitude change. J Confl Resolut 1:51–60CrossRefGoogle Scholar
  50. Klaphake A, Meyerhoff J (2004) Zur Zahlungsbereitschaft für Eintrittspreise der Besucher der Parkanlagen Schloss Sanssouci und Schloss Charlottenburg. Report. BerlinGoogle Scholar
  51. Kontoleon A, Yabe M, Darby L (2005) Alternative payment vehicles in contingent valuation: the case of genetically modified foods. MPRA paper No. 1827, University of Cambridge, UK.
  52. Ladenburg J, Olsen SB (2008) Gender-specific starting point bias in choice experiments: evidence from an empirical study. J Environ Econ Manag 56(3):275–285CrossRefGoogle Scholar
  53. Ladenburg J, Olsen SB (2009) Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys. In: Paper presented at the EAERE 2009 annual conference, Amsterdam, Holland 24–27 June, 2009Google Scholar
  54. List JA (2003) Does market experience eliminate market anomalies? Q J Econ 118(1):41–71CrossRefGoogle Scholar
  55. Longo A, Hoyos D, Markandya A (2012) Willingness to pay for ancillary benefits of climate change mitigation. Environ Resour Econ 51(1):119–140CrossRefGoogle Scholar
  56. Marta-Pedroso C, Freitas H, Domingos T (2007) Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecol Econ 62:388–398Google Scholar
  57. Meyerhoff J, Liebe U (2008) Do protest responses to a contingent valuation question and a choice experiment differ? Environ Resour Econ 39(4):433–446CrossRefGoogle Scholar
  58. Meyerhoff J, Dehnhardt A (2007) The European water framework directive and economic valuation of wetlands: the restoration of floodplains along the River Elbe. Eur Environ 17(1):18–36CrossRefGoogle Scholar
  59. Meyerhoff J, Liebe U (2006) Protest beliefs in contingent valuation: explaining their motivation. Ecol Econ 57(4):583–594CrossRefGoogle Scholar
  60. Meyerhoff J, Liebe U (2009) Status quo effect in choice experiments: empirical evidence on attitudes and choice task complexity. Land Econ 85(3):515–528Google Scholar
  61. Meyerhoff J, Liebe U (2010) Determinants of protest responses in environmental valuation: a meta-study. Ecol Econ 70(2):366–374CrossRefGoogle Scholar
  62. Meyerhoff J, Angeli D (2011) Willingness to pay for reducing eutrophication in the Baltic Sea. Report for the Federal German Environmental Protection Agency. BerlinGoogle Scholar
  63. Meyerhoff J, Angeli D, Hartje V (2010c) Social benefits of implementing a national strategy on biological diversity in Germany. Environ Sci Policy 23:109–119Google Scholar
  64. Meyerhoff J, Dehnhardt A, Hartje V (2010a) Take your swim suit along: the value of improving urban bathing sites in the metropolitan area of Berlin. J Environ Plan Manag 53(1):107–124Google Scholar
  65. Meyerhoff J, Ohl C, Hartje V (2008) Präferenzen für die Ausgestaltung der Windkraft in der Landschaft. Ergebnisse einer Online-Umfrage in Deutschland. BerlinGoogle Scholar
  66. Meyerhoff J, Ohl C, Hartje V (2010b) Landscape externalities of onshore wind power generation. Energy Policy 38(1):82–92Google Scholar
  67. Meyers-Levy J (1989) Gender differences in information processing: a selective interpretation. In: Cafferata P, Tybout AM (eds) Cognitive and affective responses to advertising. Lexington, Books, CanadaGoogle Scholar
  68. Mitani Y, Flores N (2007) Does gender matter for demand revelation in threshold public goods experiments? Econ Bull 3(27):1–7Google Scholar
  69. Mitchell RC, Carson RT (1989) Using surveys to value public goods: the contingent valuation method. Resources for the future, Washington DCGoogle Scholar
  70. Mogas J, Riera P, Bennett J (2005) Accounting for afforestation externalities: a comparison of contingent valuation and choice modelling. Eur Environ 15(1):44–58CrossRefGoogle Scholar
  71. Mørkbak MR, Nordström J (2009) The impact of information on consumer preferences for different animal food production methods. J Consum Policy. doi: 10.1007/s10603-009-9106-9
  72. Mørkbak MR, Christensen T, Gyrd-Hansen D, Olsen SB (2011) Is embedding entailed in consumer valuation of food safety characteristics? Eur Rev Agric Econ. doi: 10.1093/erae/jbr021
  73. Mørkbak MR, Jensen JD (2012) Do consumers’ preferences change when on vacation? A willingness to pay study on apples and honey. The joint AAEA/EAAE conference on ‘Food Environment: The Effects of Context on Food Choice’, May 30–31, 2012, Tufts University, Boston, MAGoogle Scholar
  74. Morrison MD, Blamey RK, Bennett JW (2000) Minimising payment vehicle bias in contingent valuation studies. Environ Resour Econ 16:407–422CrossRefGoogle Scholar
  75. Nielsen JS (2011) use of the internet for willingness-to-pay surveys: a comparison of face-to-face and web-based interviews. Resour Energy Econ 33:119–129CrossRefGoogle Scholar
  76. Nielsen AB, Olsen SB, Lundhede T (2007) An economic valuation of the recreational benefits associated with nature-based forest management practices. Landsc Urban Plan 80(1–2):63–71CrossRefGoogle Scholar
  77. Olsen SB (2009) Choosing between internet and mail survey modes for choice experiment surveys considering non-market goods. Environ Res Econ 44(4):591–610Google Scholar
  78. Olsen SB, Ladenburg J, Petersen ML, Lopdrup U, Hansen AS, Dubgaard A (2005) Motorways versus Nature—A Welfare Economic Valuation of Impacts. Report from FOI and IMV, CopenhagenGoogle Scholar
  79. Rosenberger RS, Johnston RJ (2009) Selection effects in meta-analysis and benefit transfer: avoiding unintended consequences. Land Econ 85(3):410–428Google Scholar
  80. Sheehan K (2001) E-mail survey response rates: a review. J Comput Mediat Commun 6Google Scholar
  81. Söderquist T (1998) Why give up money for the Baltic Sea? Environ Resour Econ 12:249–254CrossRefGoogle Scholar
  82. Strazzera E, Genius M, Scarpa R, Hutchinson WG (2003) The effect of protest votes on the estimates of WTP for use values of recreational sites. Environ Resour Econ 25:461–476CrossRefGoogle Scholar
  83. Teal G, Loomis JB (2000) Effects of gender and parental status on the economic valuation of increasing wetlands, reducing wildlife contamination and increasing salmon populations. Soc Nat Resour 13(1):1–14CrossRefGoogle Scholar
  84. Tranberg J, Christoffersen LB, Dubgaard A, Olsen SB, Lassen C (2005) Opfattelse af risiko for oversvømmelse (Perception of the risk of flooding—in Danish only). Report from the Danish Coastal Authority, Lemvig, DenmarkGoogle Scholar
  85. von Haefen R, Massey DM, Adamowicz WL (2005) Serial nonparticipation in repeated discrete choice models. Am J Agri Econo 87(4):1061–1076Google Scholar
  86. Whittington D, Adamowicz V (2011) The use of hypothetical baselines in stated preference surveys. Discussion Paper Series, December 2011 EfD DP 11-11Google Scholar
  87. Wronka T (2004) Ökonomische Umweltbewertung: vergleichende Analyse und neuere Entwicklungen der kontingenten Bewertung am Beispiel der Artenvielfalt und Trinkwasserqualität. KielGoogle Scholar
  88. Yammarino FJ, Skinner SJ, Childers TL (1991) Understanding mail survey response behavior. Public Opin Q 55(4):613–639Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Jürgen Meyerhoff
    • 1
  • Morten Raun Mørkbak
    • 2
  • Søren Bøye Olsen
    • 3
  1. 1.Institute for Landscape Architecture and Environmental PlanningTechnische Universität BerlinBerlinGermany
  2. 2.Department of Business and Economics, COHEREUniversity of Southern DenmarkOdenseDenmark
  3. 3.Department of Food and Resource Economics, Science FacultyUniversity of Copenhagen Frederiksberg CDenmark

Personalised recommendations