Environmental and Resource Economics

, Volume 20, Issue 3, pp 241–254 | Cite as

What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?

  • John A. List
  • Craig A. Gallet


Preferences elicited in hypothetical settings have recently come underscrutiny, causing estimates from the contingent valuation method to bechallenged due to perceived ``hypothetical bias.'' Given that the receivedliterature derives value estimates using heterogeneous experimentaltechniques, understanding the effects of important design parameters onthe magnitude of hypothetical bias is invaluable. In this paper, we addressthis issue statistically by using a meta-analysis to examine data from 29experimental studies. Our empirical findings suggest that on averagesubjects overstate their preferences by a factor of about 3 in hypotheticalsettings, and that the degree of over-revelation is influenced by thedistinction between willingness-to-pay and willingness-to-accept, publicversus private goods, and several elicitation methods.

CVM hypothetical bias meta-analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Balistreri, E., G. McClelland, G. Poe and W. Schulze (1998), Can Hypothetical Questions Reveal True Values? A Laboratory Comparison of Dichotomous Choice and Open-Ended Contingent Values with Auction Values. Cornell University, working paper, WP 97-15.Google Scholar
  2. Bishop, R. and T. Heberlein (1979), ‘Measuring Values of Extramarket Goods: Are Indirect Measures Biased?’, American Journal of Agricultural Economics 61, 926–930.Google Scholar
  3. Bishop, R. (1986), ‘Assessing the Validity of Contingent Valuations: Three Field Experiments’, Science of the Total Environment 56, 434–479.Google Scholar
  4. Bishop, R. (1990), ‘The Contingent Valuation Method’, in R. L. Johnson and G. V. Johnson, eds., Economic Valuation of Natural Resources. Boulder, CO: Westview Press, pp. 81–104.Google Scholar
  5. Bishop, R., T. Heberlein and M. J. Kealy (1983), ‘Contingent Valuation of Environmental Assets: Comparisons with a Simulated Market’, Natural Resources Journal 23, 619–633.Google Scholar
  6. Blackburn, M., G. Harrison and E. E. Ruström (1994), ‘Statistical Bias Functions and Informative Hypothetical Surveys’, American Journal of Agricultural Economics 76, 1084–1088.Google Scholar
  7. Bohm, P. (1972), ‘Estimating Demand for Public Goods: An Experiment’, Europeawn Economic Review 3(2), 111–130.Google Scholar
  8. Boyce, R., G. McClelland, T. Brown, G. Peterson and W. Schulze (1992), ‘An Experimental Examination of Intrinsic Values as a Source of the WTA-WTP Disparity’, American Economic Review 82(5), 1366–1373.Google Scholar
  9. Brookshire, D. and D. Coursey (1987), ‘Measuring the Value of a Public Good: An Empirical Comparison of Elicitation Procedures’, American Economic Review 77(4): 554–566.Google Scholar
  10. Brown, T., P. Champ, R. Bishop and D. McCollum (1996), ‘Which Response Format Reveals the Truth about Donations to a Public Good?’, Land Economics 72(2), 152–166.Google Scholar
  11. Coursey, D., J. Hovis and W. Schulze (1987), ‘The Disparity between Willingness to Accept and Willingness to Pay Measures of Value’, Quarterly Journal of Economics 102, 679–690.Google Scholar
  12. Cummings, R., D. Brookshire and W. Schulze, eds. (1986), Valuing Environmental Goods: An Assessment of the Contingent Valuation Method. Totowa, NJ: Rowman and Allanheld.Google Scholar
  13. Cummings, R., G. Harrison and E. E. Rutström (1995), ‘Homegrown Values and Hypothetical Surveys: Is the Dichotomous Choice Approach Incentive Compatible?’, American Economic Review 85, 260–266.Google Scholar
  14. Cummings, R., S. Elliot, G. Harrison and J. Murphy (1997), ‘Are Hypothetical Referenda Incentive Compatible?’, Journal of Political Economy 105(3), 609–621.Google Scholar
  15. Diamond, P. and J. Hausman (1994), ‘Contingent Valuation: Is Some Number Better than No Number?’, Journal of Economic Perspectives 8, 45–64.Google Scholar
  16. Dickie, M., A. Fisher and S. Gerking (1987), ‘Market Transactions and Hypothetical Demand Data: A Comparative Study’, Journal of American Statistical Association 82, 69–75.Google Scholar
  17. Espey, M. (1998), ‘Gasoline Demand Revisited: An International Meta-Analysis of Elasticities’, Energy Economics 20, 273–295.Google Scholar
  18. Foster, V., I. Bateman and D. Harley (1997), ‘Real and Hypothetical Willingness to Pay for Environmental Preservation: A Non-Experimental Comparison’, Journal of Agricultural Economics 48(2), 123–138.Google Scholar
  19. Fox, J., J. Shogren; D. Hayes and J. Kliebenstein (1998), ‘CVM-X: Calibrating Contingent Values with Experimental AuctionMarkets’, American Journal of Agricultural Economics 80, 455–465.Google Scholar
  20. Frykblom, P. (1997), ‘Hypothetical Question Modes and Real Willingness to Pay’, Journal of Environmental Economics and Management 34, 275–287.Google Scholar
  21. Frykblom, P. (2000), ‘Willingness to Pay and the Choice of Question Format: Experimental Results’, Applied Economics Letters 7, 665–667.Google Scholar
  22. Grether, D. and C. Plott (1979), ‘Economic Theory of Choice and the Preference Reversal Phenomenon’, American Economic Review 69(1), 623–638.Google Scholar
  23. Hofler, R. and J. A. List (2000), Valuation on the Frontier: Calibrating Actual and Hypothetical Statements. University of Arizona, working paper.Google Scholar
  24. Irwin, J., G. McClelland and W. Schulze (1992), ‘Hypothetical and Real Consequences in Experimental Auctions for Insurance Against Low Probability Risks’, Journal of Behavioral Decision Making 5, 107–116.Google Scholar
  25. Jeppessen, T., J. A. List and H. Folmer (2001), ‘Environmental Regulations and New Plant Location Decisions: Evidence from a Meta-Analysis’, Journal of Regional Science, forthcoming.Google Scholar
  26. Johannesson, M., B. Liljas and P. O. Johansson (1998), ‘An Experimental Comparison of Dichotomous Choice Contingent Valuation Questions and Real Purchase Decisions’, Applied Economics 30, 643–647.Google Scholar
  27. Kealy, M., J. Dovidio and M. Rockel (1988), ‘Accuracy in Valuation is a Matter of Degree’, Land Economics 64, 158–171.Google Scholar
  28. Kealy, M., J. Montgomery and J. Dovidio (1990), ‘Reliability and Predictive Validity of Contingent Values: Does the Nature of the Good Matter?’, Journal of Environmental Economics and Management 19, 244–263.Google Scholar
  29. List, J. A. (2001), ‘Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards’, American Economic Review, forthcoming.Google Scholar
  30. List, J. A. and J. Shogren (1998a), ‘Calibration of the Difference between Actual and Hypothetical Valuations in a Field Experiment’, Journal of Economic Behavior and Organization 37(2), 193–205.Google Scholar
  31. List, J. A. and J. Shogren (1998b), ‘The Deadweight Loss of Christmas: Comment’, American Economic Review 88(5), 1350–1355.Google Scholar
  32. List, J. A. and J. Shogren (1999), ‘Calibration of Willingness-to-Accept’, Journal of Environmental Economics and Management, forthcoming.Google Scholar
  33. Loomis, J., T. Brown, T. Lucero and G. Peterson (1996), ‘Improving Validity Experiments of Contingent Valuation Methods: Results of Efforts to Reduce the Disparity of Hypothetical and Actual Willingness to Pay’, Land Economics 72(4), 450–461.Google Scholar
  34. Loomis, J., T. Brown, T. Lucero and G. Peterson (1997), ‘Evaluating the Validity of the Dichotomous Choice Question Format in Contingent Valuation’, Environmental and Resources Economics 10, 109–123.Google Scholar
  35. McClelland, G., W. Schulze and D. Coursey (1993), ‘Insurance for Low-Probability Hazards: A Biomodal Response to Unlikely Events’, Journal of Risk and Uncertainty 7, 95–116.Google Scholar
  36. Navrud, S. (1992), ‘Willingness to Pay for Preservation of Species — An Experiment with Actual Payments’, in S. Navrud, ed., Pricing the European Environment. Oslo: Scandinavian University Press; distributed by Oxford University Press, New York, pp. 231–246.Google Scholar
  37. Neill, H., R. Cummings, P. Ganderton, G. Harrison and T. McGuckin (1994), ‘Hypothetical Surveys and Real Economic Commitments’, Land Economics 70(2): 145–154.Google Scholar
  38. National Oceanic and Atmospheric Administration (1994), ‘Natural Resource Damage Assessment: Proposed Rules’, Federal Register, 4 May 59, 23098–23111.Google Scholar
  39. National Oceanic and Atmospheric Administration (1996), ‘Natural Resource Damage Assessment: Final Rules’, Federal Register, 5 January 61, 439.Google Scholar
  40. Randall, A. (1996), ‘Calibration of CV Responses: Discussion’, in D. Bjornstad and J. Kahn, eds., The Contingent Valuation of Environmental Resources. London: Edgar Elgar, pp. 198–207.Google Scholar
  41. Seip, K. and J. Strand (1992), ‘Willingness to Pay for Environmental Goods in Norway: A Contingent Valuation Study with Real Payment’, Environmental and Resource Economics 2, 91–106.Google Scholar
  42. Sinden, J. A. (1988), ‘Empirical Tests of Hypothetical Biases in Consumers' Surplus Surveys’, Australian Journal of Agricultural Economics (August & December) 32 (2&3), 98–112.Google Scholar
  43. Smith, V. K. and C. Mansfield (1998), ‘Buying Time: Real and Hypothetical Offers’, Journal of Environmental Economics and Management 36, 209–224.Google Scholar
  44. Spencer, M., S. Swallow and C. Miller (1998), ‘Valuting Water Quality Monitoring: A Contingent Valuation Experiment Involving Hypothetical and Real Payments’, Agricultural and Resource Economics Review 27, 28–42.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  1. 1.University of MarylandCollege, ParkUSA
  2. 2.California State University at SacramentoSacramentoUSA

Personalised recommendations