Environmental and Resource Economics

, Volume 64, Issue 3, pp 401–419 | Cite as

Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames

  • Kevin J. BoyleEmail author
  • Mark Morrison
  • Darla Hatton MacDonald
  • Roderick Duncan
  • John Rose


The increasing use of internet surveys for stated preference studies raises questions about the effect of the survey mode on welfare estimates. A number of studies have conducted convergent validity investigations that compare the use of the internet with other survey implementation modes such as mail, telephone and in-person. All, but one, of these comparison studies is confounded different sample frames for the different modes of survey implementation. In this study we investigate differences in internet and mail survey modes holding the sample frame constant. This is done in the context of a choice-modelling study of improvements in the River Murray in Australia. We also investigate sample frame holding the survey mode (mail) constant. We find that survey mode (internet vs. mail) influences welfare estimates, and the internet welfare estimates are about 78 % of the mail welfare estimates on average. There is not a significant effect of sample frame (internet panel vs. postal addresses) on welfare estimates for implementation of a mail survey.


Choice experiment Internet survey Mail survey  Error component model River quality improvements Survey mode comparison 


  1. American Association for Public Opinion Research (AAPOR) (2006) Standard definitions: final dispositions of case codes and outcome rates for surveys. Retrieved June 18, 2007, from
  2. Banzhaf S, Burtraw D, Evans D, Krupnick A (2006) Valuation of natural resource improvements in the Adirondacks. Land Econ 82(3):445–464CrossRefGoogle Scholar
  3. Battaglia Michael P, Hoaglin David C, Frankel Martin R (2009) Practical considerations in raking survey data. Surv Pract 2(5):1–10Google Scholar
  4. Bell J, Huber J, Viscusi WK (2011) Survey mode effects on valuation of environmental goods. Int J Environ Res Public Health 8:1222–1243CrossRefGoogle Scholar
  5. Berrens RP, Bohara AK, Jenkins-Smith H, Silva C (2003) The advent of internet surveys for political research: a comparison of telephone and internet samples. Polit Anal 11(1):1–22CrossRefGoogle Scholar
  6. Bliemer M, Rose J, Hensher D (2009) Efficient stated choice experiments for estimating nested logit models. Transp Res Part B 43(1):19–35CrossRefGoogle Scholar
  7. Bliemer M, Rose J (2006) Designing stated choice experiments: the state of the art. In: 11th international conference on travel behaviour research, Kyoto, Japan, 20th August 2006Google Scholar
  8. Carmines EG, Zeller RA (1979) Reliability and validity assessment. Sage Publications, Thousand OaksCrossRefGoogle Scholar
  9. Canavari M, Nocella G, Scarpa R (2005) Stated willingness-to-pay for organic fruit and pesticide ban. J Food Prod Mark 11(3):107–134CrossRefGoogle Scholar
  10. Covey J, Robinson A, Jones-Lee M, Loomes G (2010) Responsibility scale and the valuation of rail safety. J Risk Uncertain 40(1):85–108CrossRefGoogle Scholar
  11. CSIRO (2008) Water availability in the Murray–Darling basin. In: Report to the Australian Government from the CSIRO Murray–Darling Basin Sustainable Yields ProjectGoogle Scholar
  12. Department of Water, Land and Biodiversity Conservation (2006) Lower Lakes, Coorong and Murray Mouth Asset Environmental Management Plan. Government of South Australia, AdelaideGoogle Scholar
  13. Grandjean BD, Nelson NM, Taylor PA (2009) Comparing an internet panel survey to mail and phone surveys on willingness to pay for environmental quality: a national mode test. In: 64th annual conference of the american association for public opinion researchGoogle Scholar
  14. Hatton-MacDonald DH, Morrison MD, Rose JM, Boyle KJ (2011) Valuing a multistate river: the case of the River Murray*. Aust J Agric Resour Econ 55(3):374–392CrossRefGoogle Scholar
  15. Hensher DA, Rose JM (2008) Combining RP and SP data: biases in using the nested logit ‘trick’—contrasts with flexible mixed logit incorporating panel and scale effects. J Transp Geogr 16(2):126–133CrossRefGoogle Scholar
  16. Hillman T (2008) Ecological requirements: creating a working river in the Murray–Darling basin. In: Crase L (ed) Water policy in Australia: the impact of change and uncertainty. Resources for the Future, Washington, DCGoogle Scholar
  17. Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2(8): e124. doi: 10.1371/journal.pmed.0020124
  18. Li H, Berrens R, Bohara A, Jenkins-Smith H, Silva C, Weimer D (2004) Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences the same? Appl Econ Lett 11(3):173–176CrossRefGoogle Scholar
  19. Li H, Jenkins-Smith H, Silva C, Berrens R, Herron KG (2009) Public support for reducing U.S. reliance on fossil fuels: integrating household willingness-to-pay for energy resources and development. Ecol Econ 68(3):731–742CrossRefGoogle Scholar
  20. Lindhjem H, Navrud S (2011a) Are internet surveys an alternative to face-to-face interviews in contingent valuation? Ecol Econ 70(9):1628–1637CrossRefGoogle Scholar
  21. Lindhjem H, Navrud S (2011b) Using internet in stated preference surveys: a review and comparison of survey modes. Int Rev Environ Resour Econ 5(4):309–351CrossRefGoogle Scholar
  22. Malhotra N, Krosnick JA (2007) The effect of survey mode and sampling on inferences about political attitudes and behavior: comparing the 2000 and 2004 ANES to internet surveys with nonprobability samples. Polit Anal 15(3):286–323CrossRefGoogle Scholar
  23. Marta-Pedroso C, Freitas H, Domingos T (2007) Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecol Econ 62(3–4):388–398CrossRefGoogle Scholar
  24. Murray–Darling Basin Ministerial Council (2003) Native fish strategy for the Murray–Darling basin 2003–2013. MDBC Publication No. 25/04, Murray–Darling Basin CommissionGoogle Scholar
  25. Nielsen J (2011) Use of the internet for willingness-to-pay surveys: a comparison of face-to-face and web-based interviews. Resour Energy Econ 33(1):119–129CrossRefGoogle Scholar
  26. Olsen S (2009) Choosing between internet and mail survey modes for choice experiment surveys considering non-market goods. Environ Resour Econ 44(4):591–610CrossRefGoogle Scholar
  27. Overton I, Doody T (2008) Ecosystem changes on the River Murray floodplain over the last 100 years and predictions of climate change. In: Taniguchi M, Burnett W, Fukishima Y, Haigh M, Umezawa Y (eds) From headwaters to the ocean-hydrological changes and watershed management. Taylor and Frances Group, LondonGoogle Scholar
  28. Paton D (2000) Bird ecology in the Coorong and lakes region. In: Jensen A, Good M, Tucker P, Long M (eds) River murray barrages environmental flows. Murray–Darling Basin Commission, CanberraGoogle Scholar
  29. Paton D, Rogers B, Hill M, Bailey C, Ziembicki M (2009) Temporal changes to spatially stratified waterbird communities of the Coorong, South Australia: implications for the management of heterogeneous wetlands. Anim Conserv 12:408–417CrossRefGoogle Scholar
  30. Poe G, Giraud K, Loomis J (2005) Computational methods for measuring the difference of empirical distributions. Am J Agric Econ 87(2):353–365CrossRefGoogle Scholar
  31. Rose J, Bliemer M, Hensher D, Collins AT (2008) Constructing efficient stated choice experiments allowing for differences in error variances across subsets of alternatives. Transp Res Part B 42(4):395–406CrossRefGoogle Scholar
  32. Shih T-H, Fan X (2009) Comparing response rates in e-mail and paper surveys: a meta-analysis. Educ Res Rev 4(1):26–40CrossRefGoogle Scholar
  33. Small GW, Moody TD, Siddarth P, Bookheimer SY (2009) Your brain on Google: patterns of cerebral activation during internet searching. Am J Geriatr psychiatry 17(2):116–126CrossRefGoogle Scholar
  34. Tourangeau R, Ripps LJ, Rasinski KA (2000) The psychology of survey responses. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  35. Train K (2009) Discrete choice methods with simulation. Cambridge University Press, CambridgeGoogle Scholar
  36. Train K (2010) Discrete choice methods with simulation, 2nd edn. Cambridge University Press, CambridgeGoogle Scholar
  37. Vossler CA, Evans MF (2009) Bridging the gap between the field and the lab: environmental goods, policy maker input, and consequentiality. J Environ Econ Manage 58(3):338–345CrossRefGoogle Scholar
  38. van der Heide CM, van den Bergh JCJM, van Ierland EC, Nunes PALD (2008) Economic valuation of habitat defragmentation: a study of the Veluwe, the Netherlands. Ecol Econ 67(2):205–216CrossRefGoogle Scholar
  39. Vossler CA, Doyon M, Rondeau D (2012) Truth in consequentiality: theory and field evidence on discrete choice experiments. Am Econ J Microecon 4(4):145–171CrossRefGoogle Scholar
  40. Windle J, Rolfe J (2011) Comparing responses from internet and paper- based collection methods in more complex stated preference environmental valuation surveys. Econ Anal Policy 41(1):83–97CrossRefGoogle Scholar
  41. Winter N (2008) SURVWGT: Stata module to create and manipulate survey weights
  42. Yeager DS, Krosnick JA, Chang LC, Javitz HS, Levendusky MS, Simpser A, Wang R (2011) Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Unpublished paper, Stanford UniversityGoogle Scholar
  43. Yuan Y, Boyle KJ, You W (2015) Sample selection, individual heterogeneity and regional heterogeneity in valuing farmland conservation easements. Land Econ (forthcoming)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Kevin J. Boyle
    • 1
    Email author
  • Mark Morrison
    • 2
  • Darla Hatton MacDonald
    • 3
  • Roderick Duncan
    • 4
  • John Rose
    • 5
  1. 1.Program in Real EstateVirginia TechBlacksburgUSA
  2. 2.School of Management and MarketingBathurstAustralia
  3. 3.Natural Resource Economics and Decision Sciences, Ecosystem SciencesCSIROGlen OsmondAustralia
  4. 4.School of Accounting and FinanceCharles Sturt UniversityBathurstAustralia
  5. 5.University of South AustraliaNorth SydneyAustralia

Personalised recommendations