Happy Talk: Mode of Administration Effects on Subjective Well-Being

Abstract

There is increasing interest in subjective well-being (SWB) both in academic and policy circles. As a result, considerable research efforts are now being directed at the validity and reliability of SWB measures. This study examines how SWB reports differ by survey mode. Using data from the April 2011 to March 2012 Annual Population Survey in the UK we find that individuals consistently report higher SWB over the phone compared to face-to-face interviews. We also show that the determinants of SWB differ significantly by mode, with life circumstances tending to matter more in face-to-face interviews. These results have substantial implications for research and policy purposes.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    Although it is worth noting that omitting calendar controls has only a marginal impact on the remaining coefficients.

  2. 2.

    Further ordered probit estimations by mode support the evidence based on OLS, presented in Table 6. These results can be made available upon request.

References

  1. Anand, P., Krishnakumar, J., & Tran, N. B. (2011). Measuring welfare: Latent variable models for happiness and capabilities in the presence of unobservable heterogeneity. Journal of Public Economics, 95, 205–215.

    Article  Google Scholar 

  2. Bertrand, M., & Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91, 67–72.

    Article  Google Scholar 

  3. Blanchflower, D. G., & Oswald, A. J. (2004). Well-being over time in Britain and the USA. Journal of Public Economics, 88, 1359–1386.

    Article  Google Scholar 

  4. Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27, 281–291.

    Article  Google Scholar 

  5. Breunig, R., & McKibbin, R. (2011). The effect of survey design on household reporting of financial difficulty. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174, 991–1005.

    Article  Google Scholar 

  6. Buskirk, T. D., & Stein, K. D. (2008). Telephone vs. mail survey gives different SF-36 quality-of-life scores among cancer survivors. Journal of Clinical Epidemiology, 61, 1049–1055.

    Article  Google Scholar 

  7. Conti, G., & Pudney, S. (2011). Survey design and the analysis of satisfaction. The Review of Economics and Statistics, 93, 1087–1093.

    Article  Google Scholar 

  8. Di Tella, R., & MacCulloch, R. (2006). Some uses of happiness data in economics. Journal of Economic Perspectives, 20, 25–46.

    Article  Google Scholar 

  9. Dillman, D. A. (2000). Mail and internet surveys: The tailored design method. New York: Wiley.

    Google Scholar 

  10. Dolan, P. (2011). Using happiness to value health. UK: Office of Health Economics.

    Google Scholar 

  11. Dolan, P., & Metcalfe, R. (2011). Comparing measures of subjective well-being and views about the role they should play in policy. Newport: Office for National Statistics.

    Google Scholar 

  12. Dolan, P., & Metcalfe, R. (2012). Measuring subjective wellbeing: Recommendations on measures for use by National Governments. Journal of Social Policy, 41, 409–427.

    Article  Google Scholar 

  13. Dolan, P., Peasgood, T., & White, M. (2008). Do we really know what makes us happy? A review of the economic literature on the factors associated with subjective well-being. Journal of Economic Psychology, 29, 94–122.

    Article  Google Scholar 

  14. Evans, M., Kessler, D., Lewis, G., Peters, T. J., & Sharp, D. (2004). Assessing mental health in primary care research using standardized scales: Can it be carried out over the phone? Psychological Medicine, 34, 157–162.

    Article  Google Scholar 

  15. Ferrer-i-Carbonell, A., & Frijters, P. (2004). How important is methodology for the estimates of the determinants of happiness? Economic Journal, 114, 641–659.

    Article  Google Scholar 

  16. Fowler, F. J., Gallagher, P. M., & Nederend, S. (1999). Comparing telephone and mail responses to the CAHPS survey instrument: Consumer assessment of health plans study. Medical Care, 37, 41–49.

    Article  Google Scholar 

  17. Frey, B. S., & Stutzer, A. (2002). What can economists learn from happiness research? Journal of Economic Literature, 40, 402–435.

    Article  Google Scholar 

  18. Frijters, P., Haisken-DeNew, J. P., & Shields, M. A. (2004). Investigating the patterns and determinants of life satisfaction in Germany following reunification. Journal of Human Resources, 39, 649–674.

    Article  Google Scholar 

  19. Fujiwara, D., & Campbell, R. (2011). Valuation techniques for social cost-benefit analysis: Stated preference, revealed preference and subjective well-being approaches. A discussion of the current issues, HM Treasury and Department for Work and Pensions.

  20. Groves, R. M. (2004). Survey errors and survey costs. Hoboken, NJ: Wiley.

    Google Scholar 

  21. Hanmer, J., Hays, R. D., & Fryback, D. G. (2007). Mode of administration is important in US national estimates of health-related quality of life. Medical Care, 45, 1171–1179.

    Article  Google Scholar 

  22. Hays, R. D., Kim, S., Spritzer, K. L., Kaplan, R. M., Tally, S., Feeny, D., et al. (2009). Effects of mode and order of administration on generic health-related quality of life scores. Value in Health, 12, 1035–1039.

    Article  Google Scholar 

  23. Heffetz, O., & Rabin, M. (2013). Conclusions regarding cross-group differences in happiness depend on difficulty of reaching respondents. American Economic Review, 103, 3001–3021.

    Article  Google Scholar 

  24. Holbrook, A. L., Green, M. C., & Krosnick, J. A. (2003). Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparison of respondent satisficing and social desirability bias. Public Opinion Quarterly, 67, 79–125.

    Article  Google Scholar 

  25. Kaplan, C. P., Hilton, J. F., Park-Tanjasiri, S., & Perez-Stable, E. J. (2001). The effect of data collection mode on smoking attitudes and behaviour in young African American and Latina women. Face-to-face interview versus self-administered questionnaires. Evaluation Review, 25, 454–473.

    Article  Google Scholar 

  26. Kavetsos, G., Dimitriadou, M., & Dolan, P. (2014). Measuring happiness: Context matters. Applied Economics Letters, 21, 308–311.

    Article  Google Scholar 

  27. Kavetsos, G., & Szymanski, S. (2010). National wellbeing and international sports events. Journal of Economic Psychology, 31, 158–171.

    Article  Google Scholar 

  28. Knabe, A., Rätzel, S., Schöb, R., & Weimann, J. (2010). Dissatisfied with life but having a good day: Time-use and well-being of the unemployed. Economic Journal, 120, 867–889.

    Article  Google Scholar 

  29. Krueger, A. B., & Schkade, D. A. (2008). The reliability of subjective well-being measures. Journal of Public Economics, 92, 1833–1845.

    Article  Google Scholar 

  30. Leggett, C. G., Kleckner, N. S., Boyle, K. J., Duffield, J. W., & Mitchell, R. C. (2003). Social desirability bias in contingent valuation surveys administered through in-person interviews. Land Economics, 79, 561–575.

    Article  Google Scholar 

  31. Levinson, A. (2012). Valuing public goods using happiness data: The case of air quality. Journal of Public Economics, 96, 869–880.

    Article  Google Scholar 

  32. LFS User Guide. (2011). LFS background and methodology (Vol. 1). Newport: Office for National Statistics.

    Google Scholar 

  33. Li, C., Ford, E. S., Zhao, G., Tsai, J., & Balluz, L. S. (2012). A comparison of depression prevalence estimates measured by the Patient Health Questionnaire with two administration modes: Computer-assisted telephone interviewing versus computer-assisted personal interviewing. International Journal of Public Health, 57, 225–233.

    Article  Google Scholar 

  34. Lindhjem, H., & Navrud, S. (2011). Are internet surveys an alternative to face-to-face interviews in contingent valuation? Ecological Economics, 70, 1628–1637.

    Article  Google Scholar 

  35. Luechinger, S., & Raschky, P. A. (2009). Valuing flood disasters using the life satisfaction approach. Journal of Public Economics, 93, 620–633.

    Article  Google Scholar 

  36. Lyons, R. A., Wareham, K., Lucas, M., Price, D., Williams, J., & Hutchings, H. A. (1999). SF-36 scores vary by method of administration: Implications for study design. Journal of Public Health Medicine, 21, 41–45.

    Article  Google Scholar 

  37. Maguire, K. B. (2009). Does mode matter? A comparison of telephone, mail, and in-person treatments in contingent valuation surveys. Journal of Environmental Management, 90, 3528–3533.

    Article  Google Scholar 

  38. Marta-Pedroso, C., Freitas, H., & Domingos, T. (2007). Testing for survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews. Ecological Economics, 62, 388–398.

    Article  Google Scholar 

  39. McHorney, C. A., Kosinski, M., & Ware, J. E. (1994). Comparisons of the costs and quality of norms of the SF-36 health survey collected by mail versus telephone interview: Results from a national survey. Medical Care, 32, 551–567.

    Article  Google Scholar 

  40. McMorris, B. J., Petrie, R. S., Catalano, R. F., Fleming, C. B., Haggerty, K. P., & Abbott, R. D. (2009). Use of web and in-person survey modes to gather data from young adults on sex and drug use: An evaluation of cost, time, and survey error based on a randomized mixed-mode design. Evaluation Review, 33, 138–158.

    Article  Google Scholar 

  41. Nandi, A., & Platt, L. (2011). Effect of interview modes on measurement of identity. Understanding Society Working Paper Series. No. 2011-02.

  42. National Research Council. (2013). Subjective well-being: Measuring happiness, suffering, and other dimensions of experience. Washington D.C: The National Academies Press.

    Google Scholar 

  43. NEF. (2012). Well-being patterns uncovered: An analysis of UK data. London: New Economics Foundation.

    Google Scholar 

  44. OECD. (2013). OECD guidelines on measuring subjective well-being. Paris: OECD Publishing.

    Google Scholar 

  45. Olsen, S. B. (2009). Choosing between internet and mail survey modes for choice experiment surveys considering non-market goods. Environmental & Resource Economics, 44, 591–610.

    Article  Google Scholar 

  46. ONS. (2012). First ONS annual experimental subjective well-being results. Newport: Office for National Statistics.

    Google Scholar 

  47. Oswald, A. J., & Powdthavee, N. (2008). Does happiness adapt? A longitudinal study of disability with implications for economists and judges. Journal of Public Economics, 92, 1061–1077.

    Article  Google Scholar 

  48. Perkins, J. J., & Sanson-Fisher, R. W. (1998). An examination of self- and telephone- administered modes of administration for the Australian SF-36. Journal of Clinical Epidemiology, 51, 969–973.

    Article  Google Scholar 

  49. Presser, S., & Stinson, L. (1998). Data collection mode and social desirability bias in self-reported religious attendance. American Sociological Review, 63, 137–145.

    Article  Google Scholar 

  50. Sakshaug, J. W., Yan, T., & Tourangeau, R. (2010). Nonresponse error, measurement error, and mode of data collection: Tradeoffs in a multi-mode survey of sensitive and non-sensitive items. Public Opinion Quarterly, 74, 907–933.

    Article  Google Scholar 

  51. Schwarz, N., Strack, F., Hippler, H.-J., & Bishop, G. (1991). The impact of administration mode on response effects in survey measurement. Applied Cognitive Psychology, 5, 193–212.

    Article  Google Scholar 

  52. Stiglitz, J., Sen, A., & Fitoussi, J. (2009). Report by the commission on the measurement of economic performance and social progress. Paris: Commission on the Measurement of Economic Performance and Social Progress.

    Google Scholar 

  53. Sutton, S., & Davidson, R. (1997). Prefrontal brain symmetry: A biological substrate of the behavioural approach and inhibition systems. Psychological Science, 8, 204–210.

    Article  Google Scholar 

  54. Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275–304.

    Article  Google Scholar 

  55. Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883.

    Article  Google Scholar 

  56. Treasury, H. M. (2011). The Green Book: Appraisal and evaluation in central government. London: Treasury Guidance.

    Google Scholar 

  57. Urry, H., Nitschke, J., Dolski, I., Jackson, D., Dalton, K., Mueler, C., et al. (2004). Making a life worth living. Psychological Science, 15, 367–372.

    Article  Google Scholar 

  58. Van Praag, B. M. S., & Baarsma, B. E. (2005). Using happiness surveys to value intangibles: The case of airport noise. Economic Journal, 115, 224–246.

    Article  Google Scholar 

Download references

Acknowledgments

We thank Femke De Keulenaer, Daniel Fujiwara, Stephen Hicks, Danny Kahneman, Laura Kudrna, Kate Laffan, Robert Metcalfe, Michael Norton, Tessa Peasgood and participants at the “New scholarship on happiness” conference held at Duke University, two anonymous referees and the editor for useful comments and suggestions that improved this paper considerably. Further thanks go to the UK Data Archive for provision of the APS data, and Necla Acik-Toprak of the ESDS Government, and Simon Woodsford and Dawn Snape of the ONS Social Survey Division for providing clarifications on the APS survey design.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Georgios Kavetsos.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dolan, P., Kavetsos, G. Happy Talk: Mode of Administration Effects on Subjective Well-Being. J Happiness Stud 17, 1273–1291 (2016). https://doi.org/10.1007/s10902-015-9642-8

Download citation

Keywords

  • Subjective well-being
  • Happiness
  • Survey mode
  • Valuation

JEL Classification

  • D60
  • I30