There is increasing interest in subjective well-being (SWB) both in academic and policy circles. As a result, considerable research efforts are now being directed at the validity and reliability of SWB measures. This study examines how SWB reports differ by survey mode. Using data from the April 2011 to March 2012 Annual Population Survey in the UK we find that individuals consistently report higher SWB over the phone compared to face-to-face interviews. We also show that the determinants of SWB differ significantly by mode, with life circumstances tending to matter more in face-to-face interviews. These results have substantial implications for research and policy purposes.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Although it is worth noting that omitting calendar controls has only a marginal impact on the remaining coefficients.
Further ordered probit estimations by mode support the evidence based on OLS, presented in Table 6. These results can be made available upon request.
Anand, P., Krishnakumar, J., & Tran, N. B. (2011). Measuring welfare: Latent variable models for happiness and capabilities in the presence of unobservable heterogeneity. Journal of Public Economics, 95, 205–215.
Bertrand, M., & Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91, 67–72.
Blanchflower, D. G., & Oswald, A. J. (2004). Well-being over time in Britain and the USA. Journal of Public Economics, 88, 1359–1386.
Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27, 281–291.
Breunig, R., & McKibbin, R. (2011). The effect of survey design on household reporting of financial difficulty. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174, 991–1005.
Buskirk, T. D., & Stein, K. D. (2008). Telephone vs. mail survey gives different SF-36 quality-of-life scores among cancer survivors. Journal of Clinical Epidemiology, 61, 1049–1055.
Conti, G., & Pudney, S. (2011). Survey design and the analysis of satisfaction. The Review of Economics and Statistics, 93, 1087–1093.
Di Tella, R., & MacCulloch, R. (2006). Some uses of happiness data in economics. Journal of Economic Perspectives, 20, 25–46.
Dillman, D. A. (2000). Mail and internet surveys: The tailored design method. New York: Wiley.
Dolan, P. (2011). Using happiness to value health. UK: Office of Health Economics.
Dolan, P., & Metcalfe, R. (2011). Comparing measures of subjective well-being and views about the role they should play in policy. Newport: Office for National Statistics.
Dolan, P., & Metcalfe, R. (2012). Measuring subjective wellbeing: Recommendations on measures for use by National Governments. Journal of Social Policy, 41, 409–427.
Dolan, P., Peasgood, T., & White, M. (2008). Do we really know what makes us happy? A review of the economic literature on the factors associated with subjective well-being. Journal of Economic Psychology, 29, 94–122.
Evans, M., Kessler, D., Lewis, G., Peters, T. J., & Sharp, D. (2004). Assessing mental health in primary care research using standardized scales: Can it be carried out over the phone? Psychological Medicine, 34, 157–162.
Ferrer-i-Carbonell, A., & Frijters, P. (2004). How important is methodology for the estimates of the determinants of happiness? Economic Journal, 114, 641–659.
Fowler, F. J., Gallagher, P. M., & Nederend, S. (1999). Comparing telephone and mail responses to the CAHPS survey instrument: Consumer assessment of health plans study. Medical Care, 37, 41–49.
Frey, B. S., & Stutzer, A. (2002). What can economists learn from happiness research? Journal of Economic Literature, 40, 402–435.
Frijters, P., Haisken-DeNew, J. P., & Shields, M. A. (2004). Investigating the patterns and determinants of life satisfaction in Germany following reunification. Journal of Human Resources, 39, 649–674.
Fujiwara, D., & Campbell, R. (2011). Valuation techniques for social cost-benefit analysis: Stated preference, revealed preference and subjective well-being approaches. A discussion of the current issues, HM Treasury and Department for Work and Pensions.
Groves, R. M. (2004). Survey errors and survey costs. Hoboken, NJ: Wiley.
Hanmer, J., Hays, R. D., & Fryback, D. G. (2007). Mode of administration is important in US national estimates of health-related quality of life. Medical Care, 45, 1171–1179.
Hays, R. D., Kim, S., Spritzer, K. L., Kaplan, R. M., Tally, S., Feeny, D., et al. (2009). Effects of mode and order of administration on generic health-related quality of life scores. Value in Health, 12, 1035–1039.
Heffetz, O., & Rabin, M. (2013). Conclusions regarding cross-group differences in happiness depend on difficulty of reaching respondents. American Economic Review, 103, 3001–3021.
Holbrook, A. L., Green, M. C., & Krosnick, J. A. (2003). Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparison of respondent satisficing and social desirability bias. Public Opinion Quarterly, 67, 79–125.
Kaplan, C. P., Hilton, J. F., Park-Tanjasiri, S., & Perez-Stable, E. J. (2001). The effect of data collection mode on smoking attitudes and behaviour in young African American and Latina women. Face-to-face interview versus self-administered questionnaires. Evaluation Review, 25, 454–473.
Kavetsos, G., Dimitriadou, M., & Dolan, P. (2014). Measuring happiness: Context matters. Applied Economics Letters, 21, 308–311.
Kavetsos, G., & Szymanski, S. (2010). National wellbeing and international sports events. Journal of Economic Psychology, 31, 158–171.
Knabe, A., Rätzel, S., Schöb, R., & Weimann, J. (2010). Dissatisfied with life but having a good day: Time-use and well-being of the unemployed. Economic Journal, 120, 867–889.
Krueger, A. B., & Schkade, D. A. (2008). The reliability of subjective well-being measures. Journal of Public Economics, 92, 1833–1845.
Leggett, C. G., Kleckner, N. S., Boyle, K. J., Duffield, J. W., & Mitchell, R. C. (2003). Social desirability bias in contingent valuation surveys administered through in-person interviews. Land Economics, 79, 561–575.
Levinson, A. (2012). Valuing public goods using happiness data: The case of air quality. Journal of Public Economics, 96, 869–880.
LFS User Guide. (2011). LFS background and methodology (Vol. 1). Newport: Office for National Statistics.
Li, C., Ford, E. S., Zhao, G., Tsai, J., & Balluz, L. S. (2012). A comparison of depression prevalence estimates measured by the Patient Health Questionnaire with two administration modes: Computer-assisted telephone interviewing versus computer-assisted personal interviewing. International Journal of Public Health, 57, 225–233.
Lindhjem, H., & Navrud, S. (2011). Are internet surveys an alternative to face-to-face interviews in contingent valuation? Ecological Economics, 70, 1628–1637.
Luechinger, S., & Raschky, P. A. (2009). Valuing flood disasters using the life satisfaction approach. Journal of Public Economics, 93, 620–633.
Lyons, R. A., Wareham, K., Lucas, M., Price, D., Williams, J., & Hutchings, H. A. (1999). SF-36 scores vary by method of administration: Implications for study design. Journal of Public Health Medicine, 21, 41–45.
Maguire, K. B. (2009). Does mode matter? A comparison of telephone, mail, and in-person treatments in contingent valuation surveys. Journal of Environmental Management, 90, 3528–3533.
Marta-Pedroso, C., Freitas, H., & Domingos, T. (2007). Testing for survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews. Ecological Economics, 62, 388–398.
McHorney, C. A., Kosinski, M., & Ware, J. E. (1994). Comparisons of the costs and quality of norms of the SF-36 health survey collected by mail versus telephone interview: Results from a national survey. Medical Care, 32, 551–567.
McMorris, B. J., Petrie, R. S., Catalano, R. F., Fleming, C. B., Haggerty, K. P., & Abbott, R. D. (2009). Use of web and in-person survey modes to gather data from young adults on sex and drug use: An evaluation of cost, time, and survey error based on a randomized mixed-mode design. Evaluation Review, 33, 138–158.
Nandi, A., & Platt, L. (2011). Effect of interview modes on measurement of identity. Understanding Society Working Paper Series. No. 2011-02.
National Research Council. (2013). Subjective well-being: Measuring happiness, suffering, and other dimensions of experience. Washington D.C: The National Academies Press.
NEF. (2012). Well-being patterns uncovered: An analysis of UK data. London: New Economics Foundation.
OECD. (2013). OECD guidelines on measuring subjective well-being. Paris: OECD Publishing.
Olsen, S. B. (2009). Choosing between internet and mail survey modes for choice experiment surveys considering non-market goods. Environmental & Resource Economics, 44, 591–610.
ONS. (2012). First ONS annual experimental subjective well-being results. Newport: Office for National Statistics.
Oswald, A. J., & Powdthavee, N. (2008). Does happiness adapt? A longitudinal study of disability with implications for economists and judges. Journal of Public Economics, 92, 1061–1077.
Perkins, J. J., & Sanson-Fisher, R. W. (1998). An examination of self- and telephone- administered modes of administration for the Australian SF-36. Journal of Clinical Epidemiology, 51, 969–973.
Presser, S., & Stinson, L. (1998). Data collection mode and social desirability bias in self-reported religious attendance. American Sociological Review, 63, 137–145.
Sakshaug, J. W., Yan, T., & Tourangeau, R. (2010). Nonresponse error, measurement error, and mode of data collection: Tradeoffs in a multi-mode survey of sensitive and non-sensitive items. Public Opinion Quarterly, 74, 907–933.
Schwarz, N., Strack, F., Hippler, H.-J., & Bishop, G. (1991). The impact of administration mode on response effects in survey measurement. Applied Cognitive Psychology, 5, 193–212.
Stiglitz, J., Sen, A., & Fitoussi, J. (2009). Report by the commission on the measurement of economic performance and social progress. Paris: Commission on the Measurement of Economic Performance and Social Progress.
Sutton, S., & Davidson, R. (1997). Prefrontal brain symmetry: A biological substrate of the behavioural approach and inhibition systems. Psychological Science, 8, 204–210.
Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275–304.
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883.
Treasury, H. M. (2011). The Green Book: Appraisal and evaluation in central government. London: Treasury Guidance.
Urry, H., Nitschke, J., Dolski, I., Jackson, D., Dalton, K., Mueler, C., et al. (2004). Making a life worth living. Psychological Science, 15, 367–372.
Van Praag, B. M. S., & Baarsma, B. E. (2005). Using happiness surveys to value intangibles: The case of airport noise. Economic Journal, 115, 224–246.
We thank Femke De Keulenaer, Daniel Fujiwara, Stephen Hicks, Danny Kahneman, Laura Kudrna, Kate Laffan, Robert Metcalfe, Michael Norton, Tessa Peasgood and participants at the “New scholarship on happiness” conference held at Duke University, two anonymous referees and the editor for useful comments and suggestions that improved this paper considerably. Further thanks go to the UK Data Archive for provision of the APS data, and Necla Acik-Toprak of the ESDS Government, and Simon Woodsford and Dawn Snape of the ONS Social Survey Division for providing clarifications on the APS survey design.
About this article
Cite this article
Dolan, P., Kavetsos, G. Happy Talk: Mode of Administration Effects on Subjective Well-Being. J Happiness Stud 17, 1273–1291 (2016). https://doi.org/10.1007/s10902-015-9642-8
- Subjective well-being
- Survey mode