Skip to main content
SpringerLink
Log in

Dissatisfied with life or with being interviewed? Happiness and the motivation to participate in a survey

  • Original Paper
  • Published:
Social Choice and Welfare Aims and scope Submit manuscript

Abstract

People with little motivation to participate in surveys can affect empirical research when they abstain from but also when they actually participate in interviews. This paper investigates whether happiness data are susceptible to such measurement bias. Evidence from the German Socio-Economic Panel Study (SOEP) reveals a strong relationship between self-reported life satisfaction and several indicators of respondent motivation, such as subsequent panel attrition. One explanation for this finding is that respondents on the margin of participation truly have lower life satisfaction. Alternatively, their low motivation may be the cause for an underreporting of life satisfaction. To learn more about this, an instrumental variable approach identifies future panel quitters with low motivation by using the occurrence of interviewer attrition in the year after the interview. The results of this analysis suggest that self-reported life satisfaction declines because of low respondent motivation. A discussion of the implications for analyses of happiness data underscores the potential importance of respondent motivation regardless of the explanation for why interviewees with low motivation report lower life satisfaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. One example is the debate on the link between happiness and age. See e.g. Frijters and Beatton (2012); Kassenboehmer and Haisken-DeNew (2012); Wunder et al. (2013) and Baetschmann (2014). Apart from the concern about data validity, another reason for researchers to investigate and to better understand survey factors lies in the potential for causal identification strategies. Factors such as the day of the interview or the interview mode have been used as instruments for well-being (Wunder and Heineck 2013). Similarly, there are increasing numbers of quasi-experimental investigations on people’s well-being exploiting interview dates in survey data (e.g. Metcalfe et al. 2011; Chadi 2015; Goebel et al. 2015; Schueller 2016).

  2. See e.g. the data documentation on panel attrition in the SOEP (Kroh 2009, 2010, 2012).

  3. Only a few researchers so far use information on the number of contacts with potential respondents to discuss the role of difficult-to-reach survey participants in data collection. For instance, Kreuter et al. (2010) investigate data on benefit recipients and look at the role played by number of calls, and Heffetz and Reeves (2019) compare the responses of easy- and difficult-to-reach respondents in various governmental surveys. Heffetz and Rabin (2013) introduce the idea of using contact attempts to discuss potential nonresponse bias in happiness data.

  4. Exemplary happiness studies based on SOEP data are from Frijters et al. (2004), Clark et al. (2008) and Luechinger et al. (2010). Note that, in line with most of the studies in the research field, the present paper treats the terms happiness, life satisfaction and (subjective) well-being as interchangeable.

  5. When there is contact via mail only, so that questionnaires are filled out without interviewer presence, there is usually no data available on the contact attempts. Thus, the survey mode of “self-written questionnaires by mail” has to be left out of the analysis, although motivation can be expected to be very low. As the SOEP offers the “mail” mode as a last opportunity for reluctant individuals, it is interesting to note that life satisfaction is very low in this group (Chadi 2012), which could be seen as indicative of a link between respondent motivation and reported happiness. For an overview on the interview modes analyzed here, see Table 1.

  6. To tackle potential concerns regarding the ordinal character of the dependent variable, ordered probit models can be implemented to check the results’ robustness. Furthermore, due to the hierarchical data structure with individual, household and interviewer levels, using mixed linear models is an option. Note that all further analyses which are discussed but not presented in the paper are available from the author upon request.

  7. A separate investigation of an expanded SOEP dataset reveals that the incidence of a current interviewer change is positively related to the probability of interviewee attrition in the subsequent year. Note that this incidence of a change in the person conducting the interview is also positively related to reported life satisfaction (Chadi 2013b).

  8. Note that the SOEP data used in this paper do not distinguish between observations in the two federal states of Saarland and Rhineland-Palatinate until the wave of 1999, making it necessary to edit all original regional data prior to merging the datasets.

  9. The idea that busy, but not necessarily unhappy, types of people drive the findings receives additional support in further subgroup analyses. While there is no effect heterogeneity across income levels, the effects are stronger for younger people than for people above the median age. Furthermore, the effects are driven by married people living together with their spouses as well as by respondents with at least one child in the household.

References

  • Abowd JM, Stinson MH (2013) Estimating measurement error in annual job earnings: a comparison of survey and administrative data. Rev Econ Stat 95:1451–1467

    Article  Google Scholar 

  • Angelini V, Cavapozzi D, Corazzini L, Paccagnella O (2014) Do danes and Italians rate life satisfaction in the same way? Using Vignettes to correct for individual-specific scale biases. Oxf Bull Econ Stat 76:643–666

    Article  Google Scholar 

  • Angelini V, Bertoni M, Corazzini L (2017) Unpacking the determinants of life satisfaction: a survey experiment. J R Stat Soc Ser A 180:225–246

    Article  Google Scholar 

  • Baetschmann G (2014) Heterogeneity in the relationship between happiness and age: evidence from the German Socio-economic panel. Ger Econ Rev 15:393–410

    Article  Google Scholar 

  • Behr A, Bellgardt E, Rendtel U (2005) Extent and determinants of panel attrition in the european community household panel. Eur Sociol Rev 21:489–512

    Article  Google Scholar 

  • Benjamin DJ, Heffetz O, Kimball MS, Rees-Jones A (2012) What do you think would make you happier? What do you think you would choose? Am Econ Rev 102:2083–2110

    Article  Google Scholar 

  • Benjamin DJ, Heffetz O, Kimball MS, Rees-Jones A (2014) Can marginal rates of substitution be inferred from happiness data? Evidence from residency choices. Am Econ Rev 104:3498–3528

    Article  Google Scholar 

  • Campanelli P, O’muircheartaigh C (1999) Interviewers, interviewer continuity, and panel survey nonresponse. Qual Quant 33:59–76

    Article  Google Scholar 

  • Campanelli P, O’muircheartaigh C (2002) The importance of experimental control in testing the impact of interviewer continuity on panel survey nonresponse. Qual Quant 36:129–144

    Article  Google Scholar 

  • Chadi A (2010) How to distinguish voluntary from involuntary unemployment: on the relationship between the willingness to work and unemployment-induced unhappiness. Kyklos 63:317–329

    Article  Google Scholar 

  • Chadi A (2012) I would really love to participate in your survey! bias problems in the measurement of well-being. Econ Bull 32:3111–3119

    Google Scholar 

  • Chadi A (2013a) Third person effects in interview responses on life satisfaction. J Appl Soc Sci Stud 133:323–333

    Google Scholar 

  • Chadi A (2013b) the role of interviewer encounters in panel responses on life satisfaction. Econ Lett 121:550–554

    Article  Google Scholar 

  • Chadi A (2015) Concerns about the Euro and happiness in Germany during times of crisis. Eur J Polit Econ 40:126–146

    Article  Google Scholar 

  • Chadi A (2016) Identification of Attrition Bias Using Different Types of Panel Refreshments” IAAEU Discussion Paper 02/2016, Trier

  • Clark AE, Oswald AJ (1994) Unhappiness and unemployment. Econ J 104:648–659

    Article  Google Scholar 

  • Clark AE, Diener E , Georgellis Y, Lucas RE (2008) Lags and leads in life satisfaction: a test of the baseline hypothesis. Econ J 118:F222–F243

    Article  Google Scholar 

  • Conti G, Pudney S (2011) Survey design and the analysis of satisfaction. Rev Econ Stat 93:1087–1093

    Article  Google Scholar 

  • Contoyannis P, Jones AM, Rice N (2004) The dynamics of health in the british household panel survey. J Appl Econom 19:473–503

    Article  Google Scholar 

  • Deuchert E, Huber M (2017) A cautionary tale about control variables in IV estimation. Oxf Bull Econ Stat 79:411–425

    Article  Google Scholar 

  • Easterlin RA (1995) Will raising the income of all increase the happiness of all? J Econ Behav Organ 27:35–47

    Article  Google Scholar 

  • Ferrer-i-Carbonell A, Frijters P (2004) How important is methodology for the estimates of the determinants of happiness? Econ J 114:641–659

    Article  Google Scholar 

  • Fleurbaey M (2009) Beyond GDP: the quest for a measure of social welfare. J Econ Lit 47:1029–1075

    Article  Google Scholar 

  • Frey BS (2008) Happiness: A revolution in economics. MIT Press, London

    Book  Google Scholar 

  • Frey BS, Stutzer A (2002) Happiness and economics. How the economy and institutions affect well-being. Princeton University Press, Princeton

    Google Scholar 

  • Frey BS, Stutzer A (2012) The use of happiness research for public policy. Soc Choice Welf 38:659–674

    Article  Google Scholar 

  • Frijters P, Beatton T (2012) The mystery of the U-shaped relationship between happiness and age. J Econ Behav Organ 82:525–542

    Article  Google Scholar 

  • Frijters P, Haisken-DeNew JP, Shields MA (2004) Investigating the patterns and determinants of life satisfaction in germany following reunification. J Hum Res 34:649–676

    Article  Google Scholar 

  • Gerstorf D, Ram N, Mayraz G, Hidajat M, Lindenberger U, Wagner GG, Schupp J (2010) Late-life decline in well-being across adulthood in Germany, the United Kingdom, and the United States: something is seriously wrong at the end of life. Psychol Aging 25:477–485

    Article  Google Scholar 

  • Goebel J, Krekel C, Tiefenbach T, Ziebarth NR (2015) Natural disaster, policy action, and mental well-being: the case of how natural disasters can affect environmental concerns, risk aversion, and even politics: evidence from Fukushima and three European countries. J Popul Econ 28:1137–1180

    Article  Google Scholar 

  • Golsteyn BH, Hirsch S (2019) Are estimates of intergenerational mobility biased by non-response? Evidence from the Netherlands. Soc Choice Welf 52:29–63

    Article  Google Scholar 

  • Heffetz O, Rabin M (2013) Conclusions regarding cross-group differences in happiness depend on difficulty of reaching respondents. Am Econ Rev 103:3001–3021

    Article  Google Scholar 

  • Heffetz O, Reeves DB (2019) Difficulty to reach respondents and nonresponse bias: evidence from large government surveys. Rev Econ Stat 101:176–191

    Article  Google Scholar 

  • Hill DH, Willis RJ (2001) Reducing panel attrition: a search for effective policy instruments. J Hum Res 36:416–438

    Article  Google Scholar 

  • Jones AM, Koolman X, Rice N (2006) Health-related non-response in the British household panel survey and european community household panel: using inverse-probability-weighted estimators in non-linear models. J R Stat Soc Ser A 169:543–569

    Article  Google Scholar 

  • Kapteyn A, Smith JP, van Soest A (2010) Life satisfaction. In: Diener E, Kahneman D, Helliwell J (eds) International differences in well-being, pp 70–107. Oxford University Press, Oxford

    Google Scholar 

  • Kassenboehmer SC, Haisken-DeNew JP (2009) You’re fired! The causal negative effect of entry unemployment on life satisfaction. Econ J 119:448–462

    Article  Google Scholar 

  • Kassenboehmer SC, Haisken-DeNew JP (2012) Heresy or enlightenment? The well-being age u–shape effect is flat. Econ Lett 117:235–238

    Article  Google Scholar 

  • Kreuter F, Mueller G, Trappmann M (2010) Nonresponse and Measurement error in employment research: making use of administrative data. Pub Opin Q 74:880–906

    Article  Google Scholar 

  • Kroh M (2009) Documentation of Sample Sizes and Panel Attrition in the German Socio Economic Panel (SOEP) (1984 until 2008), DIW Data Documentation No. 47

  • Kroh M (2010) Documentation of Sample Sizes and Panel Attrition in the German Socio Economic Panel (SOEP) (1984 until 2009), DIW Data Documentation No. 50

  • Kroh M (2012) Documentation of sample sizes and panel attrition in the German Socio Economic Panel (SOEP) (1984 until 2011), DIW Data Documentation No. 66

  • Laurie H, Smith RA, Scott L (1999) Strategies for reducing nonresponse in a longitudinal panel survey. J Off Stat 15:269–282

    Google Scholar 

  • Layard R (2005) Happiness. Lessons from a New Science. Allen Lane, London

    Google Scholar 

  • Lipps O (2009) Attrition of Households and Individuals in Panel Surveys, DIW SOEPpapers on Multidisciplinary Panel Data Research 164

  • Luechinger S, Meier S, Stutzer A (2010) Why does unemployment hurt the employed? Evidence from the life satisfaction gap between the public and the private sector. J Hum Res 45:998–1045

    Article  Google Scholar 

  • Lynn P, Kaminska O, Goldstein H (2014) Panel attrition: how important is interviewer continuity? J Off Stat 30:443–457

    Article  Google Scholar 

  • Metcalfe R, Powdthavee N, Dolan P (2011) Destruction and distress: using a quasi-experiment to show the effects of the september 11 attacks on mental well-being in the United Kingdom. Econ J 121:F81–F103

    Article  Google Scholar 

  • Meyer BD, Sullivan JX (2003) Measuring the wellbeing of the poor using income and consumption. J Hum Res 38:1180–1220

    Article  Google Scholar 

  • Meyer BD, Mok WKC, Sullivan JX (2015) Household surveys in crisis. J Econ Perspect 29:199–226

    Article  Google Scholar 

  • Nederhof A (1987) When neutrality is negative: pleasantness of most recent survey experience and nonresponse. Qual Quant 21:425–432

    Article  Google Scholar 

  • Nicoletti C, Peracchi F (2005) Survey response and survey characteristics: microlevel evidence from the european community household panel. J R Stat Soc Ser A 168:763–781

    Article  Google Scholar 

  • Olsen RJ (2005) The problem of respondent attrition: survey methodology is key. Mon Labor Rev 128:63–70

    Google Scholar 

  • Olson K (2013) Do non-response follow-ups improve or reduce data quality?: a review of the existing literature. J R Stat Soc Ser A 176:129–145

    Article  Google Scholar 

  • Pannenberg M, Rendtel U (1996) Documentation of Sample Sizes and Panel Attrition in the German Socio Economic Panel (SOEP), Discussion Paper No. 137, DIW Berlin

  • Pickery J, Loosveldt G, Carton A (2001) The effects of interviewer and respondent characteristics on response behavior in panel surveys: a multilevel approach. Sociol Methods Res 29:509–523

    Article  Google Scholar 

  • Schueller S (2016) The effects of 9/11 on attitudes toward immigration and the moderating role of education. Kyklos 69:604–632

    Article  Google Scholar 

  • Schraepler JP, Schupp J, Wagner GG (2010) Individual and Neighborhood Determinants of Survey Nonresponse, DIW SOEPpapers on Multidisciplinary Panel Data Research 288

  • Schwandt H (2016) Unmet aspirations as an explanation for the age U-shape in wellbeing. J Econ Behav Organ 122:75–87

    Article  Google Scholar 

  • Stevenson B, Wolfers J (2009) The paradox of declining female happiness. Am Econ J Econ Policy 1:190–225

    Article  Google Scholar 

  • Stiglitz JE, Sen A, Fitoussi JP (2009) Report of the Commission on the Measurement of Economic Performance and Social Progress, CMEPSP (www.stiglitz-sen-toussi.fr)

  • Stutzer A (2004) The role of income aspirations in individual happiness. J Econ Behav Organ 54:89–109

    Article  Google Scholar 

  • Van Landeghem B (2012) A test for the convexity of human well-being over the life cycle: longitudinal evidence from a 20-year panel. J Econ Behav Organ 81:571–582

    Article  Google Scholar 

  • Van Landeghem B (2014) A test based on panel refreshment for panel conditioning in stated utility measures. Econ Lett 124:236–238

    Article  Google Scholar 

  • Van Praag B, Ferrer-i-Carbonell A (2008). Happiness quantified: a satisfaction calculus approach, (Revised edition), Oxford, New York

  • Wagner GG, Frick JR, Schupp J (2007) The German socio-economic panel study (soep): scope, evolution and enhancements. J Appl Soc Sci Stud 127:139–169

    Google Scholar 

  • Weimann J, Knabe A, Schöb R (2015) Measuring happiness: the economics of well-being. MIT Press, Cambridge

    Book  Google Scholar 

  • Wunder C, Heineck G (2013) Working time preferences, hours mismatch and well-being of couples: are there spillovers? Labour Econ 24:244–252

    Article  Google Scholar 

  • Wunder C, Wiencierz A, Schwarze J, Kuechenhoff H (2013) Well-being over the life span: semiparametric evidence from British and German longitudinal data. Rev Econ Stat 95:154–167

    Article  Google Scholar 

  • Zabel JE (1998) An analysis of attrition in the panel study of income dynamics and the survey of income and program participation with an application to a model of labor market behavior. J Hum Res 33:479–506

    Article  Google Scholar 

Download references

Acknowledgements

I am grateful to Peter Krause and Martin Kroh for very helpful advice. For comments and discussions, I thank Daniel Arnold, Olivier Bargain, Ori Heffetz, Martin Huber, Joachim Winter, Jinan Zeidan, two anonymous referees as well as participants of the International SOEP User Conference, Berlin 2012; the CISEPS International Conference on Public Happiness, Rome 2013; the TIBER Conference, Tilburg 2013; the IAAEU seminar, Trier 2013; the Statistical Week of the German Statistical Society, Berlin 2013; the LAGV Public Economics Conference, Aix-en-Provence 2014; the Quality of Life Conference, Berlin 2014; and the Annual Meetings of the International Association for Applied Econometrics, London 2014; the Austrian Economic Association, Vienna 2014; and the German Economic Association, Hamburg 2014.

Author information

Authors and Affiliations

  1. Department of Economics, University of Konstanz, Universitätsstraße 10, 78457, Constance, Germany

    Adrian Chadi

Authors
  1. Adrian Chadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian Chadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 9, 10, 11, 12.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chadi, A. Dissatisfied with life or with being interviewed? Happiness and the motivation to participate in a survey. Soc Choice Welf 53, 519–553 (2019). https://doi.org/10.1007/s00355-019-01195-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00355-019-01195-5

Search

Navigation