Quality of Life Research

, Volume 25, Issue 10, pp 2497–2510 | Cite as

Question order sensitivity of subjective well-being measures: focus on life satisfaction, self-rated health, and subjective life expectancy in survey instruments

  • Sunghee Lee
  • Colleen McClain
  • Noah Webster
  • Saram Han
Article

Abstract

Purpose

This study examines the effect of question context created by order in questionnaires on three subjective well-being measures: life satisfaction, self-rated health, and subjective life expectancy.

Methods

We conducted two Web survey experiments. The first experiment (n = 648) altered the order of life satisfaction and self-rated health: (1) life satisfaction asked immediately after self-rated health; (2) self-rated health immediately after life satisfaction; and (3) two items placed apart. We examined their correlation coefficient by experimental condition and further examined its interaction with objective health. The second experiment (n = 479) asked life expectancy before and after parental mortality questions. Responses to life expectancy were compared by order using ANOVA, and we examined interaction with parental mortality status using ANCOVA. Additionally, response time and probes were examined.

Results

Correlation coefficients between self-rated health and life satisfaction differed significantly by order: 0.313 (life satisfaction first), 0.508 (apart), and 0.643 (self-rated health first). Differences were larger among respondents with chronic conditions. Response times were the shortest when self-rated health was asked first. When life expectancy asked after parental mortality questions, respondents reported considering parents more for answering life expectancy; and respondents with deceased parents reported significantly lower expectancy, but not those whose parents were alive.

Conclusion

Question context effects exist. Findings suggest placing life satisfaction and self-rated health apart to avoid artificial attenuation or inflation in their association. Asking about parental mortality prior to life expectancy appears advantageous as this leads respondents to consider parental longevity more, an important factor for true longevity.

Keywords

Instrument design Question order Self-rated health Global life satisfaction Subjective life expectancy Survey research 

References

  1. 1.
    Diener, E., Oishi, S., & Lucas, R. E. (2003). Personality, culture, and subjective well-being: Emotional and cognitive evaluations of life. Annual Review of Psychology, 54(1), 403–425.PubMedCrossRefGoogle Scholar
  2. 2.
    Campbell, A., Converse, P. E., & Rodgers, W. L. (1976). The query of American-life. New York, NY: Russell Sage.Google Scholar
  3. 3.
    Ryan, R. M., & Deci, E. L. (2001). On happiness and human potentials: A review of research on hedonic and eudaimonic well-being. Annual Review of Psychology, 52(1), 141–166.PubMedCrossRefGoogle Scholar
  4. 4.
    Kahneman, D., Diener, E., & Schwarz, N. (1999). Well-being: The foundations of hedonic psychology. New York, NY: Russell Sage Foundation.Google Scholar
  5. 5.
    Schwarz, N., & Strack, F. (1999). Report of subjective well-being: Judgmental processes and their methodological implications. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being: The foundations of hedonic psychology (pp. 61–84). New York, NY: Russell Sage Foundation.Google Scholar
  6. 6.
    Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. London: Academic Press.Google Scholar
  7. 7.
    Schwarz, N. (1996). Cognition and communication: judgmental biases, research methods, and the logic of conversation. Hillsdale, NJ: Erlbaum.Google Scholar
  8. 8.
    Schuman, H. (1992). Context effects: State of the past/state of the art. In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 5–20). New York, NY: Springer.CrossRefGoogle Scholar
  9. 9.
    Schwarz, N. (1999). Self-reports: how the questions shape the answers. American Psychologist, 54(2), 93.CrossRefGoogle Scholar
  10. 10.
    Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  11. 11.
    Strack, F., Schwarz, N., & Wänke, M. (1991). Semantic and pragmatic aspects of context effects in social and psychological research. Social Cognition, 9(1), 111–125.CrossRefGoogle Scholar
  12. 12.
    Lee, S., & Schwarz, N. (2014). Question context and priming meaning of health: Effect on differences in self-rated health between Hispanics and non-Hispanic whites. American Journal of Public Health, 104(1), 179–185.PubMedPubMedCentralCrossRefGoogle Scholar
  13. 13.
    Garbarski, D., Schaeffer, N. C., & Dykema, J. (2015). The effects of response option order and question order on self-rated health. Quality of Life Research, 24, 1443–1453.PubMedCrossRefGoogle Scholar
  14. 14.
    Sudman, S., Bradburn, N. M., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.Google Scholar
  15. 15.
    Zaller, J. (1992). The nature and origins of mass opinion. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  16. 16.
    Higgins, E. T. (1989). Knowledge accessibility and activation: Subjectivity and suffering from unconscious sources. In J. S. Uleman & J. A. Barghs (Eds.), Unintended thought (pp. 75–123). New York, NY: Guilford Press.Google Scholar
  17. 17.
    Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103(3), 299–314.CrossRefGoogle Scholar
  18. 18.
    Wyer, R. S., & Srull, T. K. (1986). Human cognition in its social context. Psychological Review, 93(3), 322–359.PubMedCrossRefGoogle Scholar
  19. 19.
    Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 103(3), 582–591.PubMedCrossRefGoogle Scholar
  20. 20.
    Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.PubMedCrossRefGoogle Scholar
  21. 21.
    Organisation for Economic Cooperation and Development (OECD). (2013). OECD guidelines on measuring subjective well-being. OECD Publishing. Retrieved from http://dx.doi.org/10.1787/9789264191655-en.
  22. 22.
    Pavot, W., & Diener, E. (1993). Review of the satisfaction with life scale. Psychological Assessment, 5(2), 164–172.CrossRefGoogle Scholar
  23. 23.
    Diener, E. (1994). Assessing subjective well-being: Progress and opportunities. Social Indicators Research, 31, 103–157.CrossRefGoogle Scholar
  24. 24.
    Barbato, A., Monzani, E., & Schiavi, T. (2004). Life satisfaction in a sample of outpatients with severe mental disorders: A survey in northern Italy. Quality of Life Research, 13, 969–973.PubMedCrossRefGoogle Scholar
  25. 25.
    Bjørnskov, C. (2010). How comparable are the Gallup World Poll life satisfaction data? Journal of Happiness Studies, 11(1), 41–60.CrossRefGoogle Scholar
  26. 26.
    Organisation for Economic Cooperation and Development (OECD). (2015). Life satisfaction. Retrieved from http://www.oecdbetterlifeindex.org/topics/life-satisfaction/.
  27. 27.
    Idler, E. L., & Kasl, S. (1991). Health perceptions and survival: Do global evaluations of health status really predict mortality? Journal of Gerontology, 46(2), S55–S65.PubMedCrossRefGoogle Scholar
  28. 28.
    Ferraro, K. F., Farmer, M. M., & Wybraniec, J. A. (1997). Health trajectories: Long-term dynamics among black and white adults. Journal of Health and Social Behavior, 38(1), 38–54.PubMedCrossRefGoogle Scholar
  29. 29.
    Kuusio, H., Heponiemi, T., Aalto, A. M., Sinervo, T., & Elovainio, M. (2012). Differences in well-being between GPs, medical specialists, and private physicians: The role of psychosocial factors. Health Services Research, 47(1), 68–85.PubMedCrossRefGoogle Scholar
  30. 30.
    Riise, H. K. R., Riise, T., Natvig, G. K., & Daltveit, A. K. (2014). Poor self-rated health associated with an increased risk of subsequent development of lung cancer. Quality of Life Research, 23, 145–153.PubMedCrossRefGoogle Scholar
  31. 31.
    Shi, L., Starfield, B., Politzer, R., & Regan, J. (2002). Primary care, self-rated health, and reductions in social disparities in health. Health Services Research, 37(3), 529–550.PubMedPubMedCentralCrossRefGoogle Scholar
  32. 32.
    DeSalvo, K. B., Fan, V. S., McDonell, M. B., & Fihn, S. D. (2005). Predicting mortality and healthcare utilization with a single question. Health Services Research, 40(4), 1234–1246.PubMedPubMedCentralCrossRefGoogle Scholar
  33. 33.
    Veenstra, M., Moum, T., & Garratt, A. M. (2006). Patient experiences with information in a hospital setting: Associations with coping and self-rated health in chronic illness. Quality of Life Research, 15, 967–978.PubMedCrossRefGoogle Scholar
  34. 34.
    Andersen, F. K., Christensen, K., & Frederiksen, H. (2007). Self-rated health and age: A cross-sectional and longitudinal study of 11,000 Danes aged 45–102. Scandinavian Journal of Public Health, 35(2), 164–171.PubMedCrossRefGoogle Scholar
  35. 35.
    Fleishman, J. A., & Cohen, J. W. (2010). Using information on clinical conditions to predict high-cost patients. Health Services Research, 45(2), 532–552.PubMedPubMedCentralCrossRefGoogle Scholar
  36. 36.
    Mossey, J. M., & Shapiro, E. (1982). Self-rated health: A predictor of mortality among the elderly. American Journal of Public Health, 72(8), 800–808.PubMedPubMedCentralCrossRefGoogle Scholar
  37. 37.
    Idler, E. L., & Benyamini, Y. (1997). Self-rated health and mortality: A review of twenty-seven community studies. Journal of Health and Social Behavior, 38(1), 21–37.PubMedCrossRefGoogle Scholar
  38. 38.
    Mohan, R., Beydoun, H. A., Beydoun, M. A., Barnes-Eley, M., Davis, J., Lance, R., & Schellhammer, P. (2011). Self-rated health as a tool for estimating health-adjusted life expectancy among patients newly diagnosed with localized prostate cancer: A preliminary study. Quality of Life Research, 20(5), 713–721.PubMedCrossRefGoogle Scholar
  39. 39.
    Kleinman, A. (1980). Patients and healers in the context of cultures. Berkeley, CA: University of California Press.Google Scholar
  40. 40.
    Bury, M. (1982). Chronic illness as biological disruption. Sociology of Health and Illness, 12, 451–468.Google Scholar
  41. 41.
    Zullig, K. J., & White, R. J. (2011). Physical activity, life satisfaction, and self-rated health of middle school students. Applied Research in Quality of Life, 6(3), 277–289.CrossRefGoogle Scholar
  42. 42.
    Herman, K. M., Hopman, W. M., & Rosenberg, M. W. (2013). Self-rated health and life satisfaction among Canadian adults: associations of perceived weight status versus BMI. Quality of Life Research, 22, 2693–2705.PubMedCrossRefGoogle Scholar
  43. 43.
    Okun, M. A., & George, L. K. (1984). Physician-and self-ratings of health, neuroticism and subjective well-being among men and women. Personality and Individual Differences, 5(5), 533–539.CrossRefGoogle Scholar
  44. 44.
    Lawton, M. P. (1977). Morale: What are we measuring? In C. N. Nydeggar (Ed.), Measuring morale: A guide to effective measurement. Washington, DC: The Gerontological Society.Google Scholar
  45. 45.
    Elder, T. E. (2013). The predictive validity of subjective mortality expectations: evidence from the health and retirement study. Demography, 50(2), 569–589.PubMedCrossRefGoogle Scholar
  46. 46.
    Hurd, M. D., & McGarry, K. (1995). Evaluation of the subjective probabilities of survival in the health and retirement study. Journal of Human Resources, 30, S268–S292.Google Scholar
  47. 47.
    Perozek, M. (2008). Using subjective expectations to forecast longevity: Do survey respondents know something we don’t know? Demography, 45(1), 95–113.PubMedPubMedCentralCrossRefGoogle Scholar
  48. 48.
    Van Doorn, C., & Kasl, S. V. (1998). Can parental longevity and self-rated life expectancy predict mortality among older persons? Results from an Australian cohort. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 53(1), S28–S34.CrossRefGoogle Scholar
  49. 49.
    Lee, S., & Smith, J. (2016). Methodological aspects of subjective life expectancy: Effects of culture-specific reporting heterogeneity among older adults in the U.S. The Journals of Gerontology: Series B Psychological Sciences and Social Sciences, 71(3), 558–568.Google Scholar
  50. 50.
    Smith, T. W. (1982). Conditional order effect. GSS technical report. no. 33. Chicago, IL: NORC.Google Scholar
  51. 51.
    Strack, F., Martin, L. L., & Schwarz, N. (1988). Priming and communication: Social determinants of information use in judgments of life satisfaction. European Journal of Social Psychology, 18, 429–442.CrossRefGoogle Scholar
  52. 52.
    Schwarz, N., Strack, F., & Mai, H. P. (1991). Assimilation and contrast effects in part–whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55(1), 3–23.CrossRefGoogle Scholar
  53. 53.
    Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics 3: Speech acts (pp. 41–58). New York, NY: Academic Press.Google Scholar
  54. 54.
    Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D. (2001). Bad is stronger than good. Review of General Psychology, 5(4), 323–370.CrossRefGoogle Scholar
  55. 55.
    Griffin, B., Loh, V., & Hesketh, B. (2013). A mental model of factors associated with subjective life expectancy. Social Science and Medicine, 82, 79–86.PubMedCrossRefGoogle Scholar
  56. 56.
    Pyszczynski, T., Greenberg, J., & Solomon, S. (1999). A dual-process model of defense against conscious and unconscious death-related thoughts: An extension of terror management theory. Psychological Review, 106(4), 835.PubMedCrossRefGoogle Scholar
  57. 57.
    Jonas, E., Schimel, J., Greenberg, J., & Pyszczynski, T. (2002). The Scrooge effect: Evidence that mortality salience increases prosocial attitudes and behavior. Personality and Social Psychology Bulletin, 28(10), 1342–1353.CrossRefGoogle Scholar
  58. 58.
    Renkema, L. J., Stapel, D. A., & Van Yperen, N. W. (2008). Go with the flow: Conforming to others in the face of existential threat. European Journal of Social Psychology, 38(4), 747–756.CrossRefGoogle Scholar
  59. 59.
    Han, S., Qin, J., & Ma, Y. (2010). Neurocognitive processes of linguistic cues related to death. Neuropsychologia, 48(12), 3436–3442.PubMedCrossRefGoogle Scholar
  60. 60.
    Klackl, J., Jonas, E., & Kronbichler, M. (2013). Existential neuroscience: Neurophysiological correlates of proximal defenses against death-related thoughts. Social Cognitive and Affective Neuroscience, 8(3), 333–340.PubMedCrossRefGoogle Scholar
  61. 61.
    Luo, S., Shi, Z., Yang, X., Wang, X., & Han, S. (2014). Reminders of mortality decrease midcingulate activity in response to others’ suffering. Social Cognitive and Affective Neuroscience, 9(4), 477–486.PubMedCrossRefGoogle Scholar
  62. 62.
    Mathews, P., & Sear, R. (2008). Life after death: An investigation into how mortality perceptions influence fertility preferences using evidence from an internet-based experiment. Journal of Evolutionary Psychology, 6(3), 155–172.CrossRefGoogle Scholar
  63. 63.
    Murphy, J., Link, M. W., Childs, J. H., Tesfaye, C. L., Dean, E., Stern, M., & Schober, M. F. (2014). Social media in public opinion research: Report of the AAPOR task force on emerging technologies in public opinion research. American Association for Public Opinion Research. Retrieved from https://www.aapor.org/AAPORKentico/AAPOR_Main/media/MainSiteFiles/AAPOR_Social_Media_Report_FNL.pdf.
  64. 64.
    Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.PubMedCrossRefGoogle Scholar
  65. 65.
    Antoun, C., Zhang, C., Conrad, F. G., & Schober, M. F. (2015). Comparisons of online recruitment strategies for convenience samples: Craigslist, Google AdWords, Facebook, and Amazon Mechanical Turk. Field Methods,. doi:10.1177/1525822X15603149.Google Scholar
  66. 66.
    Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20(3), 351–368.CrossRefGoogle Scholar
  67. 67.
    Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419.Google Scholar
  68. 68.
    Horton, J. J., & Chilton, L. B. (2010). The labor economics of paid crowdsourcing. In Proceedings of the 11th ACM Conference on Electronic Commerce (pp. 209–218). Association for Computing Machinery. Retrieved from http://arxiv.org/pdf/1001.0627.pdf.
  69. 69.
    Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44(1), 1–23.PubMedCrossRefGoogle Scholar
  70. 70.
    Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1(2), 213–220.CrossRefGoogle Scholar
  71. 71.
    Groves, R. M., Fultz, N. H., & Martin, E. (1992). Direct questioning about comprehension in a survey setting. Questions about questions: Inquiries into the cognitive bases of surveys. New York, NY: Russell Sage, 49–61.Google Scholar
  72. 72.
    Behr, D., Kaczmirek, L., Bandilla, W., & Braun, M. (2012). Asking probing questions in web surveys: Which factors have an impact on the quality of responses? Social Science Computer Review, 30(4), 487–498.CrossRefGoogle Scholar
  73. 73.
    Bassili, J. N., & Scott, B. S. (1996). Response latency as a signal to question problems in survey research. Public Opinion Quarterly, 60(3), 390–399.CrossRefGoogle Scholar
  74. 74.
    Tourangeau, R., Couper, M. P., & Conrad, F. G. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368–393.CrossRefGoogle Scholar
  75. 75.
    Malhotra, N. (2008). Completion time and response order effects in web surveys. Public Opinion Quarterly, 72(5), 914–934.CrossRefGoogle Scholar
  76. 76.
    Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: The effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology, 22(1), 51–68.CrossRefGoogle Scholar
  77. 77.
    Shannon, C. E. (1948). A mathematical theory of communication. The Bell Systems Technical Journal, 27, 379–423 and 623–656.Google Scholar
  78. 78.
    Fisher, R. A. (1921). On the probable error of a coefficient of correlation deduced from a small sample. Metron, 1, 3–32.Google Scholar
  79. 79.
    Silver, N. C., & Dunlap, W. P. (1987). Averaging correlation coefficients: Should Fisher’s z transformation be used? Journal of Applied Psychology, 72(1), 146.CrossRefGoogle Scholar
  80. 80.
    Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Sunghee Lee
    • 1
  • Colleen McClain
    • 1
  • Noah Webster
    • 1
  • Saram Han
    • 2
  1. 1.Institute for Social ResearchUniversity of MichiganAnn ArborUSA
  2. 2.School of Hotel AdministrationCornell UniversityIthacaUSA

Personalised recommendations