The Measurement of Subjective Wellbeing: Item-Order Effects in the Personal Wellbeing Index—Adult

Abstract

When multi-item questionnaires are included in psychological research, many factors can influence the response given. One such factor that has traditionally been overlooked is the potential impact of item-order effects. This paper extends upon the work of Kaplan et al. (J Happiness Stud 14:1443–1458, 2013. doi: 10.1007/s10902-012-9388-5) who explored item-order effects in measures of job satisfaction, and applies similar principles to the measurement of life satisfaction, or subjective wellbeing, by exploring item-order effects within the Personal Wellbeing Index (PWI; IWBG in Personal Wellbeing Index, 5th edn. Australian Centre on Quality of Life, Deakin University, Melbourne, 2013. http://www.deakin.edu.au/research/acqol/intruments/wellbeing-index/index.php). In a preliminary study, participants completed the PWI in its standard format (general-specific) and psychometric properties were compared to those who completed the PWI in an alternate format (specific-general). Analyses revealed that the PWI performed adequately for both groups, though there were subtle indications of item-order effects. In a second study, the order of the PWI domains was randomised (random-order) and compared to the standard format (fixed-order). Results revealed lower mean scores and more variation in scores when items were randomised. Overall, the PWI performed as expected for most interrogative analyses. It achieved a single factor solution, no matter the order of items, and the same domains emerged as significant unique predictors of general life satisfaction. The study highlights the importance of exploring item-order effects as part of the psychometric validation procedure, and it is recommended that all new scales be subject to this investigation to reduce measurement error and improve accuracy in psychological assessment.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Australian Bureau of Statistics. (2016). Household use of information technology, Australia, 20142015, cat no. 8146.0. Canberra, Australia: ABS.

  2. Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1982). Political information processing: Question order and context effects. Political Behavior, 4, 177–200.

    Article  Google Scholar 

  3. Carlson, J. E., Mason, R., Saltiel, J., & Sangster, R. (1995). Assimilation and contrast effects in general/specific questions. Rural Sociology, 60, 666–673.

    Article  Google Scholar 

  4. Cummins, R. (2010). Subjective wellbeing, homeostatically protected mood and depression: A synthesis. Journal of Happiness Studies, 11, 1–17. doi:10.1007/s10902-009-9167-0.

    Article  Google Scholar 

  5. Cummins, R. A., Eckersley, R., Pallant, J., Van Vugt, J., & Misajon, R. (2003). Developing a national index of subjective wellbeing: The Australian Unity Wellbeing Index. Social Indicators Research, 64, 159–190.

    Article  Google Scholar 

  6. Cummins, R. A., & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case for subjective quality of life measurement. In Proceedings of the second international conference on quality of life in cities (pp. 74–93). National University of Singapore, Singapore.

  7. Cummins, R. A., Woerner, J., Weinberg, M., Collard, J., Hartley-Clark, J., Horfiniak, K., et al. (2013). Australian Unity Wellbeing Index Survey 30.0: The wellbeing of Australianssocial media, personal achievement and work. Melbourne, Australia: Centre on Quality of Life, School of Psychology, Deakin University. http://www.acqol.com.au/reports/auwbi.php.

  8. Cummins, R. A., & Wooden, M. (2014). Personal resilience in times of crisis: The implications of SWB homeostasis and set-points. Journal of Happiness Studies, 15, 223–235.

    Article  Google Scholar 

  9. Diener, E. (1984). Subjective well-being. Psychological Bulletin, 95, 542–575. doi:10.1037/0033-2909.95.3.542.

    Article  Google Scholar 

  10. Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–76.

    Article  Google Scholar 

  11. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics, Vol. 3: Speech acts (Vol. 3, pp. 225–242). New York: Seminar Press.

    Google Scholar 

  12. Haviland, S. E., & Clark, H. H. (1974). What’s new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512–521.

    Article  Google Scholar 

  13. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  14. IBM. (2013). IBM SPSS statistics for windows, Version 22.0. Armonk, NY: IBM Corp.

  15. International Wellbeing Group. (2013). Personal Wellbeing Index (5th ed.). Melbourne: Australian Centre on Quality of Life, Deakin University. Retrieved from: http://www.deakin.edu.au/research/acqol/intruments/wellbeing-index/index.php.

  16. Kaplan, S., Luchman, J., & Mock, L. (2013). General and specific question sequence effects in satisfaction surveys: Integrating directional and correlational effects. Journal of Happiness Studies, 14, 1443–1458. doi:10.1007/s10902-012-9388-5.

    Article  Google Scholar 

  17. Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51, 201–219.

    Article  Google Scholar 

  18. Lau, A. L. D., & Cummins, R. A. (2005). Test–retest reliability of the Personal Wellbeing Index. Unpublished research report, The Hong Kong Polytechnic University.

  19. Nimon, K. F., & Oswald, F. L. (2013). Understanding the results of multiple linear regression: Beyond standardized regression coefficients. Organizational Research Methods, 16, 650–674.

    Article  Google Scholar 

  20. OECD. (2013). Guidelines on measuring subjective wellbeing. Retrieved from: http://www.oecd.org/statistics/Guidelines%20on%20Measuring%20Subjective%20Well-being.pdf.

  21. Schimmack, U. (2008). The structure of subjective well-being. In M. Eid & R. J. Larsen (Eds.), The science of subjective well-being (pp. 97–123). New York, NY: Guilford Press.

    Google Scholar 

  22. Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording and context. Amherst, MA: Academic Press Inc.

    Google Scholar 

  23. Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.

    Article  Google Scholar 

  24. Schwarz, N., & Bless, H. (1992). Constructing reality and its alternatives: An inclusion/exclusion model of assimilation and contrast effects in social judgment. In L. L. Martin & A. Tesser (Eds.), The construction of social judgments (pp. 217–245). Hillsdale, NJ: Lawrence Erlbaum Associates Inc.

    Google Scholar 

  25. Schwarz, N., Strack, F., & Mai, H. P. (1991). Assimilation and contrast effects in part-whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55, 3–23.

    Article  Google Scholar 

  26. Srull, T. K., & Wyer, R. S. (1980). Category accessibility and social perception: Some implications for the study of person memory and interpersonal judgments. Journal of Personality and Social Psychology, 38, 841–856. doi:10.1037/0022-3514.38.6.841.

    Article  Google Scholar 

  27. Strack, F., & Martin, L. L. (1987). Thinking, judging, and communicating: A process account of context effects in attitude surveys. In F. Strack & L. L. Martin (Eds.), Social information processing and survey methodology (pp. 123–148). New York: Springer.

    Google Scholar 

  28. Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103, 299–314. doi:10.1037/0033-2909.103.3.299.

    Article  Google Scholar 

  29. Tourangeau, R., Rasinski, K. A., & Bradburn, N. (1991). Measuring happiness in surveys: A test of the subtraction hypothesis. Public Opinion Quarterly, 55, 255–266.

    Article  Google Scholar 

  30. World Health Organization. (2013). Joint meeting of experts on targets and indicators for health and well-being in Health 2020. Copenhagen: WHO Regional Office for Europe.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Melissa K. Weinberg.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Weinberg, M.K., Seton, C. & Cameron, N. The Measurement of Subjective Wellbeing: Item-Order Effects in the Personal Wellbeing Index—Adult. J Happiness Stud 19, 315–332 (2018). https://doi.org/10.1007/s10902-016-9822-1

Download citation

Keywords

  • Subjective wellbeing
  • Measurement
  • Psychometric testing
  • Order effects