Advertisement

Social Indicators Research

, Volume 87, Issue 1, pp 27–45 | Cite as

Inducing Score Reliability from Previous Reports: An Examination of Life Satisfaction Studies

  • Matt VassarEmail author
  • Jason W. Ridge
  • Aaron D. Hill
Article

Abstract

The sample specific nature of reliability is often overlooked in substantive research, with investigators frequently referencing reliability estimates from previous studies as the basis for presuming the score integrity for their data. The purpose of the current study was to examine this practice, known as reliability induction, across studies using a popular life satisfaction (LS) measure, the Satisfaction with Life Scale. A comprehensive literature search was conducted, yielding a final sample of 196 studies using the instrument. Fifty-two studies involving 76 samples induced reliability coefficients from other sources. These studies were coded in terms of sample characteristics and variability estimates to derive comparisons in these two areas between the induced reports and the original studies. Results suggest that none of the studies inducing reliability had sufficient justification for induction. It was also found that estimating reliability coefficients for the data in hand has increased relative to induced reports and studies omitting reliability information. Practical implications for substantive researchers are discussed and suggestions for additional research are provided.

Keywords

Reliability induction Life satisfaction Satisfaction with life scale SWLS 

References

*Indicates citations of papers referenced in tables

  1. Arrindell, W. A., Meeuwesen, L, & Huyse, F. J. (1991). The Satisfaction With Life Scale (SWLS): Psychometric properties in a non-psychiatric medical outpatients sample. Personality and Individual Differences, 12, 117–123.CrossRefGoogle Scholar
  2. *Atkinson, M. J., & Zibin, S. (1996). Quality of life measurement among persons with chronic mental illness: A critique of measures and methods. Ottawa: Health promotion and programs branch. Health Canada.Google Scholar
  3. *Baker, L. M., Wilson, F. L., & Winebarger, A. (2004). An exploratory study of the health problems, stigmatization, life satisfaction, and literacy skills of urban, street-level sex workers. Women Health, 39, 83–96.Google Scholar
  4. *Bettencourt, B. A., & Dorr, N. (1997). Collective self-esteem as a mediator of the relationship between allocentrism and subjective well-being. Personality and Social Psychology Bulletin, 23, 955–964.Google Scholar
  5. Biswas-Diener, R., Vitterso, J., & Diener, E. (2005). Most people are pretty happy, but there is cultural variation: The Inughuit, The Amish, and The Maasai. Journal of Happiness Studies, 6, 205–226.CrossRefGoogle Scholar
  6. Blais, M. R., Vallerand, R. J., Pelletier, L. G., & Briere, N. M. (1989). L’Echelle de satisfaction de vie: Validation Canadienne-Francaise du “Satisfaction With Life Scale”. Canadian Journal of Behavioral Science, 21, 210–223.Google Scholar
  7. *Bogner, J. A., Corrigan, J. D., Mysiw, W. J., Clinchot, D., & Fugate, L. (2001). A comparison of substance abuse and violence in the prediction of long-term rehabilitation outcomes after traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 82, 571–577.Google Scholar
  8. *Brillhart, B. (2004). Studying the quality of life and life satisfaction among persons with spinal cord injury undergoing urinary management. Rehabilitation Nursing, 29, 122–126.Google Scholar
  9. *Bryant, R. A., Marosszeky, J. E., Crooks, J., Baguley, I. J., & Gurka, J. A. (2001). Posttraumatic stress disorder and psychosocial functioning after severe traumatic brain injury. Journal of Nervous and Mental Disease, 189, 109–113.Google Scholar
  10. Buhi, E. (2005). Reliability reporting practices in rape myth research. Journal of School Health, 75, 63–66.CrossRefGoogle Scholar
  11. *Carless, D., Douglas, K., Fox, K., & McKenna, J. (2006). An alternative view of psychological well-being in cardiac rehabilitation: Considering temperament and character. European Journal of Cardiovascular Nursing, 5, 237–243.Google Scholar
  12. *Compton, W. (1998). Measures of mental health and a five factor theory of personality. Psychological Reports, 83, 371–381.Google Scholar
  13. *Corrigan, J. D., & Bogner, J. (2004). Latent factors in measures of rehabilitation outcomes after traumatic brain injury. Journal of Head Trauma Rehabilitation, 19, 445–458.Google Scholar
  14. *Corrigan, J. D., Bogner, J. A., Mysiw, W. J., Clinchot, D., & Fugate, L. (2001). Life satisfaction after traumatic brain injury. Journal of Head Trauma Rehabilitation, 16, 543–555.Google Scholar
  15. *Corrigan, J. D., Smith-Knapp, K., & Granger, C. V. (1998). Outcomes in the first 5 years after traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 79, 298–305.Google Scholar
  16. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth, TX: Holt, Rinehart & Winston.Google Scholar
  17. Diener, E. (1984). Subjective well-being. Psychological Bulletin, 95, 542–575.CrossRefGoogle Scholar
  18. Diener, E., & Diener, C. (1996). Most people are happy. Psychological Science, 7, 181–185.CrossRefGoogle Scholar
  19. Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–75.CrossRefGoogle Scholar
  20. *Dijkers, M. (1999). Correlates of life satisfaction among persons with spinal cord injury. Archives of Physical Medicine and Rehabilitation, 80, 867–876.Google Scholar
  21. *Dominy, N. L., Johnson, W. B., & Koch, C. (2000). Perception of parental acceptance in women with binge eating disorder. Journal of Psychology, 134, 23–36.Google Scholar
  22. *Dorahy, M. J., Lewis, C. A., Schumaker, J. F., Akuamoah-Boateng, R., Duze, M. C., & Sibiya, T. E. (1998). A cross-cultural analysis of religion and life satisfaction. Mental Health, Religion and Culture, 1, 37–43.Google Scholar
  23. *Dorahy, M. J., Schumaker, J. F., Simpson, P. L., & Deshpande, C. G. (1996). Depression and life satisfaction in India and Australia. Journal of Personality and Clinical Studies, 12, 1–7.Google Scholar
  24. *Dowler, R., Richards, J. S., Putzke, J. D., Gordon, W., & Tate, D. (2001). Impact of demographic and medical factors on satisfaction with life after spinal cord injury: A normative study. Journal of Spinal Cord Medicine, 24, 87–91.Google Scholar
  25. *Emmons, R. A., & Diener, E. (1985). Personality correlates of subjective well-being. Personality and Social Psychology Bulletin, 11, 89–97.Google Scholar
  26. *Engel, J. M., Jensen, M. P., Hoffman, A. J., & Kartin, D. (2003). Pain in persons with cerebral palsy: Extension and cross validation. Archives of Physical Medicine and Rehabilitation, 84, 1125–1128.Google Scholar
  27. *Ferrari, J. R., Billows, W., Jason, L. A., & Grill, G. J. (1997). Matching the needs of the homeless with those of the disabled: Empowerment through caregiving. Journal of Prevention & Intervention in the Community, 15, 83–92.Google Scholar
  28. *Ferrario, S. R., Zotti, A. M., Massara, G., & Nuvolone, G. (2003). A comparative assessment of psychological and psychosocial characteristics of cancer patients and their caregivers. Psycho-Oncology, 12, 1–7.Google Scholar
  29. Gronlund, N. E., & Linn, R. (1990). Measurement and evaluation in teaching (6th ed.). New York: Macmillan.Google Scholar
  30. Gullicksen, H. (1950). Theory of mental tests. New York: Wiley.Google Scholar
  31. *Halpin, S. A., & Allen, M. W. (2004). Changes in psychosocial well-being during stages of gay identity development. Journal of Homosexuality, 47, 109–126.Google Scholar
  32. *Hamarat, E., Thompson, D., Zabrucky, K., Steele, D., & Matheny, K. (2001). Perceived stress and coping resource availability as predictors of life satisfaction in young, middle-aged, and older adults. Experimental Aging Research, 27, 181–196.Google Scholar
  33. *Hamilton, S. B., Lynch, R. S., Naginey, J. L., Peters, K. A., & Piske, K. R. (1989). Relationships between the life values of United-States college students and their cognitive/affective responses to the threat of nuclear-War. Journal of Adolescence, 12, 55–68.Google Scholar
  34. *Hammarberg, K., Astbury, J., & Baker, H. W. G. (2001). Women’s experience of IVF: A follow-up study. Human Reproduction, 16, 374–383.Google Scholar
  35. *Harrington, R., & Loffredo, D. A. (2001). The relationship between life satisfaction, self-consciousness, and the Myers-Briggs type inventory dimensions. Journal of Psychology, 135, 439–450.Google Scholar
  36. *Hong, S. M., & Giannakopoulos, E. (1994). Effects of age, sex, and university status on life satisfaction. Psychological Reports, 74, 99–103.Google Scholar
  37. *Kafka, G. J., & Kozma, A. (2002). The construct validity of Ryff’s Scales of Psychological Well-Being (SPWB) and their relationship to measures of subjective well-being. Social Indicators Research, 57, 171–190.Google Scholar
  38. Larsen, R. J., Diener, E., & Emmons, R. A. (1986). Affect intensity and reactions to daily life events. Journal of Personality and Social Psychology, 51, 803–814.CrossRefGoogle Scholar
  39. *Meade, M. A., Lewis, A., Jackson, M. N., & Hess, D. W. (2004). Race, employment, and spinal cord injury. Archives of Physical Medicine and Rehabilitation, 85, 1782–1792.Google Scholar
  40. *Minardi, H. A., & Blanchard, M. (2004). Older people with depression: pilot study. Journal of Advanced Nursing, 46, 23–32.Google Scholar
  41. Neto, F. (1993). The satisfaction with life scale: Psychometric properties in an adolescent sample. Journal of Youth and Adolescence, 22, 125–134.CrossRefGoogle Scholar
  42. Oishi, S. (2006). The concept of life satisfaction across cultures: An IRT analysis. Journal of Research in Personality, 40, 411–423.CrossRefGoogle Scholar
  43. *Patterson, D. R., Ptacek, J. T., Cromes, F., Fauerbach, J. A., & Engrav, L. (2000). The 2000 Clinical Research Award: Describing and predicting distress and satisfaction with life for burn survivors. Journal of Burn Care & Rehabilitation, 21, 490–498.Google Scholar
  44. Pavot, W., & Diener, E. (1993). Review of the satisfaction with life scale. Psychological Assessment, 5, 164–172.CrossRefGoogle Scholar
  45. Pavot, W., Diener, E., Colvin, C. R., & Sandvik, E. (1991). Futher validation of the satisfaction with life scale: Evidence for the cross-method convergence of well-being measures. Journal of Personality Assessment, 57, 149–161.CrossRefGoogle Scholar
  46. Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  47. *Perna, F. M., Ahlgren, R. L., & Zaichkowsky, L. (1999). The influence of career planning, race, and athletic injury on life satisfaction among recently retired collegiate male athletes. Sport Psychologist, 13, 144–156.Google Scholar
  48. *Petersen, R. W., Ung, K., Holland, C., & Quinlivan, J. A. (2005). The impact of molar pregnancy on psychological symptomatology, sexual function, and quality of life. Gynecological Oncology, 97, 535–542.Google Scholar
  49. *Pilcher, J. J. (1998). Affective and daily event predictors of life satisfaction in college students. Social Indicators Research, 43, 291–306.Google Scholar
  50. *Prince, P. N., & Gerber, G. J. (2001). Measuring subjective quality of life in people with serious mental illness using the SEIQoL-DW. Quality of Life Research, 10, 117–122.Google Scholar
  51. *Putzke, J. D., Barrett, J. J., Richards, J. S., & DeVivo, M. J. (2003). Age and spinal cord injury: An emphasis on outcomes among the elderly. Journal of Spinal Cord Medicine, 2, 37–44.Google Scholar
  52. *Putzke, J. D., Elliott, T. R., & Richards, J. S. (2001). Marital status and adjustment 1 year post-spinal-cord-injury. Journal of Clinical Psychology in Medical Settings, 8, 101–107.Google Scholar
  53. *Putzke, J. D., & Richards, J. S. (2001). Nursing home residence: Quality of life among individuals with spinal cord injury. American Journal of Physical Medicine & Rehabilitation, 8, 404–409.Google Scholar
  54. *Putzke, J. D., Richards, J. S., & Dowler, R. N. (2000). The impact of pain in spinal cord injury: a case control study. Rehabilitation Psychology, 45, 386–401.Google Scholar
  55. *Putzke, J. D., Richards, J. S., Hicken, B. L., & DeVivo, M. J. (2002). Predictors of life satisfaction: A spinal cord injury cohort study. Archives of Physical Medicine and Rehabilitation, 8, 555–561.Google Scholar
  56. *Reistetter, T. A., Spencer, J. C., Trujillo, L., & Abreu, B. C. (2005). Examining the Community Integration Measure (CIM): A replication study with life satisfaction. Neurorehabilitation, 20, 139–148.Google Scholar
  57. *Richards, J. S., Bombardier, C. H., Tate, D., Dijkers, M., Gordon, W., Shewchuk, R., & DeVivo, M. J. (1999). Access to the environment and life satisfaction after spinal cord injury. Archives of Physical Medicine and Rehabilitation, 80, 1501–1506.Google Scholar
  58. *Riley, B. B., Perna, R., Tate, D. G., Forchheimer, M., Anderson, C., & Luera, G. (1998). Types of spiritual well-being among persons with chronic illness: Their relation to various forms of quality of life. Archives of Physical Medicine and Rehabilitation, 79, 258–264.Google Scholar
  59. Reinhardt, B. (1996). Factors affecting coefficient alpha: A mini Monte Carlo study. In B. Thompson (Ed.), Advances in social science methodology. (pp. 3–20). Greenwich: JAI Press.Google Scholar
  60. Sawilowski, S. S. (2000). Psychometrics versus datametrics: Comment on Vacha-Haase’s “Reliability Generalization” method and some EPM editorial policies. Educational and Psychological Measurement, 60, 157–173.CrossRefGoogle Scholar
  61. *Schumaker, J. F., Shea, J. D., Monfries, M. M., & Grothmarnat, G. (1993). Loneliness and life satisfaction in Japan and Australia. Journal of Psychology, 127, 65–71.Google Scholar
  62. Schwartz, N., & Strack, F. (1999). Reports of subjective well-being: Judgmental processes and their methodological implications. In D. Kahneman, E. Diener, & N. Schwartz (Eds.). Well-being: The foundations of hedonic psychology (pp. 61–84). New York: Russell Sage.Google Scholar
  63. *Sherman, J. E., DeVinney, D. J., & Sperling, K. B. (2004). Social support and adjustment after spinal cord injury: Influence of past peer-mentoring experiences and current live-in partner. Rehabilitation Psychology, 49, 140–149.Google Scholar
  64. *Simpson, P., Schumaker, J. F., Dominy, N. L., & Shrestha, S. (1996). Depression and life satisfaction in Nepal and Austria. Journal of Social Psychology, 136, 783–790.Google Scholar
  65. *Skok, A., Harvey, D., & Reddihough, D. (2006). Perceived stress, perceived social support, and well-being among mothers of school aged children with cerebral palsy. Journal of Intellectual and Developmental Disability, 31, 53–57.Google Scholar
  66. Smead, V. S. (1991). Measuring well-being is not easy. Paper presented at the Annual Convention of the American Association of Applied and Preventive Psychology.Google Scholar
  67. *Smith, C., & Hale, B. (2005). Exercise-dependence in bodybuilders: Antecedents and reliability of measurement. Journal of Sports Medicine and Physical Fitness, 45, 401–408.Google Scholar
  68. *Stolberg, V. B. (2003). Compulsive eating and substance abuse factors among African-American community college students. Journal of Ethnicity in Substance Abuse, 2, 77–95.Google Scholar
  69. *Tate, D., Forchheimer, M., Krause, J. S., & Meade, M. A. (2004). Patterns of alcohol and substance use and abuse in persons with spinal chord injury: Risk factors and correlates. Archives of Physical Medicine and Rehabilitation, 85, 1837–1847.Google Scholar
  70. *Tate, D. G., Riley, B. B., Perna, R., & Roller, S. (1997). Quality of life issues among women with physical disabilities or breast cancer. Archives of Physical Medicine and Rehabilitation, 78, S18–S25.Google Scholar
  71. *Tucker, K. L., Ozer, D. J., Lyubomirsky, S., & Boehm, J. K. (2006). Testing for measurement invariance in the satisfaction with life scale: A comparison of Russians and North Americans. Social Indicators Research, 78, 341–360.Google Scholar
  72. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6–20.CrossRefGoogle Scholar
  73. Vacha-Haase, T., Henson, R. K., & Caruso, J. C. (2002). Reliability generalization: Moving toward improved understanding and use of score reliability. Educational and Psychological Measurement, 62, 562–569.CrossRefGoogle Scholar
  74. Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509–522.CrossRefGoogle Scholar
  75. Vassar, M. (in press). A note on the score reliability for the Satisfaction With Life Scale: An RG study. Social Indicators Research.Google Scholar
  76. Vassar, M. & Hale, D. (2007). Reliability reporting practices in youth life satisfaction research. Social Indicators Research, 83(3), 487–496.CrossRefGoogle Scholar
  77. *Wardell, D., Rintala, D., Duan, Z., & Tan, G. (2006). A pilot study of healing touch and progressive relaxation for chronic neuropathic pain in persons with spinal chord injury. Journal of Holistic Nursing, 24, 231–240.Google Scholar
  78. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070.CrossRefGoogle Scholar
  79. *Weinman, J., Wright, S., & Johnston, M. (1995). Health status and health related quality of life, measures in health psychology. A user’s portfolio. NFER-Nelson, Windsor, UK.Google Scholar
  80. Wilkinson, L. (2002) American psychological association task force on statistical inference. Statistical methods in psychology journals: Guidelines and explanations. American Psychological Journal 54, 594–604.CrossRefGoogle Scholar
  81. *Wu, C. H., & Yao, G. (2006). Analysis of factorial invariance across gender in the Taiwan version of the satisfaction with life scale. Personality and Individual Differences, 40, 1259–1268.Google Scholar
  82. *Yetim, U. (1993). Life satisfaction: A study based on the organization of personal projects. Social Indicators Research, 19, 277–289.CrossRefGoogle Scholar
  83. *Yetim, U. (2003). The impacts of individualism/collectivism, self-esteem, and feeling of mastery on life satisfaction among the Turkish University students and academicians. Social Indicators Research, 61, 297–317.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.College of Osteopathic MedicineOklahoma State UniversityTulsaUSA
  2. 2.Department of ManagementWilliam S. Spears School of Business, Oklahoma State UniversityStillwaterUSA

Personalised recommendations