Quality of Life Research

, Volume 26, Issue 1, pp 139–148 | Cite as

Do measures of depressive symptoms function differently in people with spinal cord injury versus primary care patients: the CES-D, PHQ-9, and PROMIS®-D

  • Karon F. Cook
  • Michael A. Kallen
  • Charles Bombardier
  • Alyssa M. Bamer
  • Seung W. Choi
  • Jiseon Kim
  • Rana Salem
  • Dagmar Amtmann



To evaluate whether items of three measures of depressive symptoms function differently in persons with spinal cord injury (SCI) than in persons from a primary care sample.


This study was a retrospective analysis of responses to the Patient Health Questionnaire depression scale, the Center for Epidemiological Studies Depression scale, and the National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS®) version 1.0 eight-item depression short form 8b (PROMIS-D). The presence of differential item function (DIF) was evaluated using ordinal logistic regression.


No items of any of the three target measures were flagged for DIF based on standard criteria. In a follow-up sensitivity analyses, the criterion was changed to make the analysis more sensitive to potential DIF. Scores were corrected for DIF flagged under this criterion. Minimal differences were found between the original scores and those corrected for DIF under the sensitivity criterion.


The three depression screening measures evaluated in this study did not perform differently in samples of individuals with SCI compared to general and community samples. Transdiagnostic symptoms did not appear to spuriously inflate depression severity estimates when administered to people with SCI.


Spinal cord injuries Depression Diagnosis Psychometrics Rehabilitation Screening Differential item function Measurement invariance 



Center for Epidemiological Studies Depression scale


Confirmatory factor analysis


Differential item functioning


Diagnostic and Statistical Manual of Mental Disorders fourth edition


Exploratory factor analysis


Glasgow Coma Scale


Graded response model


Item response theory


Major depressive disorder


Patient Health Questionnaire 9


Patient-Reported Outcomes Measurement Information System (PROMIS®)-Depression


Spinal cord injury


Traumatic brain injury



Research reported in this paper was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number R03HS020700. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. The contents of this publication were developed in part under Grants from the Department of Education, National Institute of Disability and Rehabilitation Research, Grant Numbers H133B080024, H133B031129, H133N110009, and H133N060033. However, those contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal Government. Research reported in this paper was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number 5U01AR052171. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Compliance with ethical standards

Conflict of interest

Drs. Cook, Kallen, Bombarier, Choi and Amtmann and Ms. Bamer and Ms. Salem declare that they have no conflict of interest.

Ethical standards

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Craig, A., Tran, Y., & Middleton, J. (2009). Psychological morbidity and spinal cord injury: A systematic review. Spinal Cord, 47(2), 108–114.CrossRefPubMedGoogle Scholar
  2. 2.
    Herrick, S., Elliott, T. R., & Crow, F. (1994). Social support and the prediction of health complications among persons with spinal cord injuries. Rehabilitation Psychology, 39, 231–250.CrossRefGoogle Scholar
  3. 3.
    Elliott, T., & Harkins, S. (1991). Psychosocial concomitants of persistent pain among persons with spinal cord injury. NeuroRehabilitation: An Interdisciplinary Journal, 1(4), 9–18.Google Scholar
  4. 4.
    Fuhrer, M. J., Rintala, D. H., Hart, K. A., Clearman, R., & Young, M. E. (1993). Depressive symptomatology in persons with spinal cord injury who reside in the community. Archives of Physical Medicine and Rehabilitation, 74(3), 255–260.PubMedGoogle Scholar
  5. 5.
    MacDonald, M. R., Nielson, W. R., & Cameron, M. G. (1987). Depression and activity patterns of spinal cord injured persons living in the community. Archives of Physical Medicine and Rehabilitation, 68(6), 339–343.PubMedGoogle Scholar
  6. 6.
    Krause, J. S., Zhai, Y., Saunders, L. L., & Carter, R. E. (2009). Risk of mortality after spinal cord injury: An 8-year prospective study. Archives of Physical Medicine and Rehabilitation, 90(10), 1708–1715.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Fann, J. R., Bombardier, C. H., Richards, J. S., Tate, D. G., Wilson, C. S., Temkin, N., et al. (2011). Depression after spinal cord injury: Comorbidities, mental health service use, and adequacy of treatment. Archives of Physical Medicine and Rehabilitation, 92(3), 352–360.CrossRefPubMedGoogle Scholar
  8. 8.
    Kalpakjian, C. Z., Bombardier, C. H., Schomer, K., Brown, P. A., & Johnson, K. L. (2009). Measuring depression in persons with spinal cord injury: A systematic review. Journal of Spinal Cord Medicine, 32(1), 6–24.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Radloff, L. (1977). The CES-D scale: A new self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401.CrossRefGoogle Scholar
  10. 10.
    Orlando Edelen, M. O., Thissen, D., Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2006). Identification of differential item functioning using item response theory and the likelihood-based model comparison approach. Application to the Mini-Mental State Examination. Medical Care, 44(11 Suppl 3), S134–S142.CrossRefPubMedGoogle Scholar
  11. 11.
    Teresi, J. A. (2006). Different approaches to differential item functioning in health applications. Advantages, disadvantages and some neglected topics. Medical Care, 44(11 Suppl 3), S152–S170.CrossRefPubMedGoogle Scholar
  12. 12.
    Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Cook, K. F., Crane, P. K., Gibbons, L. E., et al. (2007). Evaluating measurement equivalence using the item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): Applications (with illustrations) to measures of physical functioning ability and general distress. Quality of Life Research, 16 Suppl 1, 43–68.CrossRefPubMedGoogle Scholar
  13. 13.
    Cook, K. F., Bamer, A. M., Amtmann, D., Molton, I. R., & Jensen, M. P. (2012). Six patient-reported outcome measurement information system short form measures have negligible age- or diagnosis-related differential item functioning in individuals with disabilities. Archives of Physical Medicine and Rehabilitation, 93(7), 1289–1291.CrossRefPubMedGoogle Scholar
  14. 14.
    Fieo, R., Mukherjee, S., Dmitrieva, N. O., Fyffe, D. C., Gross, A. L., Sanders, E. R., et al. (2015). Differential item functioning due to cognitive status does not impact depressive symptom measures in four heterogeneous samples of older adults. International Journal of Geriatric Psychiatry, 30(9), 911–918.CrossRefPubMedGoogle Scholar
  15. 15.
    Wanders, R. B., Wardenaar, K. J., Kessler, R. C., Penninx, B. W., Meijer, R. R., & de Jonge, P. (2015). Differential reporting of depressive symptoms across distinct clinical subpopulations: What DIFference does it make? Journal of Psychosomatic Research, 78(2), 130–136.CrossRefPubMedGoogle Scholar
  16. 16.
    Chung, H., Kim, J., Askew, R. L., Jones, S. M., Cook, K. F., & Amtmann, D. (2015). Assessing measurement invariance of three depression scales between neurologic samples and community samples. Quality of Life Research, 24(8), 1829–1834.CrossRefPubMedGoogle Scholar
  17. 17.
    Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., et al. (2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS(R)): Depression, anxiety, and anger. Assessment, 18(3), 263–283.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Kroenke, K., Spitzer, R. L., Williams, J. B., & Lowe, B. (2010). The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359.CrossRefPubMedGoogle Scholar
  20. 20.
    Bombardier, C. H., Kalpakjian, C. Z., Graves, D. E., Dyer, J. R., Tate, D. G., & Fann, J. R. (2012). Validity of the Patient Health Questionnaire-9 in assessing major depressive disorder during inpatient spinal cord injury rehabilitation. Archives of Physical Medicine and Rehabilitation, 93(10), 1838–1845.CrossRefPubMedGoogle Scholar
  21. 21.
    Sakakibara, B. M., Miller, W. C., Orenczuk, S. G., Wolfe, D. L., & Team, S. R. (2009). A systematic review of depression and anxiety measures used with individuals with spinal cord injury. Spinal Cord, 47(12), 841–851.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Irwin, D. E., Stucky, B., Langer, M. M., Thissen, D., Dewitt, E. M., Lai, J. S., et al. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Quality of Life Research, 19(4), 595–607.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Narrow, W. E., Clarke, D. E., Kuramoto, S. J., Kraemer, H. C., Kupfer, D. J., Greiner, L., et al. (2013). DSM-5 field trials in the United States and Canada, Part III: Development and reliability testing of a cross-cutting symptom assessment for DSM-5. American Journal of Psychiatry, 170(1), 71–82.CrossRefPubMedGoogle Scholar
  25. 25.
    Weissman, M. M., Sholomskas, D., Pottenger, M., Prusoff, B. A., & Locke, B. Z. (1977). Assessing depressive symptoms in five psychiatric populations: A validation study. American Journal of Epidemiology, 106(3), 203–214.PubMedGoogle Scholar
  26. 26.
    Allison, D. J., & Ditor, D. S. (2015). Targeting inflammation to influence mood following spinal cord injury: A randomized clinical trial. Journal of Neuroinflammation, 12, 204.CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Robinson-Whelen, S., Taylor, H. B., Hughes, R. B., & Nosek, M. A. (2013). Depressive symptoms in women with physical disabilities: Identifying correlates to inform practice. Archives of Physical Medicine and Rehabilitation, 94(12), 2410–2416.CrossRefPubMedGoogle Scholar
  28. 28.
    Spitzer, R. L., Kroenke, K., & Williams, J. B. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA, 282(18), 1737–1744.CrossRefPubMedGoogle Scholar
  29. 29.
    Amtmann, D., Cook, K. F., Johnson, K. L., & Cella, D. (2011). The PROMIS initiative: Involvement of rehabilitation stakeholders in development and examples of applications in rehabilitation research. Archives of Physical Medicine and Rehabilitation, 92(10 Suppl), S12–S19.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Fogelberg, D. J., Vitiello, M. V., Hoffman, J. M., Bamer, A. M., & Amtmann, D. (2015). Comparison of self-report sleep measures for individuals with multiple sclerosis and spinal cord injury. Archives of Physical Medicine and Rehabilitation, 96(3), 478–483.CrossRefPubMedGoogle Scholar
  31. 31.
    Krause, J. S., Saunders, L. L., Bombardier, C., & Kalpakjian, C. (2011). Confirmatory factor analysis of the Patient Health Questionnaire-9: A study of the participants from the spinal cord injury model systems. PM R, 3(6), 533–540. quiz 540.CrossRefPubMedGoogle Scholar
  32. 32.
    Jensen, M. P., Smith, A. E., Bombardier, C. H., Yorkston, K. M., Miro, J., & Molton, I. R. (2014). Social support, depression, and physical disability: Age and diagnostic group effects. Disability and Health Journal, 7(2), 164–172.CrossRefPubMedGoogle Scholar
  33. 33.
    Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39(8), 1–30.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176.CrossRefGoogle Scholar
  35. 35.
    Samejima, F. (1969). Estimation of Latent Ability Using a Response Pattern of Graded Scores (Psychometric Monograph no. 17). Richmond, VA: Psychometric Society. Retrieved from Scholar
  36. 36.
    Lance, C., Butts, M., & Michels, L. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9, 202–220.CrossRefGoogle Scholar
  37. 37.
    West, S. G., Taylor, A. B., & Wu, W. (2012). Model fit and model selection in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 209–231). New York, NY: Guilford Press.Google Scholar
  38. 38.
    Cook, K. F., Kallen, M. A., & Amtmann, D. (2009). Having a fit: Impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research, 18(4), 447–460.CrossRefPubMedPubMedCentralGoogle Scholar
  39. 39.
    Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.Google Scholar
  40. 40.
    Hu, L., & Bentler, P. M. (1988). Fit indices in covariance structure modeling: Sensitivity to underparameterization model misspecification. Psychological Methods, 3, 424–453.CrossRefGoogle Scholar
  41. 41.
    Hu, L., & Bentler, P. M. (1996). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(s), 1–55.Google Scholar
  42. 42.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.CrossRefPubMedGoogle Scholar
  43. 43.
    Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  44. 44.
    Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type(ordinal) item scores. Ottawa: Directorate of Human Resources Research and Evaluation, Department of National Defense.Google Scholar
  45. 45.
    Jodoin, M. G., & Gierl, M. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329–349.CrossRefGoogle Scholar
  46. 46.
    Kim, S. H., Cohen, A. S., Alagoz, C., & Kim, S. (2007). DIF detection effect size measures for polytomously scored items. Journal of Educational Measurement, 44, 93–116.CrossRefGoogle Scholar
  47. 47.
    Crane, P. K., van Belle, G., & Larson, E. B. (2004). Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23(2), 241–256.CrossRefPubMedGoogle Scholar
  48. 48.
    Crane, P. K., Gibbons, L. E., Ocepek-Welikson, K., Cook, K., Cella, D., Narasimhalu, K., et al. (2007). A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression. Quality of Life Research, 16(Suppl 1), 69–84.CrossRefPubMedGoogle Scholar
  49. 49.
    Koenig, H. G., George, L. K., Peterson, B. L., & Pieper, C. F. (1997). Depression in medically ill hospitalized older adults: Prevalence, characteristics, and course of symptoms according to six diagnostic schemes. American Journal of Psychiatry, 154(10), 1376–1383.CrossRefPubMedGoogle Scholar
  50. 50.
    Simon, G. E., & Von Korff, M. (2006). Medical co-morbidity and validity of DSM-IV depression criteria. Psychological Medicine, 36(1), 27–36.CrossRefPubMedGoogle Scholar
  51. 51.
    Williams, J. W, Jr., Noel, P. H., Cordes, J. A., Ramirez, G., & Pignone, M. (2002). Is this patient clinically depressed? JAMA, 287(9), 1160–1170.CrossRefPubMedGoogle Scholar
  52. 52.
    Fann, J. R., Bombardier, C. H., Richards, J. S., Wilson, C. S., Heinemann, A. W., Warren, A. M., et al. (2015). Venlafaxine extended-release for depression following spinal cord injury: A randomized clinical trial. JAMA Psychiatry, 72(3), 247–258.CrossRefPubMedGoogle Scholar
  53. 53.
    Perkes, S. J., Bowman, J., & Penkala, S. (2014). Psychological therapies for the management of co-morbid depression following a spinal cord injury: A systematic review. Journal of Health Psychology, 19(12), 1597–1612.CrossRefPubMedGoogle Scholar
  54. 54.
    Coventry, P. A., Hudson, J. L., Kontopantelis, E., Archer, J., Richards, D. A., Gilbody, S., et al. (2014). Characteristics of effective collaborative care for treatment of depression: A systematic review and meta-regression of 74 randomised controlled trials. PLoS ONE, 9(9), e108114.CrossRefPubMedPubMedCentralGoogle Scholar
  55. 55.
    Gilbody, S., Bower, P., Fletcher, J., Richards, D., & Sutton, A. J. (2006). Collaborative care for depression: A cumulative meta-analysis and review of longer-term outcomes. Archives of Internal Medicine, 166(21), 2314–2321.CrossRefPubMedGoogle Scholar
  56. 56.
    Gilbody, S., Whitty, P., Grimshaw, J., & Thomas, R. (2003). Educational and organizational interventions to improve the management of depression in primary care: A systematic review. JAMA, 289(23), 3145–3151.CrossRefPubMedGoogle Scholar
  57. 57.
    Green, C., Richards, D. A., Hill, J. J., Gask, L., Lovell, K., Chew-Graham, C., et al. (2014). Cost-effectiveness of collaborative care for depression in UK primary care: Economic evaluation of a randomised controlled trial (CADET). PLoS ONE, 9(8), e104225.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Karon F. Cook
    • 1
  • Michael A. Kallen
    • 1
  • Charles Bombardier
    • 2
  • Alyssa M. Bamer
    • 2
  • Seung W. Choi
    • 3
  • Jiseon Kim
    • 2
  • Rana Salem
    • 2
  • Dagmar Amtmann
    • 2
  1. 1.Department of Medical Social Sciences, Feinberg School of MedicineNorthwestern UniversityChicagoUSA
  2. 2.Department of Rehabilitation Medicine, School of MedicineUniversity of WashingtonSeattleUSA
  3. 3.Data Recognition Corporation | CTBMontereyUSA

Personalised recommendations