Screening for mental disorders in heart failure patients using computer-adaptive tests

Abstract

Purpose

Item response theory is increasingly used in the development of psychometric tests. This paper evaluates whether these modern psychometric methods can improve self-reported screening for depression and anxiety in patients with heart failure.

Methods

The mental health status of 194 patients with heart failure was assessed using six screening tools for depression (Patient Health Questionnaire -9 (9 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Depression Short Form 8a (8 items)) and Anxiety (GAD-7 (7 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Anxiety Short Form 8a (8 items)). An in-person structured clinical interview was used as the current gold standard to identify the presence of a mental disorder. The diagnostic accuracy of all static tools was compared when item response theory (IRT)-based person parameter were estimated instead of sum scores. Furthermore, we compared performance of static instruments with post hoc simulated individual-tailored computer-adaptive test (CATs) for both disorders and a common negative affect CAT.

Results

In general, screening for depression was highly efficient and showed a better performance than screening for anxiety with only minimal differences among the assessed instruments. IRT-based person parameters yielded the same diagnostic accuracy as sum scores. CATs showed similar screening performance compared to legacy instruments but required significantly fewer items to identify patients without mental conditions. Ideal cutoffs varied between male and female samples.

Conclusions

Overall, the diagnostic performance of all investigated instruments was similar, regardless of the methods being used. However, CATs can individually tailor the test to each patient, thus significantly decreasing the respondent burden for patients with and without mental conditions. Such approach could efficiently increase the acceptability of mental health screening in clinical practice settings.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. 1.

    Lichtman, J. H., Bigger, J. T., Blumenthal, J. A., Frasure-Smith, N., Kaufmann, P. G., Lespérance, F., et al. (2008). Depression and coronary heart disease: Recommendations for screening, referral, and treatment. Circulation, 118(17), 1768–1775. doi:10.1161/CIRCULATIONAHA.108.190769.

    PubMed  Article  Google Scholar 

  2. 2.

    Hasnain, M., Vieweg, W. V. R., Lesnefsky, E. J., & Pandurangi, A. K. (2011). Depression screening in patients with coronary heart disease: A critical evaluation of the AHA guidelines. Journal of Psychosomatic Research, 71(1), 6–12. doi:10.1016/j.jpsychores.2010.10.009.

    PubMed  Article  Google Scholar 

  3. 3.

    Thombs, B. D., Roseman, M., Coyne, J. C., de Jonge, P., Delisle, V. C., Arthurs, E., et al. (2013). Does evidence support the American Heart Association’s recommendation to screen patients for depression in cardiovascular care? An updated systematic review. PloS one, 8(1), e52654. doi:10.1371/journal.pone.0052654.

    CAS  Article  Google Scholar 

  4. 4.

    Thombs, B., Jonge, P. De, Coyne, J., Whooley, M., Frasure-Smith, N., Mitchell, A., et al. (2008). Depression screening and patient outcomes in cardiovascular care. JAMA, 300 (18).

  5. 5.

    Whooley, M. A., & Wong, J. M. (2013). Depression and cardiovascular disorders. Annual Review of Clinical Psychology, 9, 327–354. doi:10.1146/annurev-clinpsy-050212-185526.

    PubMed  Article  Google Scholar 

  6. 6.

    Herrmann, C., Scholz, K., & Kreuzer, H. (1991). Psychologic screening of patients of a cardiologic acute care clinic with the German version of the Hospital Anxiety and Depression Scale. Psychotherapie, Psychosomatik, Medizinische Psychologie, 41(2), 83–92.

    CAS  PubMed  Google Scholar 

  7. 7.

    Holzapfel, N., Zugck, C., Müller-Tasch, T., Löwe, B., Wild, B., Schellberg, D., et al. (2007). Routine screening for depression and quality of life in outpatients with congestive heart failure. Psychosomatics, 48(2), 112–116. doi:10.1176/appi.psy.48.2.112.

    PubMed  Article  Google Scholar 

  8. 8.

    Streiner, D. L. (2010). Measure for measure: New developments in measurement and item response theory. Canadian Journal of Psychiatry, 55(3), 180–186.

    Google Scholar 

  9. 9.

    Thomas, M. L. (2010). The value of item response theory in clinical assessment: A review. Assessment, 18(3), 291–307. doi:10.1177/1073191110374797.

    PubMed  Article  Google Scholar 

  10. 10.

    Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27–48. doi:10.1146/annurev.clinpsy.032408.153553.

    PubMed  Article  Google Scholar 

  11. 11.

    Linacre, J. M. (2000). Computer-adaptive testing: A methodology whose time has come.

  12. 12.

    Cella, D., Gershon, R., Lai, J.-S., Choi, S., Yount, S., Rothrock, N., et al. (2007). The future of outcomes measurement: Item banking, tailored short-forms, and computerized adaptive assessment. Quality of Life Research, 16 Suppl 1(5 Suppl 1), 133–41. doi:10.1007/s11136-007-9204-6.

  13. 13.

    Bjorner, J. B., Chang, C.-H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 95–108. doi:10.1007/s11136-007-9168-6.

    PubMed  Article  Google Scholar 

  14. 14.

    Gibbons, L. E., Feldman, B. J., Crane, H. M., Mugavero, M., Willig, J. H., Patrick, D., et al. (2011). Migrating from a legacy fixed-format measure to CAT administration: Calibrating the PHQ-9 to the PROMIS depression measures. Quality of Life Research, 20(9), 1349–1357. doi:10.1007/s11136-011-9882-y.

    PubMed Central  PubMed  Article  Google Scholar 

  15. 15.

    Choi, S. W., Reise, S. P., Pilkonis, P. A., Hays, R. D., & Cella, D. (2010). Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Quality of Life Research, 19(1), 125–136. doi:10.1007/s11136-009-9560-5.

    PubMed Central  PubMed  Article  Google Scholar 

  16. 16.

    Fliege, H., Becker, J., Walter, O. B., Rose, M., Bjorner, J. B., & Klapp, B. F. (2009). Evaluation of a computer-adaptive test for the assessment of depression (D-CAT) in clinical application. International Journal of Methods in Psychiatric Research, 18(1), 23–36. doi:10.1002/mpr.274.

    PubMed  Article  Google Scholar 

  17. 17.

    Kocalevent, R.-D., Rose, M., Becker, J., Walter, O. B., Fliege, H., Bjorner, J. B., et al. (2009). An evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception. Journal of Clinical Epidemiology, 62(3), 278–87, 287.e1–3. doi:10.1016/j.jclinepi.2008.03.003.

    Google Scholar 

  18. 18.

    Garcia, S. F., Cella, D., Clauser, S. B., Flynn, K. E., Lad, T., Lai, J., et al. (2010). Standardizing patient-reported outcomes assessment in cancer clinical trials : A patient-reported outcomes measurement information system initiative. Journal of Clinical Oncology, 25(32), 5106–5112. doi:10.1200/JCO.2007.12.2341.

    Article  Google Scholar 

  19. 19.

    Cella, D., Yount, S., Rothrock, N., & Gershon, R. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5), 3–11.

    Article  Google Scholar 

  20. 20.

    Rose, M., Anatchkova, M., Fletcher, J., Blank, A., Bjorner, J. B., Löwe, B., et al. (2012). Short and precise patient self-assessment of heart failure symptoms using a computerized adaptive test. Circulation. Heart Failure, 5(3), 331–339. doi:10.1161/CIRCHEARTFAILURE.111.964916.

    PubMed  Article  Google Scholar 

  21. 21.

    Spitzer, R. L. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. JAMA, 282(18), 1737–1744. doi:10.1001/jama.282.18.1737.

    CAS  PubMed  Article  Google Scholar 

  22. 22.

    Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  23. 23.

    Löwe, B., Gräfe, K., Zipfel, S., Witte, S., Loerch, B., & Herzog, W. (2004). Diagnosing ICD-10 depressive episodes: Superior criterion validity of the Patient Health Questionnaire. Psychotherapy and Psychosomatics, 73(6), 386–390. doi:10.1159/000080393.

    PubMed  Article  Google Scholar 

  24. 24.

    Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. doi:10.1001/archinte.166.10.1092.

    PubMed  Article  Google Scholar 

  25. 25.

    Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., et al. (2008). Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Medical Care, 46(3), 266–274. doi:10.1097/MLR.0b013e318160d093.

    PubMed  Article  Google Scholar 

  26. 26.

    Kroenke, K., Spitzer, R., Williams, J. B. W., Monahan, P., & Löwe, B. (2007). Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine, 146, 317–326.

    PubMed  Article  Google Scholar 

  27. 27.

    Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002). The validity of the Hospital Anxiety and Depression Scale. An updated literature review. Journal of Psychosomatic Research, 52(2), 69–77.

    PubMed  Article  Google Scholar 

  28. 28.

    Pilkonis, P. A., Choi, S. W. S., Reise, S. P. S., Stover, A. M., Riley, W. T., & Cella, D. (2011). Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS®): Depression, anxiety, and anger. Assessment, 18(3), 263–283. doi:10.1177/1073191111411667.

    PubMed Central  PubMed  Article  Google Scholar 

  29. 29.

    Patient-reported outcomes measurement information system. (n.d.). PROMIS depression scoring manual. Web Page. Retrieved from https://www.assessmentcenter.net/documents/PROMIS Depression scoring manual.pdf.

  30. 30.

    Patient-reported outcomes measurement information system. (n.d.). PROMIS anxiety scoring manual. Web Page. Retrieved from https://www.assessmentcenter.net/documents/PROMIS anxiety scoring manual.pdf.

  31. 31.

    Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The Patient Health Questionnaire somatic, anxiety, and depressive symptom scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359. doi:10.1016/j.genhosppsych.2010.03.006.

    PubMed  Article  Google Scholar 

  32. 32.

    Wittkampf, K. A., Naeije, L., Schene, A. H., Huyser, J., & van Weert, H. C. (2007). Diagnostic accuracy of the mood module of the Patient Health Questionnaire: A systematic review. General Hospital Psychiatry, 29(5), 388–395. doi:10.1016/j.genhosppsych.2007.06.004.

    PubMed  Article  Google Scholar 

  33. 33.

    Wittchen, H. U., Zaudig, M., & Fydrich, T. (1997). Strukturiertes Klinisches Interview für DSM-IV. Göttingen: Hogrefe. Göttingen: Hogrefe.

  34. 34.

    Wahl, I., Löwe, B., Bjorner, J. B., Fischer, H. F., Langs, G., Voderholzer, U., et al. (9999). Standardization of depression measurement: A common metric for eleven self-report depression scales. Journal of Clinical Epidemiology.

  35. 35.

    Fliege, H., Becker, J., Walter, O. B., Bjorner, J. B., Klapp, B. F., & Rose, M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14(10), 2277–2291. doi:10.1007/s11136-005-6651-9.

    PubMed  Article  Google Scholar 

  36. 36.

    Rose, M., Bjorner, J. B., Becker, J., Fries, J. F., & Ware, J. E. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology, 61(1), 17–33. doi:10.1016/j.jclinepi.2006.06.025.

    CAS  PubMed  Article  Google Scholar 

  37. 37.

    Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2).

  38. 38.

    Brown, T. A., & Kenny, D. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.

    Google Scholar 

  39. 39.

    Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611–630.

    Article  Google Scholar 

  40. 40.

    Santor, D. A., & Ramsay, J. O. (1998). Progress in the technology of measurement: Applications of item response models. Psychological Assessment, 10(4), 345–359. doi:10.1037//1040-3590.10.4.345.

    Article  Google Scholar 

  41. 41.

    Clark, L. A., & Watson, D. (1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336.

    CAS  PubMed  Article  Google Scholar 

  42. 42.

    Choi, S. W. (2009). FIRESTAR: Computerized Adaptive Testing (CAT) simulation program for polytomous IRT models. Retrieved September 03, 2013, from http://www.nihpromis.org/resources/resourcehome.

  43. 43.

    Choi, S. W. (2009). Firestar : Computerized adaptive testing simulation program response theory models. Applied Psychological Measurement, 33(8), 644–645.

    Article  Google Scholar 

  44. 44.

    Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  45. 45.

    DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3), 837–845.

    CAS  PubMed  Article  Google Scholar 

  46. 46.

    R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

  47. 47.

    Raiche, G. (2010). nFactors: An R package for parallel analysis and non graphical solutions to the Cattell scree test. R package.

  48. 48.

    Chalmers, R. P. (2012). mirt: a multidimensional item response theory package for the r Environment. Journal of Statistical Software, 48(6).

  49. 49.

    Wickham, H. (2009). ggplot2. New York: Springer.

    Book  Google Scholar 

  50. 50.

    Mazza, A., Punzo, A., & McGuire, B. (2012). KernSmoothIRT: Nonparametric item response theory. R package version 3.0.

  51. 51.

    Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response analysis. Journal of Statistical Software, 17, 1–25.

    Article  Google Scholar 

  52. 52.

    Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., et al. (2011). pROC: An open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 77. doi:10.1186/1471-2105-12-77.

    PubMed Central  PubMed  Article  Google Scholar 

  53. 53.

    Haworth, J. E., Moniz-Cook, E., Clark, A. L., Wang, M., Waddington, R., & Cleland, J. G. F. (2005). Prevalence and predictors of anxiety and depression in a sample of chronic heart failure patients with left ventricular systolic dysfunction. European Journal of Heart Failure, 7(5), 803–808. doi:10.1016/j.ejheart.2005.03.001.

    CAS  PubMed  Article  Google Scholar 

  54. 54.

    Chauvet-Gélinier, J.-C., Trojak, B., Vergès-Patois, B., Cottin, Y., & Bonin, B. (2013). Review on depression and coronary heart disease. Archives of cardiovascular diseases, 106(2), 103–110. doi:10.1016/j.acvd.2012.12.004.

    PubMed  Article  Google Scholar 

  55. 55.

    Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(Suppl 1), 5–18. doi:10.1007/s11136-007-9198-0.

    PubMed  Article  Google Scholar 

  56. 56.

    Thissen, D., Pommerich, M., Billeaud, K., & Williams, V. S. L. (1995). Item response theory for scores on tests including polytocous items with ordered responses. Applied Psychological Measurement, 19(1), 39–49. doi:10.1177/014662169501900105.

    Article  Google Scholar 

  57. 57.

    Shin, J. J., Hamad, E., Murthy, S., & Piña, I. L. (2012). Heart failure in women. Clinical Cardiology, 35(3), 172–177. doi:10.1002/clc.21973.

    PubMed  Article  Google Scholar 

  58. 58.

    Strömberg, A., & Martensson, J. (2003). Gender differences in patients with heart failure. European Journal of Cardiovascular Nursing, 2(1), 7–18.

    PubMed  Article  Google Scholar 

Download references

Acknowledgement

The study was funded as part of the RECODE-HF study by the German Federal Ministry of Education and Research under file number 01GY1150 and approved by the ethics committee of the University Hospital Göttingen under file number 19/8/11.

Conflict of interest

CHL is receiving royalties from Hans Huber Publishers, Berne, Switzerland, for the German version of the Hospital Anxiety and Depression Scale HADS. All other authors state that there are no conflicts of interest.

Author information

Affiliations

Authors

Corresponding author

Correspondence to H. Felix Fischer.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 101 kb)

Supplementary material 2 (TIFF 4502 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Fischer, H.F., Klug, C., Roeper, K. et al. Screening for mental disorders in heart failure patients using computer-adaptive tests. Qual Life Res 23, 1609–1618 (2014). https://doi.org/10.1007/s11136-013-0599-y

Download citation

Keywords

  • Depression
  • Anxiety
  • Diagnostic method
  • Item response theory