Quality of Life Research

, Volume 23, Issue 5, pp 1609–1618 | Cite as

Screening for mental disorders in heart failure patients using computer-adaptive tests

  • H. Felix Fischer
  • Cassandra Klug
  • Koosje Roeper
  • Eva Blozik
  • Frank Edelmann
  • Marion Eisele
  • Stefan Störk
  • Rolf Wachter
  • Martin Scherer
  • Matthias Rose
  • Christoph Herrmann-Lingen



Item response theory is increasingly used in the development of psychometric tests. This paper evaluates whether these modern psychometric methods can improve self-reported screening for depression and anxiety in patients with heart failure.


The mental health status of 194 patients with heart failure was assessed using six screening tools for depression (Patient Health Questionnaire -9 (9 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Depression Short Form 8a (8 items)) and Anxiety (GAD-7 (7 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Anxiety Short Form 8a (8 items)). An in-person structured clinical interview was used as the current gold standard to identify the presence of a mental disorder. The diagnostic accuracy of all static tools was compared when item response theory (IRT)-based person parameter were estimated instead of sum scores. Furthermore, we compared performance of static instruments with post hoc simulated individual-tailored computer-adaptive test (CATs) for both disorders and a common negative affect CAT.


In general, screening for depression was highly efficient and showed a better performance than screening for anxiety with only minimal differences among the assessed instruments. IRT-based person parameters yielded the same diagnostic accuracy as sum scores. CATs showed similar screening performance compared to legacy instruments but required significantly fewer items to identify patients without mental conditions. Ideal cutoffs varied between male and female samples.


Overall, the diagnostic performance of all investigated instruments was similar, regardless of the methods being used. However, CATs can individually tailor the test to each patient, thus significantly decreasing the respondent burden for patients with and without mental conditions. Such approach could efficiently increase the acceptability of mental health screening in clinical practice settings.


Depression Anxiety Diagnostic method Item response theory 



The study was funded as part of the RECODE-HF study by the German Federal Ministry of Education and Research under file number 01GY1150 and approved by the ethics committee of the University Hospital Göttingen under file number 19/8/11.

Conflict of interest

CHL is receiving royalties from Hans Huber Publishers, Berne, Switzerland, for the German version of the Hospital Anxiety and Depression Scale HADS. All other authors state that there are no conflicts of interest.

Supplementary material

11136_2013_599_MOESM1_ESM.doc (102 kb)
Supplementary material 1 (DOC 101 kb)
11136_2013_599_MOESM2_ESM.tiff (4.4 mb)
Supplementary material 2 (TIFF 4502 kb)


  1. 1.
    Lichtman, J. H., Bigger, J. T., Blumenthal, J. A., Frasure-Smith, N., Kaufmann, P. G., Lespérance, F., et al. (2008). Depression and coronary heart disease: Recommendations for screening, referral, and treatment. Circulation, 118(17), 1768–1775. doi: 10.1161/CIRCULATIONAHA.108.190769.PubMedCrossRefGoogle Scholar
  2. 2.
    Hasnain, M., Vieweg, W. V. R., Lesnefsky, E. J., & Pandurangi, A. K. (2011). Depression screening in patients with coronary heart disease: A critical evaluation of the AHA guidelines. Journal of Psychosomatic Research, 71(1), 6–12. doi: 10.1016/j.jpsychores.2010.10.009.PubMedCrossRefGoogle Scholar
  3. 3.
    Thombs, B. D., Roseman, M., Coyne, J. C., de Jonge, P., Delisle, V. C., Arthurs, E., et al. (2013). Does evidence support the American Heart Association’s recommendation to screen patients for depression in cardiovascular care? An updated systematic review. PloS one, 8(1), e52654. doi: 10.1371/journal.pone.0052654.CrossRefGoogle Scholar
  4. 4.
    Thombs, B., Jonge, P. De, Coyne, J., Whooley, M., Frasure-Smith, N., Mitchell, A., et al. (2008). Depression screening and patient outcomes in cardiovascular care. JAMA, 300 (18).Google Scholar
  5. 5.
    Whooley, M. A., & Wong, J. M. (2013). Depression and cardiovascular disorders. Annual Review of Clinical Psychology, 9, 327–354. doi: 10.1146/annurev-clinpsy-050212-185526.PubMedCrossRefGoogle Scholar
  6. 6.
    Herrmann, C., Scholz, K., & Kreuzer, H. (1991). Psychologic screening of patients of a cardiologic acute care clinic with the German version of the Hospital Anxiety and Depression Scale. Psychotherapie, Psychosomatik, Medizinische Psychologie, 41(2), 83–92.PubMedGoogle Scholar
  7. 7.
    Holzapfel, N., Zugck, C., Müller-Tasch, T., Löwe, B., Wild, B., Schellberg, D., et al. (2007). Routine screening for depression and quality of life in outpatients with congestive heart failure. Psychosomatics, 48(2), 112–116. doi: 10.1176/appi.psy.48.2.112.PubMedCrossRefGoogle Scholar
  8. 8.
    Streiner, D. L. (2010). Measure for measure: New developments in measurement and item response theory. Canadian Journal of Psychiatry, 55(3), 180–186.Google Scholar
  9. 9.
    Thomas, M. L. (2010). The value of item response theory in clinical assessment: A review. Assessment, 18(3), 291–307. doi: 10.1177/1073191110374797.PubMedCrossRefGoogle Scholar
  10. 10.
    Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27–48. doi: 10.1146/annurev.clinpsy.032408.153553.PubMedCrossRefGoogle Scholar
  11. 11.
    Linacre, J. M. (2000). Computer-adaptive testing: A methodology whose time has come.Google Scholar
  12. 12.
    Cella, D., Gershon, R., Lai, J.-S., Choi, S., Yount, S., Rothrock, N., et al. (2007). The future of outcomes measurement: Item banking, tailored short-forms, and computerized adaptive assessment. Quality of Life Research, 16 Suppl 1(5 Suppl 1), 133–41. doi: 10.1007/s11136-007-9204-6.
  13. 13.
    Bjorner, J. B., Chang, C.-H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 95–108. doi: 10.1007/s11136-007-9168-6.PubMedCrossRefGoogle Scholar
  14. 14.
    Gibbons, L. E., Feldman, B. J., Crane, H. M., Mugavero, M., Willig, J. H., Patrick, D., et al. (2011). Migrating from a legacy fixed-format measure to CAT administration: Calibrating the PHQ-9 to the PROMIS depression measures. Quality of Life Research, 20(9), 1349–1357. doi: 10.1007/s11136-011-9882-y.PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Choi, S. W., Reise, S. P., Pilkonis, P. A., Hays, R. D., & Cella, D. (2010). Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Quality of Life Research, 19(1), 125–136. doi: 10.1007/s11136-009-9560-5.PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Fliege, H., Becker, J., Walter, O. B., Rose, M., Bjorner, J. B., & Klapp, B. F. (2009). Evaluation of a computer-adaptive test for the assessment of depression (D-CAT) in clinical application. International Journal of Methods in Psychiatric Research, 18(1), 23–36. doi: 10.1002/mpr.274.PubMedCrossRefGoogle Scholar
  17. 17.
    Kocalevent, R.-D., Rose, M., Becker, J., Walter, O. B., Fliege, H., Bjorner, J. B., et al. (2009). An evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception. Journal of Clinical Epidemiology, 62(3), 278–87, 287.e1–3. doi: 10.1016/j.jclinepi.2008.03.003.Google Scholar
  18. 18.
    Garcia, S. F., Cella, D., Clauser, S. B., Flynn, K. E., Lad, T., Lai, J., et al. (2010). Standardizing patient-reported outcomes assessment in cancer clinical trials : A patient-reported outcomes measurement information system initiative. Journal of Clinical Oncology, 25(32), 5106–5112. doi: 10.1200/JCO.2007.12.2341.CrossRefGoogle Scholar
  19. 19.
    Cella, D., Yount, S., Rothrock, N., & Gershon, R. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5), 3–11.CrossRefGoogle Scholar
  20. 20.
    Rose, M., Anatchkova, M., Fletcher, J., Blank, A., Bjorner, J. B., Löwe, B., et al. (2012). Short and precise patient self-assessment of heart failure symptoms using a computerized adaptive test. Circulation. Heart Failure, 5(3), 331–339. doi: 10.1161/CIRCHEARTFAILURE.111.964916.PubMedCrossRefGoogle Scholar
  21. 21.
    Spitzer, R. L. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. JAMA, 282(18), 1737–1744. doi: 10.1001/jama.282.18.1737.PubMedCrossRefGoogle Scholar
  22. 22.
    Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.PubMedCentralPubMedCrossRefGoogle Scholar
  23. 23.
    Löwe, B., Gräfe, K., Zipfel, S., Witte, S., Loerch, B., & Herzog, W. (2004). Diagnosing ICD-10 depressive episodes: Superior criterion validity of the Patient Health Questionnaire. Psychotherapy and Psychosomatics, 73(6), 386–390. doi: 10.1159/000080393.PubMedCrossRefGoogle Scholar
  24. 24.
    Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. doi: 10.1001/archinte.166.10.1092.PubMedCrossRefGoogle Scholar
  25. 25.
    Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., et al. (2008). Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Medical Care, 46(3), 266–274. doi: 10.1097/MLR.0b013e318160d093.PubMedCrossRefGoogle Scholar
  26. 26.
    Kroenke, K., Spitzer, R., Williams, J. B. W., Monahan, P., & Löwe, B. (2007). Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine, 146, 317–326.PubMedCrossRefGoogle Scholar
  27. 27.
    Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002). The validity of the Hospital Anxiety and Depression Scale. An updated literature review. Journal of Psychosomatic Research, 52(2), 69–77.PubMedCrossRefGoogle Scholar
  28. 28.
    Pilkonis, P. A., Choi, S. W. S., Reise, S. P. S., Stover, A. M., Riley, W. T., & Cella, D. (2011). Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS®): Depression, anxiety, and anger. Assessment, 18(3), 263–283. doi: 10.1177/1073191111411667.PubMedCentralPubMedCrossRefGoogle Scholar
  29. 29.
    Patient-reported outcomes measurement information system. (n.d.). PROMIS depression scoring manual. Web Page. Retrieved from Depression scoring manual.pdf.
  30. 30.
    Patient-reported outcomes measurement information system. (n.d.). PROMIS anxiety scoring manual. Web Page. Retrieved from anxiety scoring manual.pdf.
  31. 31.
    Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The Patient Health Questionnaire somatic, anxiety, and depressive symptom scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359. doi: 10.1016/j.genhosppsych.2010.03.006.PubMedCrossRefGoogle Scholar
  32. 32.
    Wittkampf, K. A., Naeije, L., Schene, A. H., Huyser, J., & van Weert, H. C. (2007). Diagnostic accuracy of the mood module of the Patient Health Questionnaire: A systematic review. General Hospital Psychiatry, 29(5), 388–395. doi: 10.1016/j.genhosppsych.2007.06.004.PubMedCrossRefGoogle Scholar
  33. 33.
    Wittchen, H. U., Zaudig, M., & Fydrich, T. (1997). Strukturiertes Klinisches Interview für DSM-IV. Göttingen: Hogrefe. Göttingen: Hogrefe.Google Scholar
  34. 34.
    Wahl, I., Löwe, B., Bjorner, J. B., Fischer, H. F., Langs, G., Voderholzer, U., et al. (9999). Standardization of depression measurement: A common metric for eleven self-report depression scales. Journal of Clinical Epidemiology.Google Scholar
  35. 35.
    Fliege, H., Becker, J., Walter, O. B., Bjorner, J. B., Klapp, B. F., & Rose, M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14(10), 2277–2291. doi: 10.1007/s11136-005-6651-9.PubMedCrossRefGoogle Scholar
  36. 36.
    Rose, M., Bjorner, J. B., Becker, J., Fries, J. F., & Ware, J. E. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology, 61(1), 17–33. doi: 10.1016/j.jclinepi.2006.06.025.PubMedCrossRefGoogle Scholar
  37. 37.
    Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2).Google Scholar
  38. 38.
    Brown, T. A., & Kenny, D. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.Google Scholar
  39. 39.
    Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611–630.CrossRefGoogle Scholar
  40. 40.
    Santor, D. A., & Ramsay, J. O. (1998). Progress in the technology of measurement: Applications of item response models. Psychological Assessment, 10(4), 345–359. doi: 10.1037//1040-3590.10.4.345.CrossRefGoogle Scholar
  41. 41.
    Clark, L. A., & Watson, D. (1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336.PubMedCrossRefGoogle Scholar
  42. 42.
    Choi, S. W. (2009). FIRESTAR: Computerized Adaptive Testing (CAT) simulation program for polytomous IRT models. Retrieved September 03, 2013, from
  43. 43.
    Choi, S. W. (2009). Firestar : Computerized adaptive testing simulation program response theory models. Applied Psychological Measurement, 33(8), 644–645.CrossRefGoogle Scholar
  44. 44.
    Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  45. 45.
    DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3), 837–845.PubMedCrossRefGoogle Scholar
  46. 46.
    R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
  47. 47.
    Raiche, G. (2010). nFactors: An R package for parallel analysis and non graphical solutions to the Cattell scree test. R package.Google Scholar
  48. 48.
    Chalmers, R. P. (2012). mirt: a multidimensional item response theory package for the r Environment. Journal of Statistical Software, 48(6).Google Scholar
  49. 49.
    Wickham, H. (2009). ggplot2. New York: Springer.CrossRefGoogle Scholar
  50. 50.
    Mazza, A., Punzo, A., & McGuire, B. (2012). KernSmoothIRT: Nonparametric item response theory. R package version 3.0.Google Scholar
  51. 51.
    Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response analysis. Journal of Statistical Software, 17, 1–25.CrossRefGoogle Scholar
  52. 52.
    Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., et al. (2011). pROC: An open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 77. doi: 10.1186/1471-2105-12-77.PubMedCentralPubMedCrossRefGoogle Scholar
  53. 53.
    Haworth, J. E., Moniz-Cook, E., Clark, A. L., Wang, M., Waddington, R., & Cleland, J. G. F. (2005). Prevalence and predictors of anxiety and depression in a sample of chronic heart failure patients with left ventricular systolic dysfunction. European Journal of Heart Failure, 7(5), 803–808. doi: 10.1016/j.ejheart.2005.03.001.PubMedCrossRefGoogle Scholar
  54. 54.
    Chauvet-Gélinier, J.-C., Trojak, B., Vergès-Patois, B., Cottin, Y., & Bonin, B. (2013). Review on depression and coronary heart disease. Archives of cardiovascular diseases, 106(2), 103–110. doi: 10.1016/j.acvd.2012.12.004.PubMedCrossRefGoogle Scholar
  55. 55.
    Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(Suppl 1), 5–18. doi: 10.1007/s11136-007-9198-0.PubMedCrossRefGoogle Scholar
  56. 56.
    Thissen, D., Pommerich, M., Billeaud, K., & Williams, V. S. L. (1995). Item response theory for scores on tests including polytocous items with ordered responses. Applied Psychological Measurement, 19(1), 39–49. doi: 10.1177/014662169501900105.CrossRefGoogle Scholar
  57. 57.
    Shin, J. J., Hamad, E., Murthy, S., & Piña, I. L. (2012). Heart failure in women. Clinical Cardiology, 35(3), 172–177. doi: 10.1002/clc.21973.PubMedCrossRefGoogle Scholar
  58. 58.
    Strömberg, A., & Martensson, J. (2003). Gender differences in patients with heart failure. European Journal of Cardiovascular Nursing, 2(1), 7–18.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • H. Felix Fischer
    • 1
    • 7
  • Cassandra Klug
    • 2
    • 3
  • Koosje Roeper
    • 2
    • 3
  • Eva Blozik
    • 4
  • Frank Edelmann
    • 3
    • 5
  • Marion Eisele
    • 4
  • Stefan Störk
    • 6
  • Rolf Wachter
    • 3
    • 5
  • Martin Scherer
    • 4
  • Matthias Rose
    • 1
  • Christoph Herrmann-Lingen
    • 2
    • 3
  1. 1.Department of Psychosomatic Medicine, Clinic for Internal MedicineCharité – UniversitätsmedizinBerlinGermany
  2. 2.Department of Psychosomatic Medicine and PsychotherapyUniversity of Göttingen Medical CenterGöttingenGermany
  3. 3.German Center for Cardiovascular ResearchUniversity of Göttingen Medical CenterGöttingenGermany
  4. 4.Department of Primary Medical Care, Center for Psychosocial MedicineUniversity Medical Center Hamburg-EppendorfHamburgGermany
  5. 5.Clinic for Cardiology and PneumologyUniversity of Göttingen Medical CenterGöttingenGermany
  6. 6.Comprehensive Heart Failure Center, Department of Internal Medicine I - CardiologyUniversity Hospital WürzburgWürzburgGermany
  7. 7.Institute for Social Medicine, Epidemiology and Health EconomicsCharité – UniversitätsmedizinBerlinGermany

Personalised recommendations