Skip to main content

Advertisement

SpringerLink
Log in

Without Wasting a Word: Extreme Improvements in Efficiency and Accuracy Using Computerized Adaptive Testing for Mental Health Disorders (CAT-MH)

  • Psychiatry in the Digital Age (J Shore, Section Editor)
  • Published:
Current Psychiatry Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

We review recent literature on the adaptive assessment of complex mental health disorders and provide a detailed comparison of classical test theory and adaptive testing based on multidimensional item response theory.

Recent Findings

Adaptive tests for a wide variety of mental health traits (e.g., depression, anxiety, mania, substance misuse, suicidality) are now available in a cloud-based environment. These tests have been validated in a variety of settings against lengthy structured clinical interviews with excellent results and even higher reliability than fixed-length tests. Applications include screening and assessments in emergency departments, psychiatric and primary care clinics, student health clinics, perinatal medicine clinics, child welfare settings, and the judicial system.

Summary

The future of mental health measurement will be based on automated screening and assessments. Adaptive tests will provide increased precision of measurement and decreased burden of measurement. Integration into the electronic health record is important and now easily accomplished.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Gauss CF. Theoria motus corporum coelestium in sectionibus conicis solem ambien-tium. Perthes et Besser, Hamburg. Werke, 1809; 7: 1–280. Translated by C. H. Davis as Theory of the Motion of the Heavenly Bodies Moving about the Sun in Conic Sections. Little, Brown, Boston, 1857. Reprinted by Dover, New York, 1963.

  2. Spearman C. Demonstration of formulae for true measurement of correlation. Am J Psychol. 1907;18(2):161–9.

    Article  Google Scholar 

  3. • DN L. On problems connected with item selection and test construction. Proc Roy Soc Edinb. 1943;61:273–87. This is a foundational paper on item response theory.

    Google Scholar 

  4. • Lord, F. (1952). A Theory of Test Scores (Psychometric Monograph No. 7). Richmond, VA: Psychometric Corporation. Retrieved from www.psychometrika.org/journal/online/MN07.pdf. This is a foundational paper on item response theory.

  5. •• Gibbons RD Computerized adaptive diagnosis and testing of mental health disorders. Annu Rev Clin Psychol. 2016;12:83–104. This paper describes the development of MIRT-based CAT, providing both statistical and clinical details.

    Article  Google Scholar 

  6. Bock RD, Aitkin M. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika. 1981;46:443–59.

    Article  Google Scholar 

  7. •• Gibbons RD, Hedeker D. Full-information item bi-factor analysis. Psychometrika. 1992;57:423–36. This paper developed the bifactor IRT model.

    Article  Google Scholar 

  8. Gibbons RD, Bock RD, Hedeker D, Weiss D, Segawa E, et al. Full-information item bi-factor analysis of graded response data. Appl Psychol Meas. 2007;31(2007):4–19.

    Article  Google Scholar 

  9. •• Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T, et al. The CAT-DI: a computerized adaptive test for depression. Arch Gen Psychiatry. 2012;69:1104–12. This is the first paper to describe the development of an MIRT-based CAT for mental health measurement.

    Article  Google Scholar 

  10. Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T, et al. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiatr. 2014;171:187–94.

    Article  Google Scholar 

  11. Beiser D, Vu M, Gibbons RD. Test-retest reliability of a computerized adaptive depression test. Psychiatr Serv. 2016;67:1039–41.

    Article  Google Scholar 

  12. Sani S, Busnello J, Kochanski R, Cohen Y, Gibbons RD. High frequency measurement of depressive severity in a patient treated for severe treatment resistant depression with deep brain stimulation. Transl Psychiatry. 2017;7:e1207.

    Article  CAS  Google Scholar 

  13. Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, et al. PROMIS cooperative group. Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS): depression, anxiety, and anger. Assessment. 2011;18:263–83.

    Article  Google Scholar 

  14. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23:56–62.

    Article  CAS  Google Scholar 

  15. Gibbons RD, Hooker G, Finkelman MD, Weiss DJ, Pilkonis PA, et al. The CAD-MDD: a computerized adaptive diagnostic screening tool for depression. J Clin Psychiatry. 2013;74:669–74.

    Article  Google Scholar 

  16. Brieman L. Random forests. Mach Learn. 2001;45:5–32.

    Article  Google Scholar 

  17. Achtyes ED, Halstead S, Smart L, Moore T, Frank E, et al. Validation of computerized adaptive testing in an outpatient non-academic setting. Psychiatr Serv. 2015;66:1091–6.

    Article  Google Scholar 

  18. • Kim JJ, Silver RK, Elue R, Adams MG, La Porte LM, et al. The experience of depression, anxiety and mania among perinatal women. Arch Womens Ment Health. 2017;19:94–100. This paper introduced MIRT-based CAT for the measurement of perinatal depression, anxiety and mania.

    Google Scholar 

  19. • Gibbons RD, Kupfer D, Frank E, Moore T, Boudreaux E. Development of a computerized adaptive suicide scale. J Clin Psychiatry. 2017;78:1376–82. This paper introduced the first adaptive suicide scale.

    Article  Google Scholar 

  20. Gibbons RD, Alegria M, Cai L, Herrera L, Markle SL, et al. Successful validation of the CAT-MH scales in a sample of Latin American migrants in the United States and Spain: Psychological Assessments. Published on-line ahead of print 30(10), 1267–76.

  21. www.adaptivetestingtechnologies.com. August 22, 2017

  22. Heisey-Grove D, Patel VONC. Data brief: any, certified, and basic: quantifying physician EHR adoption through 2014. In: The Office of the National Coordinator for Health Information Technology, editor. ; 2015. p. 1–10.

    Google Scholar 

  23. Mobile Fact Sheet. 2017. (Accessed August 22, 2017, at http://www.pewinternet.org/fact-sheet/mobile/.)

  24. Internet/Broadband Fact Sheet. 2017. (Accessed August 22, 2017, at http://www.pewinternet.org/fact-sheet/internet-broadband/.)

  25. Byrne JM, Elliott S, Firek A. Initial experience with patient-clinician secure messaging at a VA medical center. JAMIA. 2009;16:267–70.

    PubMed  Google Scholar 

  26. Nijland N, van Gemert-Pijnen JE, Kelders SM, Brandenburg BJ, Seydel ER. Factors influencing the use of a web-based application for supporting the self-care of patients with type 2 diabetes: a longitudinal study. J Med Internet Res. 2011;13:e71.

    Article  Google Scholar 

  27. Nazi KM, Hogan TP, McInnes DK, Woods SS, Graham G. Evaluating patient access to electronic health records: results from a survey of veterans. Med Care. 2013;51:S52–6.

    Article  Google Scholar 

  28. Ketterer T, West DW, Sanders VP, Hossain J, Kondo MC, et al. Correlates of patient portal enrollment and activation in primary care pediatrics. Acad Pediatr. 2013;13:264–71.

    Article  Google Scholar 

  29. Lam R, Lin VS, Senelick WS, Tran HP, Moore AA, et al. Older adult consumers’ attitudes and preferences on electronic patient-physician messaging. Am J Manag Care. 2013;19:eSP7–11.

    PubMed  PubMed Central  Google Scholar 

  30. Neuner J, Fedders M, Caravella M, Bradford L, Schapira M. Meaningful use and the patient portal: patient enrollment, use, and satisfaction with patient portals at a later-adopting center. Am J Med Qual. 2015;30:105–13.

    Article  Google Scholar 

  31. • Beiser DJ, Ward CE, Vu M, Laiteerapong N, Gibbons RD. Depression in emergency department patients and association with healthcare utilization. Acad Emerg Med. Published on-line 18 March 2019 https://doi.org/10.1111/acem.13726. This paper introduced the use of CAT for emergency medicine.

    Article  Google Scholar 

  32. • Siu AL, the US Preventive Services Task Force. Screening for depression in adults: US Preventive Services Task Force recommendations statement. JAMA. 2016;315(4):380–7. This paper mandated depression screening in primary care.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

Robert Gibbons is a founder of Adaptive Testing Technologies which distributes the CAT-MH™ suite of tests. The terms of this arrangement have been reviewed and approved by the University of Chicago in accordance with its conflict of interest policies.

Funding

This work was supported by grants R01-MH-66302, R01-MH-100155, 2 R01-MH-100155-06, U01-MH-104311, and 1R44MH118780 from the National Institute of Mental Health.

Author information

Authors and Affiliations

  1. Blum-Riese Professor of Biostatistics, University of Chicago, Chicago, IL, USA

    Robert D. Gibbons

  2. Department of Family Medicine, University of Colorado, Aurora, CO, USA

    Frank V. deGruy

Authors
  1. Robert D. Gibbons
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank V. deGruy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert D. Gibbons.

Ethics declarations

Conflict of Interest

Frank V. deGruy declares no potential conflicts of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Psychiatry in the Digital Age

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gibbons, R.D., deGruy, F.V. Without Wasting a Word: Extreme Improvements in Efficiency and Accuracy Using Computerized Adaptive Testing for Mental Health Disorders (CAT-MH). Curr Psychiatry Rep 21, 67 (2019). https://doi.org/10.1007/s11920-019-1053-9

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s11920-019-1053-9

Keywords

Search

Navigation