Skip to main content

Advertisement

SpringerLink
Log in

The Usability of CAT System for Assessing the Depressive Level of Japanese—A Study on Psychometric Properties and Response Behavior

  • Published:
International Journal of Behavioral Medicine Aims and scope Submit manuscript

Abstract

Background

An innovative measurement system using a computerized adaptive testing technique based on the item response theory (CAT) has been expanding to measure mental health status. However, little is known about details in its measurement properties based on the empirical data. Moreover, the response time (RT) data, which are not available by a paper-and-pencil measurement but available by a computerized measurement, would be worth investigating for exploring the response behavior.

Purpose

We aimed at constructing the CAT to measure depressive symptomatology in a community population and exploring its measurement properties. Also, we examined the relationships between RTs, individual item responses, and depressive levels.

Method

For constructing the CAT system, responses of 2061 workers and university students to 24 depression scale plus four negatively revised positive affect items were subjected to a polytomous IRT analysis. The stopping rule was set for standard error of estimation < 0.30 or the maximum 15 items displayed. The CAT and non-adaptive computer-based test (CBT) were administered to 209 undergraduates, and 168 of them administered again after 1 week.

Results

On average, the CAT was converged by 10.4 items. The θ values estimated by CAT and CBT were highly correlated (r = 0.94 and 0.95 for the 1st and 2nd measurements) and with the traditional scoring procedures (r’s > 0.90). The test–retest reliability was at a satisfactory level (r = 0.86). RTs to some items significantly correlated with the θ estimates. The mean RT varied by the item contents and wording, i.e., the RT to positive affect items required additional 2 s or longer than the other subscale items.

Conclusion

The CAT would be a reliable and practical measurement tool for various purposes including stress check at workplace.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Ferrari AJ, Charlson FJ, Norman RE, et al. Burden of depressive disorders by country, sex, age, and year: findings from the Global Burden of Disease Study 2010. PLoS Med. 2013;10, e1001547. doi:10.1371/journal.pmed.1001547.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Gibbons RD, Weiss DJ, Pilkonis PA, et al. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiat. 2014;171:187–94.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Forkmann T, Boecker M, Wirtz M, et al. Validation of the Rasch-based depression screening in a large scale German general population sample. HQLO. 2010;8:105. http://www.hqlo.com/content/8/1/105

  4. Lord FM, Novick MR. Statistical theories of mental test scores. Reading: Addison-Welsley Publishing Company; 1968.

    Google Scholar 

  5. Lord FM. Applications of item response theory to practical testing problems. Erlbaum Associates; 1980

  6. Hambleton RK, Cook LL. Latent trait models and their use in the analysis of educational test data. J Educ Meas. 1977;14:75–96.

    Article  Google Scholar 

  7. Hambleton RK, Swaminathan H. Item response theory: principles and applications. Boston: Kluwer; 1985.

    Book  Google Scholar 

  8. Embretson SE, Reise SP. Item response theory for psychologists. Mahwah: Lawrence Erlbaum Associates, Inc., Publishers; 2000.

    Google Scholar 

  9. Wainer H, Dorans NJ, Eignor D, et al. Computerized adaptive testing: a primer. 2nd ed. Hillsdale: Erlbaum Associates; 2000.

    Google Scholar 

  10. Fliege H, Becker J, Walter OB, et al. Evaluation of a computer-adaptive test for the assessment of depression (D-CAT) in clinical application. Int J Methods Psychiatr Res. 2009;18:23–36.

    Article  PubMed  Google Scholar 

  11. Weiss DJ. Adaptive testing by computer. J Consul Clin Psychol. 1985;53:774–89.

    Article  CAS  Google Scholar 

  12. Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38 Suppl 9:73–82.

    Google Scholar 

  13. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002;40:812–23.

    Article  PubMed  Google Scholar 

  14. Fliege H, Becker J, Walter OB, et al. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14:2277–91.

    Article  PubMed  Google Scholar 

  15. Smits N, Cuijpers P, van Straten A. Applying computerized adaptive testing to the CES-D scale: a simulation study. Psychiat Res. 2011;188:147–55.

    Article  Google Scholar 

  16. American Psychological Association. Publication manual of the American Psychological Association, 5th ed. Washington, DC: Author; 2005.

  17. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psych Meas. 1977;1:385–401.

    Article  Google Scholar 

  18. Shima S, Shikano T, Kitamura T, et al. New self-rating scale for depression. Clinical Psychiatry. 1985;27:717–23 [In Japanese].

    Google Scholar 

  19. Iwata N, Umesue M, Egashira K, et al. Can positive affect items be used to assess depressive disorders in the Japanese population? Psychol Med. 1998;28:153–8.

    Article  CAS  PubMed  Google Scholar 

  20. Iwata N, Okuyama Y, Kawakami Y, et al. Prevalence of depressive symptoms in a Japanese occupational setting: a preliminary study. Am J Public Health. 1989;79:1486–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Iwata N, Saito K, Roberts RE. Responses to a self-administered depression scale among younger adolescents in Japan. Psychiat Res. 1994;53:275–87.

    Article  CAS  Google Scholar 

  22. Iwata N, Roberts CR, Kawakami N. Japan-U.S. comparison of responses to depression scale items among adult workers. Psychiat Res. 1995;58:237–45.

    Article  CAS  Google Scholar 

  23. Iwata N, Buka S. Race/ethnicity and depressive symptoms: a cross-cultural/ethnic comparison among university students in East Asia, North and South America. Soc Sci Med. 2002;55:2243–52.

    Article  PubMed  Google Scholar 

  24. Owen RJ. A bayesian sequential procedure for quantal response in the context of adaptive mental testing. J Am Stat Assoc. 1975;70:351–6.

    Article  Google Scholar 

  25. Muthén LK, Muthén BO. Mplus user’s guide. Los Angeles, CA: Muthén & Muthén; 2006.

  26. Samejima, F. Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph. 17, 1969.

  27. Muraki E, Bock RD. PARSCALE: IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software Int; 1993.

    Google Scholar 

  28. Costello EJ, Angold A. Scales to assess child and adolescent depression: checklists, screens, and nets. J Am Acad Child Psy. 1988;27:726–37.

    Article  CAS  Google Scholar 

  29. Roberts RE, Lewinsohn PM, Seeley JR. Screening for adolescent depression: a comparison of depression scales. J Am Acad Child Psy. 1991;30:58–66.

    Article  CAS  Google Scholar 

  30. Iwata N. Cultural distinctiveness in response bias. In: Dollard MF, Shimazu A, Nordin RB, Brough P, Tuckey MR, editors. Psychosocial factors at work in the Asia Pacific. New York: Springer; 2014. p. 55–88.

    Google Scholar 

  31. Walter OB, Becker J, Bjorner JB, et al. Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Qual Life Res. 2007;16(Suppl1):143–55.

    Article  PubMed  Google Scholar 

  32. Japan Industrial Safety and Health Law. Article 66–10 amended [Act No. 82 of June 25, 2014]

Download references

Acknowledgments

This study was partly supported by Grand-in-Aid for Scientific Research (C), from the Japan Ministry of Education, Culture, Sports, Science and Technology (project number 18590620) and Health and Labour Sciences Research Grants (Research on Occupational Safety and Health; H17-Rodo-Ippan-005, principle investigator: Norito Kawakami), from the Japan Ministry of Health, Labour and Welfare.

Ethical Standards

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. Informed consent was verbally obtained from all patients for being included in the study.

Conflict of Interest

Noboru Iwata, Kenichi Kikuchi, and Yuya Fujihara declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Department of Psychology, Hiroshima International University, 555-36, Kurose-Gakuendai, Higashi-Hiroshima, Hiroshima, 739-2695, Japan

    Noboru Iwata

  2. Department of Information Sciences, Toho University, 2-2-1 Miyama, Funabashi, Chiba, 274-8510, Japan

    Kenichi Kikuchi

  3. Department of Psychology, Yasuda Women’s University, 6-13-1, Yasuhigashi, Asaminami-ku, Hiroshima, Hiroshima, 731-0153, Japan

    Yuya Fujihara

Authors
  1. Noboru Iwata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenichi Kikuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuya Fujihara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noboru Iwata.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iwata, N., Kikuchi, K. & Fujihara, Y. The Usability of CAT System for Assessing the Depressive Level of Japanese—A Study on Psychometric Properties and Response Behavior. Int.J. Behav. Med. 23, 427–437 (2016). https://doi.org/10.1007/s12529-015-9503-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12529-015-9503-1

Keywords

Search

Navigation