Abstract
Background
An innovative measurement system using a computerized adaptive testing technique based on the item response theory (CAT) has been expanding to measure mental health status. However, little is known about details in its measurement properties based on the empirical data. Moreover, the response time (RT) data, which are not available by a paper-and-pencil measurement but available by a computerized measurement, would be worth investigating for exploring the response behavior.
Purpose
We aimed at constructing the CAT to measure depressive symptomatology in a community population and exploring its measurement properties. Also, we examined the relationships between RTs, individual item responses, and depressive levels.
Method
For constructing the CAT system, responses of 2061 workers and university students to 24 depression scale plus four negatively revised positive affect items were subjected to a polytomous IRT analysis. The stopping rule was set for standard error of estimation < 0.30 or the maximum 15 items displayed. The CAT and non-adaptive computer-based test (CBT) were administered to 209 undergraduates, and 168 of them administered again after 1 week.
Results
On average, the CAT was converged by 10.4 items. The θ values estimated by CAT and CBT were highly correlated (r = 0.94 and 0.95 for the 1st and 2nd measurements) and with the traditional scoring procedures (r’s > 0.90). The test–retest reliability was at a satisfactory level (r = 0.86). RTs to some items significantly correlated with the θ estimates. The mean RT varied by the item contents and wording, i.e., the RT to positive affect items required additional 2 s or longer than the other subscale items.
Conclusion
The CAT would be a reliable and practical measurement tool for various purposes including stress check at workplace.
Similar content being viewed by others
References
Ferrari AJ, Charlson FJ, Norman RE, et al. Burden of depressive disorders by country, sex, age, and year: findings from the Global Burden of Disease Study 2010. PLoS Med. 2013;10, e1001547. doi:10.1371/journal.pmed.1001547.
Gibbons RD, Weiss DJ, Pilkonis PA, et al. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiat. 2014;171:187–94.
Forkmann T, Boecker M, Wirtz M, et al. Validation of the Rasch-based depression screening in a large scale German general population sample. HQLO. 2010;8:105. http://www.hqlo.com/content/8/1/105
Lord FM, Novick MR. Statistical theories of mental test scores. Reading: Addison-Welsley Publishing Company; 1968.
Lord FM. Applications of item response theory to practical testing problems. Erlbaum Associates; 1980
Hambleton RK, Cook LL. Latent trait models and their use in the analysis of educational test data. J Educ Meas. 1977;14:75–96.
Hambleton RK, Swaminathan H. Item response theory: principles and applications. Boston: Kluwer; 1985.
Embretson SE, Reise SP. Item response theory for psychologists. Mahwah: Lawrence Erlbaum Associates, Inc., Publishers; 2000.
Wainer H, Dorans NJ, Eignor D, et al. Computerized adaptive testing: a primer. 2nd ed. Hillsdale: Erlbaum Associates; 2000.
Fliege H, Becker J, Walter OB, et al. Evaluation of a computer-adaptive test for the assessment of depression (D-CAT) in clinical application. Int J Methods Psychiatr Res. 2009;18:23–36.
Weiss DJ. Adaptive testing by computer. J Consul Clin Psychol. 1985;53:774–89.
Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38 Suppl 9:73–82.
Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002;40:812–23.
Fliege H, Becker J, Walter OB, et al. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14:2277–91.
Smits N, Cuijpers P, van Straten A. Applying computerized adaptive testing to the CES-D scale: a simulation study. Psychiat Res. 2011;188:147–55.
American Psychological Association. Publication manual of the American Psychological Association, 5th ed. Washington, DC: Author; 2005.
Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psych Meas. 1977;1:385–401.
Shima S, Shikano T, Kitamura T, et al. New self-rating scale for depression. Clinical Psychiatry. 1985;27:717–23 [In Japanese].
Iwata N, Umesue M, Egashira K, et al. Can positive affect items be used to assess depressive disorders in the Japanese population? Psychol Med. 1998;28:153–8.
Iwata N, Okuyama Y, Kawakami Y, et al. Prevalence of depressive symptoms in a Japanese occupational setting: a preliminary study. Am J Public Health. 1989;79:1486–9.
Iwata N, Saito K, Roberts RE. Responses to a self-administered depression scale among younger adolescents in Japan. Psychiat Res. 1994;53:275–87.
Iwata N, Roberts CR, Kawakami N. Japan-U.S. comparison of responses to depression scale items among adult workers. Psychiat Res. 1995;58:237–45.
Iwata N, Buka S. Race/ethnicity and depressive symptoms: a cross-cultural/ethnic comparison among university students in East Asia, North and South America. Soc Sci Med. 2002;55:2243–52.
Owen RJ. A bayesian sequential procedure for quantal response in the context of adaptive mental testing. J Am Stat Assoc. 1975;70:351–6.
Muthén LK, Muthén BO. Mplus user’s guide. Los Angeles, CA: Muthén & Muthén; 2006.
Samejima, F. Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph. 17, 1969.
Muraki E, Bock RD. PARSCALE: IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software Int; 1993.
Costello EJ, Angold A. Scales to assess child and adolescent depression: checklists, screens, and nets. J Am Acad Child Psy. 1988;27:726–37.
Roberts RE, Lewinsohn PM, Seeley JR. Screening for adolescent depression: a comparison of depression scales. J Am Acad Child Psy. 1991;30:58–66.
Iwata N. Cultural distinctiveness in response bias. In: Dollard MF, Shimazu A, Nordin RB, Brough P, Tuckey MR, editors. Psychosocial factors at work in the Asia Pacific. New York: Springer; 2014. p. 55–88.
Walter OB, Becker J, Bjorner JB, et al. Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Qual Life Res. 2007;16(Suppl1):143–55.
Japan Industrial Safety and Health Law. Article 66–10 amended [Act No. 82 of June 25, 2014]
Acknowledgments
This study was partly supported by Grand-in-Aid for Scientific Research (C), from the Japan Ministry of Education, Culture, Sports, Science and Technology (project number 18590620) and Health and Labour Sciences Research Grants (Research on Occupational Safety and Health; H17-Rodo-Ippan-005, principle investigator: Norito Kawakami), from the Japan Ministry of Health, Labour and Welfare.
Ethical Standards
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. Informed consent was verbally obtained from all patients for being included in the study.
Conflict of Interest
Noboru Iwata, Kenichi Kikuchi, and Yuya Fujihara declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Iwata, N., Kikuchi, K. & Fujihara, Y. The Usability of CAT System for Assessing the Depressive Level of Japanese—A Study on Psychometric Properties and Response Behavior. Int.J. Behav. Med. 23, 427–437 (2016). https://doi.org/10.1007/s12529-015-9503-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12529-015-9503-1