Skip to main content

Advertisement

SpringerLink
Log in

Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT)

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Within the framework of item response theory (IRT), we developed a German version of an item bank, as well as a software application that can be employed to measure anxiety by means of a computer adaptive test (CAT). A sample of n = 2348 psychiatric and psychosomatic patients answered a set of up to 13 standardized questionnaires. 81 items drawn from these questionnaires were considered pertinent to the anxiety construct. Various tests were conducted to ensure the suitability of these items for an IRT-based assessment. After these tests, 50 items remained in the item bank and were calibrated using the Generalized Partial Credit Model. Simulation studies conducted on an independent sample of n = 1528 respondents indicate that 6–8 items suffice to measure the latent trait with high precision (standard error ≤ 0.32). CAT scores correlated highly with scores estimated from all available items (r = .97) and scale scores of the State Trait Anxiety Inventory (STAI, state scale, r = .93). Within a routine clinical setting, 102 in-patients answered the Anxiety-CAT along with a number of established anxiety questionnaires. The correlation between the Anxiety-CAT and the STAI state scale was still high (r = .60), but lower than the correlations found in the simulation studies. The Anxiety-CAT was able to differentiate between mental health disorders in a similar manner as established questionnaires. These results suggest that the Anxiety-CAT does indeed exhibit the advantages expected from theory, but the results of further studies are needed in order to judge its full potential for research and clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Center of Epidemiological Studies—Depression Scale/Allgemeine Depressions-Skala [13].

  2. Beck-Depression-Inventory [14].

  3. Daily life Questionnaire/Fragebogen zum Alltagsleben [15].

  4. Berlin Mood Questionnaire/Berliner Stimmungs-Fragebogen [16].

  5. Gießen Complaint Questionnaire/Gießener Beschwerdebogen [17].

  6. Gießen-Test [18].

  7. Narcissism-Inventory/Narzissmusinventar [19, 20].

  8. Psychological General Well-Being Index [21].

  9. Perceived Stress Questionnaire [22].

  10. Short form-36 [23].

  11. Subjective Theories about the Development of Illness/Subjektive Krankheitstheorie-Ursachenvorstellung [24].

  12. State Trait Anxiety Inventory [25].

  13. Self Efficacy and Optimism Inventory/Fragebogen zur Selbstwirksamkeit & Optimismus [26].

References

  1. Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8, 341–349.

    Article  Google Scholar 

  2. Meijer, R. R. & Nering, M. L. (1999). Computerized adaptive testing. Overview and introduction. Applied Psychological Measurement, 23, 187–194.

    Article  Google Scholar 

  3. Fries, J. F., Bruce, B. & Cella, D. (2005). The promise of PROMIS: Using item response theory to improve assessment of patient-reported outcomes. Clinical and Experimental Rheumatology, 23, 33–37.

    Google Scholar 

  4. Cella, D. F. & Chang, C. H. (2000). A discussion of item response theory and its applications in health status assessment. Medical Care, 38, 66–72.

    Article  Google Scholar 

  5. Handel, R., Ben-Porath, Y. & Watt, M. (1999). Computerized adaptive assessment with the MMPI-2 in a clinical setting. Psychological Assessment, 11, 369–380.

    Article  Google Scholar 

  6. Ware, J. E., Kosinski, M., Bjorner, J. B., Bayliss, M. S., Batenhorst, A., Dahlöf, C. G. H., Tepper, S. & Dowson. A. (2003). Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Quality of Life Research, 12, 935–952.

    Article  PubMed  Google Scholar 

  7. Fliege, H., Becker, J., Walter, O. B., Bjorner, J. B., Klapp, B. F. & Rose, M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14, 2277–2291.

    Article  PubMed  Google Scholar 

  8. Rose, M., Bjorner, J. B., Becker, J., Fries, J. F. & Ware, J. E. (2007). Evaluation of a preliminary physical function item bank supports the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology (in press).

  9. Walter, O. B., Becker, J., Fliege, H., Bjorner, J. B., Kosinski, M., Walter, M., Klapp, B. F. & Rose, M. (2005). Entwicklungsschritte für einen computeradaptiven Test zur Erfassung von Angst (A-CAT). Diagnostica, 51, 88–100.

    Article  Google Scholar 

  10. Häcker, H. & Stapf, K. H. (1998). Dorsch Psychologisches Wörterbuch. Bern: Huber.

    Google Scholar 

  11. Spielberger, C. D. (1972). Anxiety. Current trends in theory and research. Oxford: Academic Press.

    Google Scholar 

  12. Dilling, H., Mombour, W. & Schmidt, M. H. (1999). Internationale Klassifikation psychischer Störungen. ICD-10 Kapitel V (F). Klinisch-diagnostische Leitlinien (3rd ed.). Bern: Huber.

    Google Scholar 

  13. Hautzinger, M. & Bailer, M. (1993). ADS. Allgemeine Depressionsskala. Testmappe mit Handanweisung. Weinheim: Beltz.

    Google Scholar 

  14. Hautzinger, M., Bailer, M., Worall, H. & Keller, F. (1994). BDI. Beck-Depressions-Inventar. ( A. T. Beck, C. H. Ward, M. Mendelson, J. Mock & J. Erbaugh, 1961). Testmappe mit Manual. Bern: Huber.

  15. Bullinger, M., Kirchberger, I. & von Steinbuechel, N. (1993). Der Fragebogen Alltagsleben – ein Verfahren zur Erfassung der gesundheitsbezogenen Lebensqualität. Zeitschrift für Medizinische Psychologie: ZMP, 2, 121–131.

    Google Scholar 

  16. Hörhold, M. & Klapp, B. F. (1993). Testungen der Invarianz und der Hierarchie eines mehrdimensionalen Stimmungsmodells auf der Basis von Zweipunkterhebungen an Patienten- und Studentenstichproben. Zeitschrift für Medizinische Psychologie: ZMP, 2, 27–35.

    Google Scholar 

  17. Brähler, E. & Scheer, J. W. (1995). GBB. Gießener Beschwerdebogen. Testmappe mit 2., ergänzter und revidierter Auflage des Handbuchs. Bern: Huber.

    Google Scholar 

  18. Beckmann, D., Brähler, E. & Richter, H. E. (1991). Der Gießen-Test (GT). Ein Test für Individual- und Guppendiagnostik. Bern: Huber.

    Google Scholar 

  19. Deneke, F. W. & Hilgenstock, B. (1989). NI. Das Narzissmusinventar. Testmappe mit Handanweisung. Bern: Huber.

    Google Scholar 

  20. Schoeneich, F., Rose, M., Danzer, G., Thier, P., Weber, C. & Klapp, B. F. (2000). Narzissmusinventar-90 (NI-90). Empiriegeleitete Itemreduktion und Identifikation veränderungssensitiver Items des Narzissmusinventars zur Messung selbstregulativer Parameter. Psychotherapie, Psychosomatik, Medizinische Psychologie, 50, 396–405.

    Article  PubMed  CAS  Google Scholar 

  21. Ludwig, M., Geier, S. & Bullinger, M. (1990). Skalen zur Erfassung des Wohlbefindens: Psychometrisches Analysen zum “Profile of Mood States” (POMS) und zum “Psychological General Well-Being Index” (PGWI). Zeitschrift für Differentielle und Diagnostische Psychologie, 11, 53–61.

    Google Scholar 

  22. Levenstein, S., Prantera, C., Varvo, V. & Scribano, M. L. (1993). Development of the perceived stress questionnaire: A new tool for psychosomatic research. Journal of Psychosomatic Research, 37, 19–32.

    Article  PubMed  CAS  Google Scholar 

  23. Bullinger, M. & Kirchberger, I. (1998). SF-36. Fragebogen zum Gesundheitszustand. Testmappe mit Handanweisung. Göttingen: Hogrefe.

    Google Scholar 

  24. Faller, H. (1997). Subjektive Krankheitstheorien bei Patienten einer psychotherapeutischen Ambulanz. Zeitschrift für Klinische Psychologie und Psychotherapie, 45, 264–278.

    Google Scholar 

  25. Laux, L., Glanzmann, P., Schaffner, P. & Spielberger, C. D. (1981). STAI. State-Trait Anxiety Inventory. ( C. D. Spielberger, R. L. Gorsuch, R. E. Lushene 1970). Testmappe mit Handanweisung. Weinheim: Beltz.

  26. Scholler, G., Fliege, H. & Klapp, B. F. (1999). Fragebogen zu Selbstwirksamkeit, Optimismus und Pessimismus. Restrukturierung, Itemselektion und Validierung eines Instrumentes an Untersuchungen klinischer Stichproben. Psychotherapie, Psy-cho-so-ma-tik, Medizinische Psychologie, 49, 275–283.

    CAS  Google Scholar 

  27. Rose, M., Hess, V., Hoerhold, M., Brähler, E. & Klapp, B. F. (1999). Mobile computergestützte psychometrische Diagnostik. Ökonomische Vorteile und Ergebnisse zur Teststabilität. Psychotherapie, Psychosomatik, Medizinische Psychologie, 49, 202–207.

    PubMed  CAS  Google Scholar 

  28. Rose, M., Walter, O. B., Fliege, H., Becker, J., Hess, V. & Klapp, B. F. (2002). 7 years of experience using Personal Digital Assistants (PDA) for psychometric diagnostics in 6000 inpatients and polyclinic patients. In H. B. Bludau & A. Koop (Eds.), Mobile computing in medicine. Lecture notes in informatics (pp. 35–44). Bonn: Köllen.

    Chapter  Google Scholar 

  29. Bjorner, J., Kosinski, M. & Ware, J. E. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the Headache Impact Test (HIT-superTM). Quality of Life Research, 12, 913–933.

    Article  PubMed  Google Scholar 

  30. Ware, J. E., Bjorner, J. B. & Kosinski, M. (2000). Practical implications of item response theory and computerized adaptive testing. A brief summary of ongoing studies of widely used headache impact scales. Medical Care, 38, II73–II82.

    Article  PubMed  Google Scholar 

  31. Bock, R. D. & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431–444.

    Article  Google Scholar 

  32. Wang, S. (1999). The accuracy of ability estimation methods for computerized adaptive testing using the generalized partial credit model. PA: Unpublished doctoral dissertation, University of Pittsburgh.

  33. Gardner, W., Kelleher, K. J. & Pajer, K. A. (2002). Multidimensional adaptive testing for mental health problems in primary care. Medical Care, 40, 812–823.

    Article  PubMed  Google Scholar 

  34. Wittchen, H. U., Kessler, R. C., Zhao, S. & Abelson, J. (1995). Reliability and clinical validity of UM-CIDI DSM-III-R generalized anxiety disorder. Journal of Psychiatric Research, 29, 95–110.

    Article  PubMed  CAS  Google Scholar 

  35. Wittchen, H. U., Pfister, H. & Garctynski, E. (1998). Composite International Diagnostic Interview (CIDI). Göttingen: Hogrefe.

    Google Scholar 

  36. Muthén, L. K. & Muthén, B. O. (2004). Mplus. The comprehensive modeling program for applied researchers. Users guide. Los Angeles: Muthén & Muthén.

    Google Scholar 

  37. Hu, L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  38. MacCallum, R., Browne, M. & Sugawara, H. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149.

    Article  Google Scholar 

  39. Ramsay, J. O. (1995). TestGraf. A program for the graphical analysis of multiple choice test and questionnaire data. Montreal: McGill University.

    Google Scholar 

  40. Holland, P. W. & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  41. Swaminathan, H. & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.

    Article  Google Scholar 

  42. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (Ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.

  43. Nagelkerke, N. J. D. (1991). A note on a general definition of the coefficient of determination. Biometrika, 78, 691–692.

    Article  Google Scholar 

  44. Bjorner, J., Kosinski, M. & Ware, J. E. (2003). The feasibility of applying item response theory to measures of migraine impact: A re-analysis of three clinical studies. Quality of Life Research, 12, 887–902.

    Article  PubMed  Google Scholar 

  45. Jodoin, M. & Gierl, M. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329–349.

    Article  Google Scholar 

  46. Muraki, E. (1992). A Generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.

    Article  Google Scholar 

  47. Muraki, E. & Bock, R. D. (1999). PARSCALE: Analysis of graded responses and ratings. Chicago, IL: Scientific Software Int., Inc.

    Google Scholar 

  48. Mineka, S., Watson, D. & Clark, L. A. (1998). Comorbidity of anxiety and unipolar mood disorders. Annual Review of Psychology, 49, 377–412.

    Article  PubMed  CAS  Google Scholar 

  49. Zinbarg, R. & Barlow, D. (1996). Structure of anxiety and the anxiety disorders: A hierarchical model. Journal of Abnormal Psychology, 105, 181–193.

    Article  PubMed  CAS  Google Scholar 

  50. Brown, T., Chorpita, B. & Barlow, D. (1998). Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology, 107, 179–192.

    Article  PubMed  CAS  Google Scholar 

  51. Chorpita, B., Albano, A. & Barlow, D. (1998). The structure of negative emotions in a clinical sample of children and adolescents. Journal of Abnormal Psychology, 107, 74–85.

    Article  PubMed  CAS  Google Scholar 

  52. Joiner, T., Catanzaro, S. & Laurent, J. (2004). Tripartite structure of positive and negative affect, depression, and anxiety in child and adolescent psychiatric inpatiens. Journal of Abnormal Psychology, 105, 401–409.

    Article  Google Scholar 

  53. Kessler, R. C., Nelson, C. B., McGonagle, K. A. & Liu, J. (1996). Comorbidity of DSM-III-R major depressive disorder in the general population: results from the US National Comorbidity Survey. The British Journal of Psychiatry. Supplement, 168, 17–30.

    Google Scholar 

  54. Hornke, L. F. (1999). Benefits from computerized adaptive testing as seen in simulation studies. European Journal of Psychological Assessment, 15, 91–98.

    Article  Google Scholar 

  55. Wainer, H., Dorans, N. J., Green, B. F., Steinberg, L., Flaugher, R., Mislevy, R. J. & Thissen, D. (1990). Computerized adaptive testing: A primer. Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  56. Cooke, D. J., Michie, C., Hart, S. D. & Hare, R. D. (1999). Evaluating the screening version of the Hare psychopathy checklist – Revised (PCL:SV): An item response theory analysis. Psychological Assessment, 11, 3–13.

    Article  Google Scholar 

  57. Hambleton, R. K., Swaminathan, H. & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publications Inc.

    Google Scholar 

  58. McDonald, R. P. (1989). Future directions for item response theory. International Journal of Educational Research, 13, 205–222.

    Article  Google Scholar 

  59. Dahlstrom, W. G., Brooks, J. D. & Peterson, C. D. (1990). The Beck depression inventory: Item order and the impact of response sets. Journal of Personality Assessment, 55, 224–233.

    Article  PubMed  CAS  Google Scholar 

  60. Walter, O. B., Fliege, H. & Rose, M. (2005). Effect of item order on item calibration and item bank construction for computer adaptive tests. Quality of Life Research, 14, 2013.

    Google Scholar 

Download references

Acknowledgements

The study was supported by the German Research Society (Deutsche Forschungsgemeinschaft RO 2258/2-1)

Author information

Authors and Affiliations

  1. Psychological Institute IV, Statistics and Quantitative Methods, University of Münster, Fliednerstr. 21, 48149, Münster, Germany

    Otto B. Walter

  2. Health Assessment Lab, Waltham, MA, USA

    Janine Becker, Jakob B. Bjorner & Matthias Rose

  3. QualityMetric Inc., Lincoln, RI, USA

    Janine Becker, Jakob B. Bjorner & Matthias Rose

  4. Department of Psychosomatic Medicine and Psychotherapy, Clinic for Internal Medicine, Charité, University Medicine Berlin, Charité, Berlin, Germany

    Janine Becker, Herbert Fliege & Burghard F. Klapp

Authors
  1. Otto B. Walter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janine Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jakob B. Bjorner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Herbert Fliege
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Burghard F. Klapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthias Rose
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Otto B. Walter.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Walter, O.B., Becker, J., Bjorner, J.B. et al. Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Qual Life Res 16 (Suppl 1), 143–155 (2007). https://doi.org/10.1007/s11136-007-9191-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-007-9191-7

Keywords

Search

Navigation