Measuring the Outcomes of Intercultural Psychotherapy

  • Jessica CarlssonEmail author
  • Sabina Palic
  • Erik Vindbjerg


Outcome measurement in intercultural psychotherapy is complex and lags behind standards of outcome measurement in Western populations. This chapter presents recommendations, challenges, and pitfalls related to the choice of assessment tools measuring outcome in intercultural psychotherapy, whether for clinical evaluation or research. Item response theory (IRT) is presented as a promising method for testing measure’s and item’s intercultural validity and reliability. IRT-validated measures are presently mostly available for disability and quality of life. Finally, with a renewed focus on personal recovery in mental health, the chapter also introduces patient-generated outcome measures, focusing on empowerment, and documentation of patient perspectives, which are particularly important in intercultural treatment settings.


Validity Assessment Self-rating Observer rating Patient-generated outcome measures 


  1. 1.
    Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, Birz S. Developing instruments for cross-cultural psychiatric research. J Nerv Ment Dis. 1988;176(0022–3018):257–63.PubMedGoogle Scholar
  2. 2.
    Hollifield M. Measuring trauma and health status in refugees: a critical review. JAMA. 2002;288(5):611–21. Scholar
  3. 3.
  4. 4.
    Knaevelsrud C, Wagner B, Karl A, Mueller J. New treatment approaches: integrating new media in the treatment of war and torture victims. Torture. 2007;17(2):67–78. Scholar
  5. 5.
    Morina N, Ewers SM, Passardi S, Schnyder U, Knaevelsrud C, Müller J, et al. Mental health assessments in refugees and asylum seekers: evaluation of a tablet-assisted screening software. Confl Heal. 2017;11(1):1–9. Scholar
  6. 6.
    Jakobsen M, Meyer DeMott MA, Heir T. Validity of screening for psychiatric disorders in unaccompanied minor asylum seekers: use of computer-based assessment. Transcult Psychiatry. 2017;54(5–6):611–25. Scholar
  7. 7.
    Cuijpers P, Li J, Hofmann SG, Andersson G. Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: a meta-analysis. Clin Psychol Rev. 2010;30(6):768–78. Scholar
  8. 8.
    Mollica RF, Wyshak G, de Marnette D, Tu B, Yang T, Khuon F, et al. Hopkins Symptom Checklist 25 - manual, Cambodian, Laotian and Vietnamese versions. Torture. 1996;6(suppl 1):35–42.Google Scholar
  9. 9.
    Weathers F, Litz B, Herman D, Huska J, Keane T. The PTSD Checklist (PCL): reliability, validity, and diagnostic utility. In: Proceedings of the 9th annual meeting of the International Society for Traumatic Stress Studies (ISTSS); 1993.Google Scholar
  10. 10.
    Foa E, Cashman L, Jaycox L, Perry K. The validation of a self-report measure of PTSD: the posttraumatic diagnostic scale. Psychol Assess. 1997;9(9):445–51.CrossRefGoogle Scholar
  11. 11.
    Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J Trauma Stress. 2015;28:489–98.CrossRefGoogle Scholar
  12. 12.
    Foa EB, McLean CP, Zang Y, Zhong J, Powers MB, Kauffman BY, et al. Psychometric properties of the posttraumatic diagnostic scale for DSM-5 (PDS-5). Psychol Assess. 2015;28:1166. Scholar
  13. 13.
    Palic S, Zerach G, Shevlin M, Zeligman Z, Elklit A, Solomon Z. Evidence of complex posttraumatic stress disorder (CPTSD) across populations with prolonged trauma of varying interpersonal intensity and ages of exposure. Psychiatry Res. 2016;246:692–9. Scholar
  14. 14.
    Hyland P, Shevlin M, Brewin CR, Cloitre M, Downes AJ, Jumbe S, et al. Validation of post-traumatic stress disorder (PTSD) and complex PTSD using the International Trauma Questionnaire. Acta Psychiatr Scand. 2017;136(3):313–22. Scholar
  15. 15.
    Kinzie JD, Manson SM, Vinh DT, Tolan NT, Anh B, Pho TN. Development and validation of a Vietnamese-language depression rating scale. Am J Psychiatr. 1982;139(10):1276–81. Scholar
  16. 16.
    Mollica RF, Wyshak G, De Marneffe D, Khuon F, Lavelle J, Marneffe D, et al. Indochinese versions of the Hopkins Symptom Checklist-25: a screening instrument for the psychiatric care of refugees. Am J Psychiatry. 1987;144(4):497–500. Scholar
  17. 17.
    Mollica RF, Caspi-Yavin Y, Lavelle J, Tor S, Yang T, Chan S, et al. Harvard Trauma Questionnaire (HTQ) - manual, Cambodian, Laotian and Vietnamese versions. Torture. 1996;Suppl 1:22–33.Google Scholar
  18. 18.
    Harvard Program in Refugee Trauma. n.d. Retrieved 12 Dec 2017.
  19. 19.
    Weiss DS, Marmar CR. The impact of event scale - revised. In: Wilson J, Keane TM, editors. Assessing psychological trauma and PTSD. New York: Guilford; 1996. p. 399–411.Google Scholar
  20. 20.
    Weathers FW, Blake DD, Schnurr PP, Kaloupek DG, Marx BP, Keane TM. The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5): The National Center for PTSD; 2013.
  21. 21.
    Foa EB, Riggs DS, Dancu CV, Rothbaum BO. Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. J Trauma Stress. 1993;6(4):459–73.CrossRefGoogle Scholar
  22. 22.
    Mollica RF, McInnes K, Sarajlic N, Lavelle J, Sarajlic I, Massagli MP, et al. Comorbidity and health status in Bosnian refugees living in Croatia. JAMA. 1999;281(5):433–9. CrossRefGoogle Scholar
  23. 23.
    Neuner F, Schauer M, Klaschik C, Karunakara U, Elbert T. A comparison of narrative exposure therapy, supportive counseling, and psychoeducation for treating posttraumatic stress disorder in an African refugee settlement. J Consult Clin Psychol. 2004;72(4):579–87. Scholar
  24. 24.
    Roberts NP, Cloitre M, Bisson J, Brewin C. The International Trauma Interview (ITI). Unpublished Measure; 2018.Google Scholar
  25. 25.
    Gray MJ, Litz BT, Hsu JL, Lombardo TW. Psychometric properties of the life events checklist. Assessment. 2004;11(4):330–41. Scholar
  26. 26.
    McNally RJ. The science and folklore of traumatic amnesia. Clin Psychol. 2004;11(1):29–33.Google Scholar
  27. 27.
    Pope HG, Hudson JI, Bodkin JA, Oliva P. Questionable validity of ‘dissociative amnesia’ in trauma victims. Br J Psychiatry. 1998;172(3):210–5.CrossRefGoogle Scholar
  28. 28.
    Fried EI, Eidhof MB, Palic S, Costantini G, Dijk HM H-v, Bockting CLH, Engelhard I, Armour C, Nielsen ABS, Karstoft K-I. Replicability and generalizability of posttraumatic stress disorder (PTSD) Networks: A cross-cultural multisite study of PTSD symptoms in four trauma patient samples. Clin Psychol Sci. 2017;6(3):335–51.CrossRefGoogle Scholar
  29. 29.
    Maller RG, Reiss S. Anxiety sensitivity in 1984 and panic attacks in 1987. J Anxiety Disord. 1992;6(3):241–7. Scholar
  30. 30.
    Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the state-trait anxiety inventory. Palo Alto: Consulting Psychologists Press; 1983.Google Scholar
  31. 31.
    Cox BJ, Parker JD, Swinson RP. Anxiety sensitivity: confirmatory evidence for a multidimensional construct. Behav Res Ther. 1996;34(7):591–8. Scholar
  32. 32.
    Donat DC. Predicting state anxiety: a comparison of multidimensional and unidimensional trait approaches. J Res Pers. 1983;17(2):256–62. Scholar
  33. 33.
    Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio: Psychological Corporation; 1996.Google Scholar
  34. 34.
    Bagby RM, Ryder AG, Schuller DR, Marshall MB. Reviews and overviews the Hamilton depression rating scale: has the gold standard become a lead weight? Am J Psychiatry. 2004;161(12):2163.CrossRefGoogle Scholar
  35. 35.
    Bech P, Csillag C, Hellström L, Fleck MPDA. The time has come to stop rotations for the identification of structures in the Hamilton Depression Scale (HAM-D17). Rev Bras Psiquiatr. 2013;35(4):360–3. Scholar
  36. 36.
    Mcintyre R, Kennedy S, Bagby RM, Bakish D. Assessing full remission. J Psychiatry Neurosci. 2002;27(4):235–9.PubMedPubMedCentralGoogle Scholar
  37. 37.
    Bech P, Allerup P, Maier W, Albus M, Lavori P, Ayuso JL. The Hamilton scales and the Hopkins Symptom Checklist (SCL-90). A cross-national validity study in patients with panic disorders. Br J Psychiatry. 1992;160(2):206–11. Scholar
  38. 38.
    Regier DA, Narrow WE, Kuhl EA, Kupfer DJ. The conceptual evolution of DSM-5. Arlington: American Psychiatric Publishing; 2011.Google Scholar
  39. 39.
    Ustün TB, Kostanjsek N, Chatterji S, Rehm J. Measuring health and disability: manual for WHO Disability Assessment Schedule (WHODAS 2.0). Geneva: World Health Organization; 2010.Google Scholar
  40. 40.
    APA. Diagnostic and statistical manual of mental disorders: DSM-IV-TR. 4th ed. Washington: American Psychiatric Association; 2000.Google Scholar
  41. 41.
    Aas IH. Global Assessment of Functioning (GAF): properties and frontier of current knowledge. Ann Gen Psychiatry. 2010;9(1):20.CrossRefGoogle Scholar
  42. 42.
    Wing J, Curtis R, Beevor A. Health of nation outcome scales. Report on research and development. London: College Research Unit, Royal College of Psychiatrists; 1996.Google Scholar
  43. 43.
    Palic S, Kappel ML, Makransky G. Rasch validation and cross-validation of the health of nation outcome scales for monitoring of psychiatric disability in traumatized refugees in Western psychiatric care. Assessment. 2016;23(6):734–43. Scholar
  44. 44.
    Palic S, Kappel LL, Nielsen SS, Carlsson J, Bech P. Comparison of psychiatric disability on the health of nation outcome scales (HoNOS) in resettled traumatized refugee outpatients and Danish inpatients. BMC Psychiatry. 2014;14(1):1–10. Scholar
  45. 45.
    Dronavalli M, Thompson SC. A systematic review of measurement tools of health and well-being for evaluating community-based interventions. J Epidemiol Community Health. 2015;69(8):805–15. Scholar
  46. 46.
    Burckhardt C, Anderson K. The Quality of Life Scale (QOLS): reliability, validity, and utilization. Health Qual Life Outcomes. 2003;1(60):1–7. Scholar
  47. 47.
    International Wellbeing Group. Personal Wellbeing Index. 5th ed. Melbourne: Australian Centre on Quality of Life, Deakin University; 2013. Scholar
  48. 48.
    Skevington SM, Lotfy M, O’Connell KA, WHOQOL Group. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL Group. Qual Life Res. 2004;13(2):299–310. Scholar
  49. 49.
    Martín-Díaz F, Reig-Ferrer A, Ferrer-Cascales R. (2006). Assessment of health-related quality of life in chronic dialysis patients with the COOP/WONCA charts. Nephron Clin Pract. 2006;104(1):c7–14. Scholar
  50. 50.
    Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167–76. Scholar
  51. 51.
    Brodaty H, Donkin M. Family caregivers of people with dementia. Dialogues Clin Neurosci. 2009;11(2):217–28. Scholar
  52. 52.
    Kong CL, Lee CC, Ip YC, Chow LP, Leung CH, Lam YC. Validation of the Hong Kong Cantonese version of World Health Organization five Well-Being Index for people with severe mental illness. East Asian Arch Psychiatry. 2016;26(1):18–21.PubMedGoogle Scholar
  53. 53.
    Roy T, Lloyd CE, Parvin M, Mohiuddin KGB, Rahman M. Prevalence of co-morbid depression in out-patients with type 2 diabetes mellitus in Bangladesh. BMC Psychiatry. 2012;12
  54. 54.
    Sales CMD, Alves PCG. Patient-centered assessment in psychotherapy: a review of individualized tools. Clin Psychol Sci Pract. 2016;23(3):265–83. Scholar
  55. 55.
    Elliott R, Mack C, Shapiro D. Simplified Personal Questionnaire procedure (unpublished manuscript); 1999.
  56. 56.
    Ashworth M, Shepherd M, Christey J, Matthews V, Wright K, Parmentier H, et al. A client-generated psychometric instrument: the development of “PSYCHLOPS”. Couns Psychother Res. 2004;4(2):27–31. Scholar
  57. 57.
    Kiresuk TJ, Sherman RE. Goal attainment scaling: a general method for evaluating comprehensive community mental health programs. Community Ment Health J. 1968;4(6):443–53. Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Jessica Carlsson
    • 1
    Email author
  • Sabina Palic
    • 1
  • Erik Vindbjerg
    • 1
  1. 1.Competence Centre for Transcultural Psychiatry, Mental Health Centre BallerupBallerupDenmark

Personalised recommendations