Psychological Injury and Law

, Volume 7, Issue 2, pp 153–164 | Cite as

The Importance of Establishing Reliability and Validity of Assessment Instruments for Mental Health Problems: an Example from Somali Children and Adolescents Living in Three Refugee Camps in Ethiopia

  • Brian J. HallEmail author
  • Eve Puffer
  • Laura K. Murray
  • Abdulkadir Ismael
  • Judith K. Bass
  • Amanda Sim
  • Paul A. Bolton


Assessing mental health problems cross-culturally for children exposed to war and violence presents a number of unique challenges. One of the most important issues is the lack of validated symptom measures to assess these problems. The present study sought to evaluate the psychometric properties of two measures to assess mental health problems: the Achenbach Youth Self-Report and the Child Posttraumatic Stress Disorder Symptom Scale. We conducted a validity study in three refugee camps in Eastern Ethiopia in the outskirts of Jijiga, the capital of the Somali region. A total of 147 child and caregiver pairs were assessed, and scores obtained were submitted to rigorous psychometric evaluation. Excellent internal consistency reliability was obtained for symptom measures for children and their caregivers. Validation of study instruments based on local case definitions was obtained for the caregivers but not consistently for the children. Sensitivity and specificity of study measures were generally low, indicating that these scales would not perform adequately as screening instruments. Combined test–retest and inter-rater reliability was low for all scales. This study illustrates the need for validation and testing of existing measures cross-culturally. Methodological implications for future cross-cultural research studies in low- and middle-income countries are discussed.


PTSD Assessment Reliability Validity Children Adolescents Refugees 



The present research was made possible by a grant from the Bill and Melinda Gates Foundation: Changing Lives: A Learning Initiative to End Violence Against Women and Children in Emergencies. Dr. Hall was supported by the National Institute of Mental Health T32 in Psychiatric Epidemiology T32MH014592-35 and through the Fogarty Global Health Fellows Program Consortium comprised of the University of North Carolina and John Hopkins Bloomberg School of Public Health, Morehouse and Tulane (1R25TW009340-01). We thank the dedicated study interviewers, Sarah Katherine Baird, IRC Community Well-Being Initiative Manager, Aden Abdi Hiss, IRC Transportation Officer, Asyia Abdulahi Elabe, IRC Caring for Child Survivors Officer, Tensay Tefera IRC CYPD-Child, Youth and Protection Department Manager, Shewaye, IRC CYPD Coordinator, Ayalew, IRC Child Protection Manager, Anjuli Shivshanker, IRC Research and Evaluation Officer, for her assistance with data collection, and the children and caregivers who participated in this study.


  1. Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychological Bulletin, 10, 213–232.Google Scholar
  2. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms & profiles. VT, Burlington: University of Vermont, Research.Google Scholar
  3. Barenbaum, J., Ruchkin, V., & Schwab-Stone, M. (2004). The psychosocial aspects of children exposed to war: practice and policy initiatives. Journal of Child Psychology and Psychiatry, 45(1), 41–62. doi: 10.1046/j.0021-9630.2003.00304.x PubMedCrossRefGoogle Scholar
  4. Bass, J. K., Ayash, C., Betancourt, T. S., Haroz, E. E., Verdeli, H., Neugebauer, R., et al. (2013). Mental health problems of displaced war-affected adolescents in northern Uganda: patterns of agreement between self and caregiver assessment. Journal of Child and Family Studies. doi: 10.1007/s10826-013-9755-9 Google Scholar
  5. Bass, J. K., Ryder, R. W., Lammers, M. C., Mukaba, T. N., & Bolton, P. (2008). Post-partum depression in Kinshasa, Democratic Republic of Congo: validation of a concept using a mixed-methods cross-cultural approach. Tropical Medicine International Health, 13(12), 1534–1542. doi: 10.1111/j.1365-3156.2008.02160.x PubMedCrossRefGoogle Scholar
  6. Betancourt, T. S., Bass, J., Borisova, I., Neugebauer, R., Speelman, L., Onyango, G., & Bolton, P. A. (2009). Assessing local instrument reliability and validity: a field-based example from northern Uganda. Social Psychiatry and Psychiatric Epidemiology, 44(8), 685–692. doi: 10.1007/s00127-008-0475-1
  7. Bolton, P. (2001). Cross-cultural validity and reliability testing of a standard psychiatric assessment instrument without a gold standard. Journal of nervous and mental disease, 189(4), 238–242. doi: 10.1097/00005053-200104000-00005 PubMedCrossRefGoogle Scholar
  8. Bolton, P., Michalopoulos, L., Ahmed, M. A., Murray, L. K., & Bass, J. (2013). The mental health and psychosocial problems of survivors of torture and genocide in Kurdistan, Northern Iraq: a brief qualitative study. Torture, 23(1), 1–14.PubMedGoogle Scholar
  9. Bolton, P., Wilk, C. M., & Ndogoni, L. (2004). Assessment of depression prevalence in rural Uganda using symptom and function criteria. Social Psychiatry and Psychiatric Epidemiology, 39(6), 442–447. doi: 10.1007/s00127-004-0763-3 PubMedCrossRefGoogle Scholar
  10. Federal Democratic Republic of Ethiopa Ministry of Health. (2012). National Mental Health Strategy (2012/13 - 2015/6).Google Scholar
  11. Foa, E. B., Johnson, K. M., Feeny, N. C., & Treadwell, K. R. H. (2001). The Child PTSD Symptom Scale: a preliminary examination of its psychometric properties. Journal of Clinical Child Psychology, 30(3), 376–384. doi: 10.1207/S15374424JCCP3003_9 PubMedCrossRefGoogle Scholar
  12. Hobfoll, S. E., Canetti, D., Hall, B. J., Brom, D., Palmieri, P. A., Johnson, R. J., et al. (2011). Are community studies of psychological trauma's impact accurate? A study among Jews and Palestinians. [Research Support, N.I.H., Extramural]. Psychological Assessment, 23(3), 599–605. doi: 10.1037/a0022817 PubMedCentralPubMedCrossRefGoogle Scholar
  13. Hollifield, M., Warner, T. D., Lian, N., Krakow, B., Jenkins, J. H., Kesler, J., et al. (2002). Measuring trauma and health status in refugees: a critical review. JAMA: Journal of the American Medical Association, 288(5), 611–621. doi: 10.1001/jama.288.5.611 PubMedCrossRefGoogle Scholar
  14. The International Rescue Committee & UNHCR (2009). Assessment report on the situation of children in Kebribeyeh and Sheder refugee camps.Google Scholar
  15. Kleinman, A. (1988). Rethinking psychiatry: from cultural category to personal experience. New York City: Free Press.Google Scholar
  16. Kohrt, B. A., & Hruschka, D. J. (2010). Nepali concepts of psychological trauma: the role of idioms of distress, ethnopsychology and ethnophysiology in alleviating suffering and preventing stigma. Culture, Medicine and Psychiatry, 34(2), 322–352. doi: 10.1007/s11013-010-9170-2 PubMedCrossRefGoogle Scholar
  17. Lustig, S. L., Kia-Keating, M., Knight, W. G., Geltman, P., Ellis, H., Kinzie, J. D., et al. (2004). Review of child and adolescent refugee mental health. Journal of the American Academy of Child & Adolescent Psychiatry, 43(1), 24–36. doi: 10.1097/00004583-200401000-00012 CrossRefGoogle Scholar
  18. Meyer, S., Murray, L. K., Puffer, E. S., Larsen, J., & Bolton, P. (2013). The nature and impact of chronic stressors on refugee children in Ban Mai Nai Soi camp, Thailand. Global Public Health, 8(9), 1027–1047. doi: 10.1080/17441692.2013.811531 PubMedCrossRefGoogle Scholar
  19. Mollica, R. F., Cardozo, B. L., Osofsky, H. J., Raphael, B., Ager, A., & Salama, P. (2004). Mental health in complex emergencies. The Lancet, 364(9450), 2058–2067. doi: 10.1016/S0140-6736(04)17519-3 CrossRefGoogle Scholar
  20. Murray, L. K., Bass, J., Chomba, E., Imasiku, M., Thea, D., Semrau, K., et al. (2011). Validation of the UCLA child post traumatic stress disorder-reaction index in Zambia. International Journal of Mental Health Systems, 5, doi: 10.1186/1752-4458-5-24
  21. Phan, T., & Silove, D. (1999). An overview of indigenous descriptions of mental phenomena and the range of traditional healing practices amongst the Vietnamese. Transcultural Psychiatry, 36(1), 79–94. doi: 10.1177/136346159903600105 CrossRefGoogle Scholar
  22. Puffer, E. S., Larsen, J., Murray, L. K., Meyer, S., & Bolton, P. (2011). Qualitative findings: community perspectives on children’s mental health in Somali refugee camps in Ethiopia. Report prepared for the International Rescue Committee and Bill and Melinda Gates Foundation.Google Scholar
  23. Rasmussen, A., Katoni, B., Keller, A. S., & Wilkinson, J. (2011). Posttraumatic idioms of distress among Darfur refugees: Hozun and Majnun. Transcultural Psychiatry, 48(4), 392–415. doi: 10.1177/1363461511409283 PubMedCrossRefGoogle Scholar
  24. Reed, R. V., Fazel, M., Jones, L., Panter-Brick, C., & Stein, A. (2012). Mental health of displaced and refugee children resettled in low-income and middle-income countries: risk and protective factors. The Lancet, 379(9812), 250–265. doi: 10.1016/S0140-6736(11)60050-0 CrossRefGoogle Scholar
  25. Sagi-Schwartz, A., Seginer, R., & Abdeen, Z. (2008). Chronic exposure to catastrophic war experiences and political violence: links to the well-being of children and their families: introduction to the special issue. International Journal of Behavioral Development, 32(4), 257–259. doi: 10.1177/0165025408092219 CrossRefGoogle Scholar
  26. Silove, D., Bateman, C. R., Brooks, R. T., Fonseca, C. A. Z., Steel, Z., Rodger, J., et al. (2008). Estimating clinically relevant mental disorders in a rural and an urban setting in postconflict Timor Leste. Archives of General Psychiatry, 65(10), 1205–1212. doi: 10.1001/archpsyc.65.10.1205 PubMedCrossRefGoogle Scholar
  27. StataCorp (2012). STATA version 12.1. College Station, TX.Google Scholar
  28. UNHCR (2012). Jijiga population statistics (as of 25 September, 2012).Google Scholar
  29. United Nations. (2012). Report of the United Nations High Commissioner for Refugees. New York: United Nations.Google Scholar
  30. Women's Refugee Commission. (2012). In search of safety and solutions: Somali refugee adolescent girls at Sheder and Aw Barre Camps, Ethiopia.Google Scholar
  31. Yan, H. (2013). State of Syria: exodus reaches 1 million; seesaw battles rage on. Accessed September 4 2013.

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Brian J. Hall
    • 1
    • 2
    Email author
  • Eve Puffer
    • 3
  • Laura K. Murray
    • 1
  • Abdulkadir Ismael
    • 4
    • 5
  • Judith K. Bass
    • 1
  • Amanda Sim
    • 4
  • Paul A. Bolton
    • 1
    • 6
  1. 1.Department of Mental HealthJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  2. 2.Department of PsychologyThe University of MacauMacauChina
  3. 3.Department of Psychology and Neuroscience, Global Health InstituteDuke UniversityDurhamUSA
  4. 4.International Rescue CommitteeNew YorkUSA
  5. 5.Addis Continental Institute of Public HealthAddisEthiopia
  6. 6.Department of International HealthJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA

Personalised recommendations