Assessing local instrument reliability and validity: a field-based example from northern Uganda
- 483 Downloads
This paper presents an approach for evaluating the reliability and validity of mental health measures in non-Western field settings. We describe this approach using the example of our development of the Acholi psychosocial assessment instrument (APAI), which is designed to assess depression-like (two tam, par and kumu), anxiety-like (ma lwor) and conduct problems (kwo maraco) among war-affected adolescents in northern Uganda. To examine the criterion validity of this measure in the absence of a traditional gold standard, we derived local syndrome terms from qualitative data and used self reports of these syndromes by indigenous people as a reference point for determining caseness. Reliability was examined using standard test–retest and inter-rater methods. Each of the subscale scores for the depression-like syndromes exhibited strong internal reliability ranging from α = 0.84–0.87. Internal reliability was good for anxiety (0.70), conduct problems (0.83), and the pro-social attitudes and behaviors (0.70) subscales. Combined inter-rater reliability and test–retest reliability were good for most subscales except for the conduct problem scale and prosocial scales. The pattern of significant mean differences in the corresponding APAI problem scale score between self-reported cases vs. noncases on local syndrome terms was confirmed in the data for all of the three depression-like syndromes, but not for the anxiety-like syndrome ma lwor or the conduct problem kwo maraco.
Keywordswar adolescents northern Uganda mental health validity culture
A number of talented and dedicated collaborators made this work possible. We are endlessly grateful to all the local research assistants who carried out these interviews in Northern Uganda and to the youth and caregivers who participated in the interviews. We are also grateful to World Vision and War Child Holland for their collaboration and support of this field research. This publication was also made possible by Grant #1K01MH077246-01A2 from the National Institute of Mental Health. Additional thanks go to Seganne Musisi, James Okello, Margarita Alegria and Sidney Atwood for their input and review of study design/and or findings.
- 1.Achenbach TM (1991) Manual for the child behavior checklist and 1991 profile. Department of Psychiatry, University of Vermont, BurlingtonGoogle Scholar
- 2.Achenbach TM (1991) Manual for the youth self report and 1991 profile. Department of Psychiatry, University of Vermont, BurlingtonGoogle Scholar
- 9.Betancourt TS, Speelman L, Onyango G, Bolton P (2008) A qualitative study of psychosocial problems of war-affected youth in northern Uganda. J Transcult Psychiatry (in press)Google Scholar
- 19.Geertz C (1973) Interpretation of cultures. Basic Books, New YorkGoogle Scholar
- 20.Goldstein JM, Simpson JC (1995) Validity: definitions and applications to psychiatric research. In: Tsuang MT, Tohen M, Zahner GEP (eds) Textbook in psychiatric epidemiology. Wiley, New York, pp 229–242Google Scholar
- 24.Kleinman A (1988) Rethinking psychiatry: from cultural category to personal experience. Free Press, New YorkGoogle Scholar
- 30.Relief Web (2006) Child soldiers global report: United Nations Office for Coordination of Humanitarian AffairsGoogle Scholar
- 32.Shrout PE (1995) Reliability. In: Tsuang MT, Tohen M, Zahner GE (eds) Textbook in psychiatric epidemiology. Wiley, New York, pp 213–242Google Scholar
- 33.UN OCHA (2007) Forgotten and neglected emergencies: UgandaGoogle Scholar
- 34.Verdeli H, Clougherty K, Onyango G, Lewandowski E, Speelman L, Betancourt TS, Neugebauer R, Stein TR, Bolton P (2008) Group interpersonal psychotherapy for depressed youth in IDP camps in Northern Uganda: adaptation and training. Child Adolesc Psychiatr Clin N Am 17:605–624PubMedCrossRefGoogle Scholar