Outcome Assessment via Handheld Computer in Community Mental Health: Consumer Satisfaction and Reliability

  • Lizabeth A. Goldstein
  • Mary Beth Connolly Gibbons
  • Sarah M. Thompson
  • Kelli Scott
  • Laura Heintz
  • Patricia Green
  • Donald Thompson
  • Paul Crits-Christoph


Computerized administration of mental health-related questionnaires has become relatively common, but little research has explored this mode of assessment in “real-world” settings. In the current study, 200 consumers at a community mental health center completed the BASIS-24 via handheld computer as well as paper and pen. Scores on the computerized BASIS-24 were compared with scores on the paper BASIS-24. Consumers also completed a questionnaire which assessed their level of satisfaction with the computerized BASIS-24. Results indicated that the BASIS-24 administered via handheld computer was highly correlated with pen and paper administration of the measure and was generally acceptable to consumers. Administration of the BASIS-24 via handheld computer may allow for efficient and sustainable outcomes assessment, adaptable research infrastructure, and maximization of clinical impact in community mental health agencies.


Community mental health Outcome assessment Technology 


  1. 1.
    Beutler LE, Malik M, Talebi H, et al. Use of psychological tests/instruments for treatment planning. In: Marush ME, (ed). The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. Mahwah, NJ: Lawrence Erlbaum Associates; 2004:111–146.Google Scholar
  2. 2.
    Derogatis LR, Culpepper WJ. Screening for psychiatric disorders. In: Marush ME, ed. The Use of Psychological Testing for Treatment Planing and Outcomes Assessment. Mahwah, NJ: Lawrence Erlbaum Associates; 2004:65–110.Google Scholar
  3. 3.
    Lambert MJ, Hawkins EJ. Use of psychological tests for assessing treatment outcomes. In: Marush ME, ed. The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. Mahwah, NJ: Lawrence Erlbaum Associates; 2004:171–195Google Scholar
  4. 4.
    Palen L-A, Graham JW, Smith EA, et al. Rates of missing responses in personal digital assistant (PDA) versus paper assessments. Evaluation Review. 2008;32:257–272.PubMedCrossRefGoogle Scholar
  5. 5.
    Ahluwalia MK. Multicultural issues in computer-based assessment. In: Suzuki LA, Ponterotto JG, eds. Handbook of multicultural assessment: Clinical, psychological, and educational applications. 3 rd ed. San Francisco, CA: Jossey-Bass; 2008:92–106.Google Scholar
  6. 6.
    Wolford G, Rosenberg SD, Rosenberg HJ, et al.A Clinical Trial Comparing Interviewer and Computer-Assisted Assessment Among Clients With Severe Mental Illness. Psychiatric Services. 2008;59:769–775.PubMedCrossRefGoogle Scholar
  7. 7.
    Eisen SV, Toche-Manley LL, Grissom GR. Computer-Administered Versus Paper-and-Pencil Mental Health Surveys. Psychiatric Services. 2004;55:1316–1317.PubMedCrossRefGoogle Scholar
  8. 8.
    Kobak KA, Greist JH, Jefferson JW, et al. Computer-administered clinical rating scales: A review. Psychopharmacology. 1996;127:291–301.PubMedGoogle Scholar
  9. 9.
    Chinman M, Young AS, Schell T, et al. Computer-assisted self-assessment in persons with severe mental illness. Journal of Clinical Psychiatry. 2004;65:1343–1351.PubMedCrossRefGoogle Scholar
  10. 10.
    Schmitz N, Hartkamp N, Brinschwitz C, et al. Computerized administration of the Symptom Checklist (SCL-90-R) and the Inventory of Interpersonal Problems (IIP-C) in psychosomatic outpatients. Psychiatry Research. 1999;87:217–221.PubMedCrossRefGoogle Scholar
  11. 11.
    Wijndaele K, Matton L, Duvigneaud N, et al. Reliability, equivalence, and respondent preference of computerized versus paper-and-pencil mental health questionnaires. Computers in Human Behavior. 2007;23:1958–1970.CrossRefGoogle Scholar
  12. 12.
    Chan-Pensley E. Alcohol-use disorders identification test: A comparison between paper and pencil and computerized versions. Alcohol & Alcoholism. 1999;34:882–885.CrossRefGoogle Scholar
  13. 13.
    Cook IA, Balasubramani GK, Eng H, et al. Electronic source materials in clinical research: Acceptability and validity of symptom self-rating in major depressive disorder. Journal of Psychiatric Research. 2007;41:737–743.PubMedCrossRefGoogle Scholar
  14. 14.
    Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health. 2008;11:322–333.PubMedCrossRefGoogle Scholar
  15. 15.
    Weber B, Schneider B, Fritze J, et al. Acceptance of computerized compared to paper-and-pencil assessment in psychiatric inpatients. Computers in Human Behavior. 2003;19:81–93.CrossRefGoogle Scholar
  16. 16.
    Summerville A, Roese NJ. Dare to compare: Fact-based versus simulation based comparison in daily life. Journal of Experimental Social Psychology. 2008;44:664–671.PubMedCrossRefGoogle Scholar
  17. 17.
    Epstein DH, Willner-Reid J, Vahabzadeh M, et al. Real-time electronic diary reports of cue exposure and mood in the hours before cocaine and heroin craving and use. Archives of General Psychiatry. 2009;66:88–94.PubMedCrossRefGoogle Scholar
  18. 18.
    Muehlenkamp JJ, Engel SG, Wadeson A, et al. Emotional state preceding and following acts of non-suicidal self-injury in bulimia nervosa patients. Behaviour Research and Therapy. 2009;47:83–87.PubMedCrossRefGoogle Scholar
  19. 19.
    Bernhardt JM, Usdan S, Mays D, et al. Alcohol assessment using wireless handheld computers: A pilot study. Addictive Behaviors. 2007;32:3065–3070.PubMedCrossRefGoogle Scholar
  20. 20.
    Younger J, Mackey S. Fibromyalgia symptoms are reduced by low-dose naltrexone. Pain Medicine. 2009;10:663–672.PubMedCrossRefGoogle Scholar
  21. 21.
    Verduyn P, Delvaux E, Van Coillie H, et al. Predicting the duration of emotional experience: Two experience sampling studies. Emotion. 2009;9:83–91.PubMedCrossRefGoogle Scholar
  22. 22.
    Granholm E, Loh C, Swendsen J. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia Bulletin. 2008;34:507–514.PubMedCrossRefGoogle Scholar
  23. 23.
    Shannon LM, Walker R, Blevins M. Developing a new system to measure outcomes in a service coordination program for youth with severe emotional disturbance. Evaluation and Program Planning. 2009;32:109–118.PubMedCrossRefGoogle Scholar
  24. 24.
    Eisen SV, Normand SL, Belanger AJ, et al. The Revised Behavior and Symptom Identification Scale (BASIS-R): Reliability and validity. Medical Care. 2004;42:1230–1341.PubMedCrossRefGoogle Scholar
  25. 25.
    Eisen SV, Gerena M, Ranganathan G, et al. Reliability and validity of the BASIS-24 mental health survey for Whites, African-Americans, and Latinos. Journal of Behavioral Health Services & Research. 2006;33:304–323.CrossRefGoogle Scholar
  26. 26.
    Palmblad M, Tiplady, B. Electronic diaries and questionnaires: Designing user interfaces that are easy for all patients to use. Quality of Life Research. 2004;13:1199–1207.PubMedCrossRefGoogle Scholar

Copyright information

© National Council for Community Behavioral Healthcare 2010

Authors and Affiliations

  • Lizabeth A. Goldstein
    • 1
  • Mary Beth Connolly Gibbons
    • 2
  • Sarah M. Thompson
    • 1
  • Kelli Scott
    • 1
  • Laura Heintz
    • 1
  • Patricia Green
    • 3
  • Donald Thompson
    • 4
  • Paul Crits-Christoph
    • 5
  1. 1.University of PennsylvaniaPhiladelphiaUSA
  2. 2.University of PennsylvaniaPhiladelphiaUSA
  3. 3.Northwestern Human ServicesPhiladelphiaUSA
  4. 4.Northwestern Human ServicesErdenheimUSA
  5. 5.University of PennsylvaniaPhiladelphiaUSA

Personalised recommendations