Administration and Policy in Mental Health and Mental Health Services Research

, Volume 41, Issue 5, pp 647–659

Pitfalls in the Assessment, Analysis, and Interpretation of Routine Outcome Monitoring (ROM) Data: Results from an Outpatient Clinic for Integrative Mental Health

  • Rogier H. J. Hoenders
  • Elisabeth H. Bos
  • Agna A. Bartels-Velthuis
  • Nina K. Vollbehr
  • Karen van der Ploeg
  • Peter de Jonge
  • Joop T. V. M. de Jong
Original Article

DOI: 10.1007/s10488-013-0511-7

Cite this article as:
Hoenders, R.H.J., Bos, E.H., Bartels-Velthuis, A.A. et al. Adm Policy Ment Health (2014) 41: 647. doi:10.1007/s10488-013-0511-7

Abstract

There is considerable debate about routine outcome monitoring (ROM) for scientific or benchmarking purposes. We discuss pitfalls associated with the assessment, analysis, and interpretation of ROM data, using data of 376 patients. 206 patients (55 %) completed one or more follow-up measurements. Mixed-model analysis showed significant improvement in symptomatology, quality of life, and autonomy, and differential improvement for different subgroups. Effect sizes were small to large, depending on the outcome measure and subgroup. Subtle variations in analytic strategies influenced effect sizes substantially. We illustrate how problems inherent to design and analysis of ROM data prevent drawing conclusions about (comparative) treatment effectiveness.

Keywords

Routine outcome monitoringTreatment outcomeIntegrative psychiatryBiasConfounding

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Rogier H. J. Hoenders
    • 1
  • Elisabeth H. Bos
    • 2
  • Agna A. Bartels-Velthuis
    • 1
    • 3
  • Nina K. Vollbehr
    • 1
  • Karen van der Ploeg
    • 1
  • Peter de Jonge
    • 2
  • Joop T. V. M. de Jong
    • 4
    • 5
    • 6
  1. 1.Center for Integrative Psychiatry, LentisGroningenThe Netherlands
  2. 2.Interdisciplinary Center Psychopathology and Emotion regulation, University Medical Center GroningenUniversity of GroningenGroningenThe Netherlands
  3. 3.University Center for Psychiatry, University Medical Center GroningenUniversity of GroningenGroningenThe Netherlands
  4. 4.Amsterdam Institute for Social Science Research (AISSR)University of AmsterdamAmsterdamThe Netherlands
  5. 5.Boston School of MedicineBostonUSA
  6. 6.Rhodes UniversityGrahamstownSouth Africa