Skip to main content
SpringerLink
Log in

Validation of the French versions of two brief, clinician-friendly outcome monitoring tools: the ORS and SRS

  • Published:
Current Psychology Aims and scope Submit manuscript

Abstract

While Routine Outcome Monitoring (ROM) of change and therapeutic alliance is proving to be a promising way to address the issues of drop-out and client deterioration in psychotherapy, at present ROM is infrequently employed in French-speaking contexts. This study aimed at testing psychometric properties of the French versions of two popular and widely used ROM tools in the Partners for Change Outcome Management System (PCOMS): The Outcome Rating Scale (ORS) and Session Rating Scale (SRS). The project necessitated a multiple study survey with online data collection of clinical and non-clinical samples, to investigate psychometric qualities (including factorial structure of the two scales), using parametric and non-parametric procedures. In addition, we documented, preliminary estimates regarding clinical cut-off and reliable change indices for ORS. Both ORS and SRS have good psychometric properties regarding brevity. Reliability, normative data, a reliable change index and scale thresholds are reported for the French versions. The clinical use of PCOMS appears promising regarding delivering, regulation, and improvement of psychotherapy in French-speaking areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. (http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=249).

  2. https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf

  3. https://www.hcp.med.harvard.edu/ncs/k6_scales.php

  4. http://www.statmodel2.com/discussion/messages/9/572.html?1319030818

References

  • Anglim, J., & Grant, S. (2016). Predicting psychological and subjective well-being from personality: Incremental prediction from 30 facets over the big 5. Journal of Happiness Studies, 17, 59–80. https://doi.org/10.1007/s10902-014-9583-7.

    Article  Google Scholar 

  • Anker, M. G., Duncan, B. L., & Sparks, J. A. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77(4), 693–704.

    Article  Google Scholar 

  • Arnaud, B., Malet, L., Teissedre, F., Izaute, M., Moustafa, F., Geneste, J., Schmidt, J., et al. (2010). Validity study of Kessler’s psychological distress scales conducted among patients admitted to French emergency department for alcohol consumption-related disorders. Alcoholism: Clinical and Experimental Research, 34(7), 1235–1245.

    Google Scholar 

  • Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M. A. Hubble, B. L. Duncan, & S. D. Miller (Eds.), The heart and soul of change: What works in therapy (pp. 133–178). APA Press.

  • Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75(6), 842–852.

    Article  Google Scholar 

  • Bertolino, B., Bargmann, S., & Miller, S. (2012). International Center for Clinical Excellence manuals on feedback-informed treatment (FIT): Manual 1--what works in therapy: A primer. Chicago, IL: International Center for Clinical Excellence.

  • Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260.

    Article  Google Scholar 

  • Bringhurst, D. L., Watson, C. S., Miller, S. D., & Duncan, B. L. (2006). The reliability and validity of the outcome rating scale: A replication study of a brief clinical measure. Journal of Brief Therapy, 5(1), 23–29.

    Google Scholar 

  • Brown, J., Dreis, S., & Nace, D. (1999). What really makes a difference in psychotherapy outcome? Why does managed care want to know? In M. Hubble, B. Duncan, & S. Miller (Eds.), The heart and soul of change (pp. 389–406). APA Press.

  • Campbell, A., & Hemsley, S. (2009). Outcome rating scale and session rating scale in psychological practice: Clinical utility of ultra-brief measures. Clinical Psychologist, 13(1), 1–9.

    Article  Google Scholar 

  • Cazauvieilh, C. (2014) Améliorer son efficacité clinique et dynamiser sa croissance professionnelle : des méthodes simples au service du changement. Session de poster, vendredi 12 décembre 2014, congrès de l’Association Française de Thérapie Comportementale et Cognitive (AFTCC), Paris, maison de la chimie.

  • Cazauvieilh, C. (2015). De la lutte pour ressentir à l'acceptation pour mieux agir. Une approche ACT intégrative du traitement des attaques de panique et des expériences de dépersonnalisation-déréalisation. In J. C. Seznec (Ed.), ACT: applications thérapeutiques. DUNOD.

    Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.

    Article  Google Scholar 

  • Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Elsevier Science.

  • Conrath, P., & Houdry, P. (2017). Les psychologues, remboursés ? Le Journal des psychologues, 343(1), 3–3. https://doi.org/10.3917/jdp.343.0003.

    Article  Google Scholar 

  • Cooper, M. (2008). Essential research findings in Counselling and psychotherapy. Sage.

  • Corbière, M., Bisson, J., Lauzon, S., & Ricard, N. (2006). Factorial validation of a French short-form of the working Alliance inventory. International Journal of Methods in Psychiatric Research, 15(1), 36–45.

    Article  Google Scholar 

  • Duncan, B. L., Miller, S. D., Wampold, B. E., & Hubble, M. A. (Eds.). (2010). The heart and soul of change: Delivering what works in therapy (2nd ed.). American Psychological Association. https://doi.org/10.1037/12075-000.

  • Evans, C., Connell, J., Barkham, M., Margison, F., McGrath, G., Mellor-Clark, J., & Audin, K. (2002). Towards a standardised brief outcome measure: Psychometric properties and utility of the CORE–OM. British Journal of Psychiatry, 180(1), 51–60.

    Article  Google Scholar 

  • Flynn, R. J., Aubry, T. D., Guindon, S., Tardif, L., Viau, M., & Gallant, A. (2003). Validation d’une version française abrégée du Outcome Questionnaire et évaluation d’un service de counselling en milieu clinique. Canadian Journal of Program Evaluation, 17(3), 57–74.

    Google Scholar 

  • French International Test Commission (2014). International guidelines on the security of tests, examinations, and other assessments. [www.intestcom.org].

  • Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.

    Article  Google Scholar 

  • Horvath, A. O., & Bedi, R. P. (2002). The Alliance. In J. Norcross (Ed.), Psychotherapy relationships that work: therapist contributions and responsiveness to patients (pp. 37–70). Oxford University Press.

  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12

    Article  PubMed  Google Scholar 

  • Janse, P., Boezen-Hilberdink, L., Van, D., Verbraak, M. K., Marc, J. P. M., & Hutschemaekers, G. J. M. (2014). Measuring feedback from clients: The psychometric properties of the dutch outcome rating scale and session rating scale. European Journal of Psychological Assessment, 30(2), 86–92.

    Article  Google Scholar 

  • John, O, P., Donahue, E, M., & Kentle, R, L. (1991). Big five inventory. PsycTESTS Dataset. https://doi.org/10.1037/t07550-000.

  • Kessler, R. G., Barker, P. R., Colpe, L. J., Epstein, J. F., Gfroerer, J. C., Hiripi, E., Howes, M. J., & Zaslavsky, A. M. (2003). Screening for serious mental illness in the general population. Archives of General Psychiatry, 60(2), 184–189.

    Article  Google Scholar 

  • Lambert, M. J. (2015). Progress feedback and the OQ-system: The past and the future. Psychotherapy, 52(4), 381–390.

    Article  Google Scholar 

  • Lambert, M. J., & Burlingame, G. M. (1996a). Outcome Questionnaire (OQ 45.2). Stevenson, NM: American Professional Credentialing Services.

    Google Scholar 

  • Lambert, M. J., & Burlingame, G. M. (1996b). Outcome Questionnaire (OQ HAI-22). Stevenson, NM: American Professional Credentialing Services.

    Google Scholar 

  • Lambert, M. J., & Ogles, B. M. (1997). The effectiveness of psychotherapy supervision. In C. E. Watkins (Ed.), Handbook of Psychotherapy Supervision (pp. 421–446). Wiley.

  • Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 139–193). Wiley.

  • Lambert, M. J., Whipple, J. L., & Kleinstauber, M. (2018). Collecting and delivering Progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520–537.

    Article  Google Scholar 

  • Miller, S. D., & Duncan, B. L. (2004). The outcome and session rating scales: Administration and scoring manual. Chicago, IL: ISTC.

    Google Scholar 

  • Miller, S. D., & Schuckard, E. (2016). Psychometrics of the ORS and SRS, Results from rcts and meta-analyses of routine outcome monitoring & feedback. IL: The available evidence. Chicago https://www.scottdmiller.com/wp-content/uploads/2016/09/Measures-and-Feedback-2016.pdf.

  • Miller, S. D., Duncan, B. L., & Johnson, L. D. (2000). The session rating scale 3.0. Chicago, IL: Authors.

  • Miller, S. D., Duncan, B. L., Brown, J., Sparks, J. A., & Claud, D. A. (2003). The outcome rating scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2(2), 91–100.

  • Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The partners for change outcome management system. Journal of Clinical Psychology, 61(2), 199–208.

    Article  Google Scholar 

  • Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449–457.

    Article  Google Scholar 

  • Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S., & Kircher, J. C. (2008). Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment: A preliminary study. Journal of Consulting and Clinical Psychology., 76, 116–124. https://doi.org/10.1037/0022-006X.76.1.116.

    Article  PubMed  Google Scholar 

  • Moggia, D., Niño-Robles, N., Miller, S.D., & Feixas, G. (2017, September). Psychometric properties of the outcome rating scale (ORS) and session rating scale 3.0 (SRS 3.0) in a spanish clinical sample. In F. Giannone (chair). Brief paper session: Quantitative & qualitative method. In Society for Psychotherapy Research UK & European Chapters 4th Joint Conference, Oxford.

  • Moggia, D., Niño-Robles, N., Miller, S., & Feixas, G. (2018). Psychometric properties of the outcome rating scale (ORS) in a spanish clinical sample. The Spanish Journal of Psychology, 21(E30). https://doi.org/10.1017/sjp.2018.32.

  • Muthén, L. K., & Muthén, B. O. (2010). Mplus user's guide. Los Angeles, CA: Muthén & Muthén.

    Google Scholar 

  • Nguyen, T. D., Attkisson, C. C., & Stegner, B. L. (1983). Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6, 299–314.

    Article  Google Scholar 

  • Nguyen, L., Blasquez, S., Bataille, B., & Chassery, C. (2012). La détresse psychologique mesurée par le score de Kessler (K6) prédit les douleurs postopératoires prolongées après chirurgie du poignet. Canadian Journal of Anesthesia/Journal Canadien D’anesthésie, 59(12), 1150–1151. https://doi.org/10.1007/s12630-012-9783-8.

    Article  PubMed  Google Scholar 

  • Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2003). Fifty years of process-outcome research:Continuity and change. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 307-390). New York: Wiley.

    Google Scholar 

  • Pinner, D. H., & Kivlighan III, D. M. (2018). The ethical implications and utility of routine outcome monitoring in determining boundaries of competence in practice. Professional Psychology: Research and Practice, 49(4), 247–254.

    Article  Google Scholar 

  • Plaisant, O., Courtois, R., Réveillère, C., Mendelsohn, G. A., & John, O. P. (2010). Validation par analyse factorielle du Big Five Inventory français (BFI-Fr). Analyse convergente avec le NEO-PI-R. Annales Médico-Psychologiques, Revue Psychiatrique, 168(2), 97–106. https://doi.org/10.1016/j.amp.2009.09.003.

    Article  Google Scholar 

  • Quirk, K., Miller, S., Duncan, B., & Owen, J. (2013). Group session rating scale: Preliminary psychometrics in substance abuse group interventions. Counselling and Psychotherapy Research, 13(3), 194–200. https://doi.org/10.1080/14733145.2013.764658.

    Article  Google Scholar 

  • Reese, R. J., Gillaspy, J. A., Owen, J. J., Flora, K. L., Cunningham, L. C., Archie, D., & Marsden, T. (2013). The influence of demand characteristics and social desirability on clients’ ratings of the therapeutic alliance. Journal of Clinical Psychology, 69(7), 696–709. https://doi.org/10.1002/jclp.21946.

    Article  PubMed  Google Scholar 

  • Seidel, J. A., Andrews, W. P., Owen, J., Miller, S. D., & Buccino, D. L. (2017). Preliminary validation of the rating of outcome scale and equivalence of ultra-brief measures of well-being. Psychological Assessment, 29(1), 65–75. https://doi.org/10.1037/pas0000311.

  • Shadish, W. R., Matt, G. E., Navarro, A. M., & Phillips, G. (2000). The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin, 126, 512–529.

    Article  Google Scholar 

  • Smith, D., Crocker, L, B., Staton, C., Gillaspy, A., & Charlton, S, R. (2010, April). Psychometric properties of the outcome rating scale in a non-clinical population. Poster session presented at the annual convention of the southwestern psychological association, Dallas, TX. Retrieved on the internet the 19/04/2018.

  • Swift, J. K., & Greenberg, R. P. (2012). Premature discontinuation in adult psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology, 80, 547–559.

    Article  Google Scholar 

  • Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the working alliance inventory. Psych Assess, 1(3), 207–210.

    Article  Google Scholar 

  • Vallerand, R. J. (1989). Vers Une méthodologie de validation trans-culturelle de questionnaires psychologiques: Implications pour la recherche en langue française [toward a methodology for the transcultural validation of psychological questionnaires: Implications for research in the French language]. Canadian Psychology/Psychologie Canadienne, 30(4), 662–680. https://doi.org/10.1037/h0079856.

    Article  Google Scholar 

  • Wampold, B. E. (2001). The great psychotherapy debate: Model, methods, and findings. Lawrence Erlbaum Associates.

  • Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work.

    Book  Google Scholar 

Download references

Acknowledgements

We want to especially thank Lee Kathryn Valente Ph.D for English editing.

The first author wants also to especially thank the comity composed of clinicians from Valoris for Children and Adults of Prescott-Russell Agency in Canada (Raymond Lemay, Chantal Tassé, Madeleine Lalonde, Nicholas Cardinal), and John Deltour (Belgium) who worked to develop a unified French version of the measures.

Author information

Authors and Affiliations

  1. INSERM, UMR 1219, équipe “Handicap, Activité, Cognition, Santé”, University of Bordeaux , Bordeaux, France

    Christophe Cazauvieilh

  2. University of Bordeaux, Bordeaux, France

    Kamel Gana

  3. International Center for Clinical Excellence, Chicago, IL, USA

    Scott D. Miller

  4. INSERM, UMR 1219, équipe “Handicap, Activité, Cognition, Santé”, University of Bordeaux, Bordeaux, France

    Bruno Quintard

Authors
  1. Christophe Cazauvieilh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamel Gana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Scott D. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno Quintard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christophe Cazauvieilh.

Ethics declarations

Conflict of Interest

Dr. Christophe Cazauvieilh uses PCOMS in his training activities. Dr. Scott D Miller is a first author on the PCOMS scales.

Research Involving Human Participants

This research complied with French and international ethical standards regarding data collection in human science and was declared to the french CNIL registry.

Informed Consent

Informed consent (written or electronically) was required from all participants (therapists and clients) before the data collection.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cazauvieilh, C., Gana, K., Miller, S.D. et al. Validation of the French versions of two brief, clinician-friendly outcome monitoring tools: the ORS and SRS. Curr Psychol 41, 6124–6136 (2022). https://doi.org/10.1007/s12144-020-00992-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12144-020-00992-x

Keywords

Search

Navigation