Skip to main content

Advertisement

SpringerLink
Log in

Clinical Decision-Making in Community Children’s Mental Health: Using Innovative Methods to Compare Clinicians With and Without Training in Evidence-Based Treatment

  • Original Paper
  • Published:
Child & Youth Care Forum Aims and scope Submit manuscript

Abstract

Background

Mental health professionals’ decision-making practice is an area of increasing interest and importance, especially in the pediatric research and clinical communities.

Objective

The present study explored the role of prior training in evidence-based treatments (EBTs) on clinicians’ assessment and treatment formulations using case vignettes. Specifically, study aims included using the naturalistic decision making (NDM) cognitive theory to (1) examine potential associations between EBT training and decision-making processes (novice vs. expert type), and (2) explore how client and family contextual information affects clinical decision-making.

Methods

Forty-eight clinicians across two groups (EBT trained = 14, Not EBT trained = 34) participated. Clinicians were comparable on professional experience, demographics, and discipline. The quasi-experimental design used an analog “think aloud” method where clinicians read case vignettes about a child with disruptive behavior problems and verbalized case conceptualization and treatment planning out loud. Responses were coded according to NDM theory.

Results

MANOVA results were significant for EBT training status such that EBT trained clinicians displayed cognitive processes more closely aligned with “expert” decision-makers and non-EBT trained clinicians’ decision processes were more similar to “novice” decision-makers, following NDM theory. Non-EBT trained clinicians assigned significantly more diagnoses, provided less detailed treatment plans and discussed fewer EBTs. Parent/family contextual information also appeared to influence decision-making.

Conclusion

This study offers a preliminary investigation of the possible broader impacts of EBT training and potential associations with development of expert decision-making skills. Targeting clinicians’ decision-making may be an important avenue to pursue within dissemination-implementation efforts in mental health practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The term evidence-based treatments (EBT) has supplanted the term empirically supported treatment (EST) because it is believed that the word evidence is more immediately and readily understood by non-psychologists of various medical and psychosocial health disciplines (Silverman and Hinshaw 2008).

References

  • Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., et al. (2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382. doi:10.1177/0011000005285875.

    Article  Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (Revised) (4th ed.). Washington, DC: American Psychiatric Association.

    Google Scholar 

  • Anzai, Y. (1991). Learning and use of representations for physics expertise. Towards a general theory of expertise (pp. 64–92).

  • Bakalis, N., Bowman, G., & Porock, D. (2003). Decision making in Greek and English registered nurses in coronary care units. International Journal of Nursing Studies, 40(7), 749–760.

    Article  PubMed  Google Scholar 

  • Baker-Ericzén, M., Hurlburt, M., Brookman-Frazee, L., Jenkins, M., & Hough, R. (2010a). Comparing child, parent and family characteristics in usual care and empirically supported treatment research samples for children with disruptive behavior disorders. Journal of Emotional and Behavioral Disorders, 18(2), 82–99. doi:10.1177/1063426609336956.

    Article  Google Scholar 

  • Baker-Ericzén, M., Jenkins, M., & Brookman-Frazee, L. (2010b). Clinician and parent perspectives on parent and family contextual factors that impact community mental health services for children with behavior problems. Child and Youth Care Forum,. doi:10.1007/s10566-010-9111-9.

    PubMed Central  PubMed  Google Scholar 

  • Becker, K. D., & Stirman, S. W. (2011). The science of training in evidence-based treatments in the context of implementation programs: Current status and prospects for the future. Administration and Policy in Mental Health and Mental Health Services Research (1–6).

  • Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30.

    Google Scholar 

  • Benner, P., Tanner, C., & Chesla, C. (1992). From beginner to expert: Gaining a differentiated clinical world in critical care nursing. Advances in Nursing Science, 14(3), 13–28.

    Article  PubMed  Google Scholar 

  • Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive Science, 16, 153–184.

    Article  Google Scholar 

  • Burns, K. (2005). Mental models and normal errors. In H. Montgomery, R. Lipshitz, & B. Brehmer (Eds.), How professionals make decisions (pp. 15–28). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Chapman, G. B., & Sonnenberg, F. A. (2000). Decision making in health care: Theory, psychology, and applications. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Chorpita, B. F., Becker, K. D., & Daleiden, E. L. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46(5), 647–652.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision. Journal of Clinical Child and Adolescent Psychology, 43, 323–338. doi:10.1080/15374416.2013.828297.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7(1), 5–20.

    Article  PubMed  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

    Article  Google Scholar 

  • Cohen, M. S., Freeman, J. T., & Thompson, B. B. (1997). Training the naturalistic decision maker. In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (pp. 257–268). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Dawes, R. M. (2006). Experience and validity of clinical judgment: The illusory correlation. Behavioral Sciences and the Law, 7(4), 457–467.

    Article  Google Scholar 

  • Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.

    Article  PubMed  Google Scholar 

  • Elliot, T. (2005). Expert decision-making in naturalistic environments: A summary of research. Australia: Defence Science and Technology Organisation, Systems Sciences Laboratory.

    Google Scholar 

  • Elstein, A. S., & Schwarz, A. (2002). Clinical problem solving and diagnostic decision making: Selective review of the cognitive literature. British Medical Journal, 324, 729–732.

    Article  PubMed Central  PubMed  Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis, revised edition: Verbal reports as data. Cambridge, MA: Bradford books/MIT Press.

    Google Scholar 

  • Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 37(1), 215–237.

    Article  PubMed  Google Scholar 

  • Feltovich, P. J., Spiro, R. J., & Coulson, R. L. (1997). Issues of expert flexibility in contexts characterized by complexity and change. Cambridge, MA: The MIT Press.

  • Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993). A description of think aloud method and protocol analysis. Qualitative Health Research, 3(4), 430–441. doi:10.1177/104973239300300403.

    Article  Google Scholar 

  • Galanter, C. A., & Patel, V. L. (2005). Medical decision making: A selective review for child psychiatrists and psychologists. Journal of Child Psychology and Psychiatry, 46(7), 675–689.

    Article  PubMed  Google Scholar 

  • Garb, H. N. (1989). Clinical judgment, clinical training, and professional experience. Psychological Bulletin, 105(3), 387–396.

    Article  PubMed  Google Scholar 

  • Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 15–26. doi:10.1007/s10488-010-0279-y.

    Article  PubMed Central  PubMed  Google Scholar 

  • Garland, A. F., Hough, R. L., McCabe, K. M., Yeh, M., Wood, P. A., & Aarons, G. A. (2001). Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry, 40(4), 409–418.

    Article  PubMed  Google Scholar 

  • Glaser, R. E., & Chi, M. T. H. (1988). Overview. In M. T. H. Chi, R. E. Glaser, & M. J. Farr (Eds.), The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Grasso, B. C., Rothschild, J. M., Genest, R., & Bates, D. W. (2003). What do we know about medication errors in inpatient psychiatry? Joint Commission Journal on Quality and Patient Safety, 29(8), 391–400.

    Google Scholar 

  • Grove, W. M. (2005). Clinical versus statistical prediction: The contribution of Paul E. Meehl. Journal of clinical psychology, 61(10), 1233–1244.

    Article  PubMed  Google Scholar 

  • Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30.

    Article  PubMed  Google Scholar 

  • Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (1998). Multisystemic treatment of antisocial behaviour in children and adolescents. New York: Guilford Press.

    Google Scholar 

  • Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi:10.1016/j.cpr.2010.02.005.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hogan, M. F. (2003). New freedom commission report: The president’s new freedom commission: recommendations to transform mental health care in America. Psychiatric Services, 54(11), 1467.

    Article  PubMed  Google Scholar 

  • Hutton, R. J. B., & Klein, G. (1999). Expert decision making. Systems Engineering, 2, 32–45.

    Article  Google Scholar 

  • Jenkins, M. M. (2012). Cognitive de-biasing and the assessment of pediatric bipolar disorder [Dissertation]. Chapel Hill: Psychology, University of North Carolina at Chapel Hill.

    Google Scholar 

  • Jenkins, M. M., Youngstrom, E. A., Washburn, J., & Youngstom, J. K. (2011). Evidence-based strategies improve assessment of pediatric bipolar disorder by community practitioners. Professional Psychology: Research and Practice, 18, 121–129. doi:10.1037/a0022506.

    Article  Google Scholar 

  • Jensen, A. L., & Weisz, J. R. (2002). Assessing match and mismatch between practitioner-generated and standardized interview-generated diagnoses for clinic-referred children and adolescents. Journal of Consulting and Clinical Psychology, 70(1), 158–168. doi:10.1037/0022-006X.70.1.158.

    Article  PubMed  Google Scholar 

  • Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515–526.

    Article  PubMed  Google Scholar 

  • Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138–147). Norwood, NJ: Ablex.

    Google Scholar 

  • Klein, G. (1997). Developing expertise in decision making. Thinking and Reasoning, 3(4), 337–352.

    Article  Google Scholar 

  • Klein, G. (2008). Naturalistic decision making. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 456–460. doi:10.1518/001872008X288385.

    Article  Google Scholar 

  • Leffler, J. M., Jackson, Y., West, A. E., McCarty, C. A., & Atkins, M. S. (2012). Training in evidence-based practice across the professional continuum. Professional Psychology: Research and Practice. doi:10.1037/a0029241.

  • Lewczyk, C. M., Garland, A. F., Hurlburt, M. S., Gearity, J., & Hough, R. L. (2003). Comparing DISC-IV and clinician diagnoses among youths receiving public mental health services. Journal of the American Academy of Child and Adolescent Psychiatry, 42(3), 349–356. doi:10.1097/00004583-200303000-00016.

    Article  PubMed  Google Scholar 

  • Lloyd, F. J., & Reyna, V. F. (2009). Clinical gist and medical education: Connecting the dots. Journal of the American Medical Association, 302(12), 1332–1333.

    Article  PubMed  Google Scholar 

  • Mass, J. (2003). Evidence-based assessment of children with behavioral and emotional disorders. Report of Emotional and Behavioral Disorders in Youth, 3, 31–34.

    Google Scholar 

  • McHugh, R. K., Murray, H. W., & Barlow, D. H. (2009). Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: The promise of transdiagnostic interventions. Behaviour Research and Therapy, 47(11), 946–953.

    Article  PubMed Central  PubMed  Google Scholar 

  • Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis: University of Minnesota Press.

    Book  Google Scholar 

  • Meehl, P. E. (1986). Causes and effects of my disturbing little book. Journal of Personality Assessment, 50(3), 370–375.

    Article  PubMed  Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Nath, S. B., & Marcus, S. C. (2006). Medical errors in psychiatry. Harvard review of psychiatry, 14(4), 204–211.

    Article  PubMed  Google Scholar 

  • Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Ollendick, T. H., & King, N. J. (2004). Empirically supported treatments for children and adolescents: Advances toward evidence-based practice. In P. M. Barrett & T. H. Ollendick (Eds.), Handbook of interventions that work with children and adolescents: Prevention and treatment (pp. 3–25). West Sussex, England: Wiley.

    Google Scholar 

  • Pappadopulos, E., Jensen, P. S., Schur, S. B., MacIntyre, J. C., I. I., Ketner, S., Van Orden, K., et al. (2002). “Real World” atypical antipsychotic prescribing practices in public child and adolescent inpatient settings. Schizophrenia Bulletin, 28(1), 111–121.

    Article  PubMed  Google Scholar 

  • Patel, V. L., & Arocha, J. F. (2001). The Nature of constraints on collaborative decision making in health care settings. In E. Salas & G. Klein (Eds.), Linking expertise and naturalistic decision making (pp. 383–405). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Patel, V. L., Arocha, J. F., & Kaufman, D. R. (1994). Diagnostic reasoning and medical expertise. The Psychology of Learning and Motivation: Advances in Research and Theory, 31, 187–252.

    Article  Google Scholar 

  • Patel, V. L., Arocha, J. F., & Kaufman, D. R. (2001). A primer on aspects of cognition for medical informatics. Journal of the American Medical Informatics Association, 8(4), 324–343.

    Article  PubMed Central  PubMed  Google Scholar 

  • Patel, V. L., Kaufman, D. R., & Arocha, J. F. (2002). Emerging paradigms of cognition in medical decision-making. Journal of Biomedical Informatics, 35(1), 52–75.

    Article  PubMed  Google Scholar 

  • Redelmeier, D. A. (2005). The cognitive psychology of missed diagnoses. Annals of Internal Medicine, 142(2), 115–120.

    Article  PubMed  Google Scholar 

  • Sackett, D. L., Rosenberg, W. M. C., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312, 71–72.

    Article  PubMed Central  PubMed  Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi:10.1007/s10488-010-0321-0.

    Article  PubMed Central  PubMed  Google Scholar 

  • Serfaty, D., MacMillan, J., Entin, E. E., & Entin, E. B. (1997). The decision-making expertise of battle commanders. Naturalistic Decision Making, 233–246.

  • Silverman, W. K., & Hinshaw, S. P. (2008). The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update. Journal of Clinical Child and Adolescent Psychology, 37(1), 1–7.

    Article  Google Scholar 

  • Simon, H. A. (1992). What is an “explanation” of behavior? Psychological Science, 3(3), 150–161. doi:10.1111/j.1467-9280.1992.tb00017.x.

    Article  Google Scholar 

  • Spring, B. (2008). Health decision making: Lynchpin of evidence-based practice. Medical Decision Making, 28(6), 866–874.

    Article  PubMed Central  PubMed  Google Scholar 

  • Stahl, C., & Klauer, K. C. (2008). A simplified conjoint recognition paradigm for the measurement of gist and verbatim memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(3), 570–586. doi:10.1037/0278-7393.34.3.570.

    PubMed  Google Scholar 

  • Tabachnick, B., & Fidell, L. (2007). Using multivariate statistics (5th ed.). Boston: Allyn and Bacon.

    Google Scholar 

  • Trask, E. V., & Garland, A. F. (2012). Are children improving? Results from outcome measurement in a large mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 39(3), 210–220.

    Article  PubMed  Google Scholar 

  • Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics and Biases. Science, 185, 1124–1131.

    Article  PubMed  Google Scholar 

  • Van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical guide to modelling cognitive processes. London: Academic Press.

    Google Scholar 

  • Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78(2), 144–155.

    Article  PubMed  Google Scholar 

  • Webster-Stratton, C. (1984). Randomized trial of two parent-training programs for families with conduct-disordered children. Journal of Consulting and Clinical Psychology, 52(4), 666–678.

    Article  PubMed  Google Scholar 

  • Weisz, J. R. (2004). Psychotherapy for children and adolescents: Evidence-based treatments and case examples. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Weisz, J. R., & Jensen, A. L. (2001). Child and adolescent psychotherapy in research and practice contexts: Review of the evidence and suggestions for improving the field. European Journal of Child and Adolescent Psychiatry, 10, 12–18.

    Article  Google Scholar 

  • Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61(7), 671–689. doi:10.1037/0003-066X.61.7.671.

    Article  PubMed  Google Scholar 

  • Whyte, G. (2001). Perspectives on naturalistic decision making from organizational behavior. Journal of Behavioral Decision Making, 14(5), 383–384.

    Article  Google Scholar 

  • Witteman, C. L. M., & van der Bercken, J. H. L. (2007). Intermediate effects in psychodiagnostic classification. European Journal of Psychological Assessment, 23, 56–61.

    Article  Google Scholar 

Download references

Acknowledgments

Support for this work comes from National Institute of Mental Health Mentored Research Scientist Development Awards K01-MH69665 (M.B.E.) and National Institute of Mental Health award R01-MH66070 (A.G). The authors thank Greg McKoon, BA, Mary Garnand Mueggenborg, MSW and Cynthia Fuller, BA for technical support, Scott Roesch, PhD for statistical consultation and Lauren Brookman-Frazee, Ph.D. for her scientific review.

Author information

Authors and Affiliations

  1. Child and Adolescent Services Research Center (CASRC), Rady Children’s Hospital, San Diego, 3020 Children’s Way, MC 5033, San Diego, CA, 92123, USA

    Mary J. Baker-Ericzén, Soojin Park & Ann F. Garland

  2. Department of Psychiatry, University of California, San Diego, CA, USA

    Melissa M. Jenkins

  3. Department of School, Family and Mental Health Professions, University of San Diego, San Diego, CA, USA

    Ann F. Garland

Authors
  1. Mary J. Baker-Ericzén
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Melissa M. Jenkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soojin Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ann F. Garland
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mary J. Baker-Ericzén.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baker-Ericzén, M.J., Jenkins, M.M., Park, S. et al. Clinical Decision-Making in Community Children’s Mental Health: Using Innovative Methods to Compare Clinicians With and Without Training in Evidence-Based Treatment. Child Youth Care Forum 44, 133–157 (2015). https://doi.org/10.1007/s10566-014-9274-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10566-014-9274-x

Keywords

Search

Navigation