Skip to main content

Which Methodological Practice(s) for Psychotherapy Science? A Systematic Review and a Proposal

Integrative Psychological and Behavioral Science volume 54, pages 215–248 (2020)Cite this article

A Correction to this article was published on 05 August 2019

This article has been updated

Abstract

Three main methodological practices have been employed in psychotherapy science: the empirical-quantitative, empirical-qualitative, and theoretical one. Some scholars have argued for a monopoly of the empirical-quantitative methodological practice, sustained by scientific monism. This systematic review aims at exploring the contribution of each methodological practice to the field. Fifteen journals were searched from 2003 to 2013. A total of 9796 publications and 9915 studies met inclusion criteria and were coded for the methodological practice employed. Empirical-quantitative studies were the most published overall, over time, and in most of the journals considered. This overwhelming prevalence increased when considering the citation rates of the publications. We argue that these results are indicative of a quantitative monopoly, which in turn is due to a lack of critical thinking sustaining scientific monism. We call for disciplinary critical thinking and the consequent scientific pluralism valuing the plurality, diversity, and multiplicity of all the existing methodological practices.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Change history

  • 05 August 2019

    We would like to note the following corrections to the original version of Zhang, Sun, Lin and Ren.

Notes

  1. It should be noted that, usually, a publication comprising more than one study can be the case for empirical papers – more often for quantitative than for qualitative ones – but not for theoretical papers.

  2. It should be noted that Alan Kazdin, authoritative methodologist and representative of the empirical-quantitative approach, has stated that “. . . after decades of psychotherapy research and thousands of studies, there is no evidence-based explanation of how or why even the most well-studied interventions produce change” (Kazdin 2009, p. 426).

References

  • Addis, M. E., & Waltz, J. (2006). Implicit and untested assumptions about the role of psychotherapy treatment manuals in evidence-based mental mealth practice. Clinical Psychology: Science and Practice, 9(4), 421–424. https://doi.org/10.1093/clipsy.9.4.421.

    Article  Google Scholar 

  • Andreasen, N. C., Carpenter, W. T., Kane, J. M., Lasser, R. A., Marder, S. R., & Weinberger, D. R. (2005). Remission in schizophrenia: Propsed criteria and Raionale for consensus. The American Journal of Psychiatry, 162(3), 441–449. https://doi.org/10.1176/appi.ajp.162.3.441.

    Article  PubMed  Google Scholar 

  • APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285. https://doi.org/10.1037/0003-066X.61.4.271.

    Article  Google Scholar 

  • Aria, M., & Cuccurullo, C. (2017). bibliometrix : An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007.

    Article  Google Scholar 

  • Bakan, D. (1966). Behaviorism and American urbanization. Journal of the History of the Behavioral Sciences, 2(1), 5–28. https://doi.org/10.1002/1520-6696(196601)2:1<5::AID-JHBS2300020103>3.0.CO;2-8.

    Article  Google Scholar 

  • Barker, C., Pistrang, N., & Elliott, R. (2002). Research methods in clinical psychology: An introduction for students and practitioners. Chichester: Wiley.

    Google Scholar 

  • Barlow, D. H., Allen, L. B., & Choate, M. L. (2004). Toward a unified treatment for emotional disorders. Behavior Therapy, 35(2), 205–230. https://doi.org/10.1016/S0005-7894(04)80036-4.

    Article  Google Scholar 

  • Bassett, A. S., Scherer, S. W., & Brzustowicz, L. M. (2010). Copy number variations in schizophrenia: Critical review and new perspectives on concepts of genetics and disease. The American Journal of Psychiatry, 167(8), 899–914. https://doi.org/10.1176/appi.ajp.2009.09071016.

    Article  PubMed  PubMed Central  Google Scholar 

  • Beidas, R., & Kendall, P. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30. https://doi.org/10.1111/j.1468-2850.2009.01187.x.

    Article  Google Scholar 

  • Bishop, S. R., Lau, M., Shapiro, S., Carlson, L., Anderson, N. D., Carmody, J., et al. (2004). Mindfulness: A proposed operational definition. Clinical Psychology: Science and Practice, 11(3), 230–241. https://doi.org/10.1093/clipsy/bph077.

    Article  Google Scholar 

  • Braakmann, D. (2015). Historical paths in psychotherapy research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 39–65). https://doi.org/10.1007/978-3-7091-1382-0_3.

    Chapter  Google Scholar 

  • Caro Gabalda, I., & Stiles, W. B. (2013). Irregular assimilation progress: Reasons for setbacks in the context of linguistic therapy of evaluation. Psychotherapy Research, 23(1), 35–53. https://doi.org/10.1080/10503307.2012.721938.

    Article  PubMed  Google Scholar 

  • Caspi, A., Hariri, A. R., Holmes, A., & Rudolf Uher, T. E. M. (2010). Genetic sensitivity to the environment: The case of the serotonin transporter gene and its implications for studying complex diseases and traits. American Journal of Psychiatry, 167(5), 509–527. https://doi.org/10.1176/appi.ajp.2010.09101452.

    Article  PubMed  Google Scholar 

  • Chambers, R. A., Taylor, J. R., & Potenza, M. N. (2003). Developmental neurocircuitry of motivation in adolescence: A critical period of addiction vulnerability. American Journal of Psychiartry, 160, 1041–1052. https://doi.org/10.1176/appi.ajp.160.6.1041.

    Article  Google Scholar 

  • Charney, D. S. (2004). Psychobiological mechanism of resilience and vulnerability: Implications for successful adaptation to extreme stress. American Journal of Psychiatry, 161(2), 195–216. https://doi.org/10.1176/appi.ajp.161.2.195.

    Article  PubMed  Google Scholar 

  • Chawla, N., & Ostafin, B. (2007). Experiential avoidance as a functional dimensional approach to psychopathology: An empirical review. Journal of Clinical Psychology, 63(9), 871–890. https://doi.org/10.1002/jclp.

    Article  PubMed  Google Scholar 

  • Coelho, H. F., Canter, P. H., & Ernst, E. (2007). Mindfulness-based cognitive therapy: Evaluating current evidence and informing future research. Journal of Consulting and Clinical Psychology, 75(6), 1000–1005. https://doi.org/10.1037/0022-006X.75.6.1000.

    Article  PubMed  Google Scholar 

  • Cooper, M., & McLeod, J. (2007). A pluralistic framework for counselling and psychotherapy: Implications for research. Counselling and Psychotherapy Research, 7(3), 135–143. https://doi.org/10.1080/14733140701566282.

    Article  Google Scholar 

  • Costello, E., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844.

    PubMed  Google Scholar 

  • Crits-christoph, P., Gibbons, M. B. C., & Mukherjee, D. (2013). Psychotherapy process-outcome research. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 298–340). Hoboken: Wiley.

    Google Scholar 

  • Danziger, K. (1985). The methodological imperative in psychology. Philosophy of the Social Sciences, 15(1), 1–13. https://doi.org/10.1177/004839318501500101.

    Article  Google Scholar 

  • Danziger, K. (1990). Deconstructing the subject: Historical origins of psychological research. New York: Cambridge University Press.

    Google Scholar 

  • Dunbar, K., & Fugelsang, J. (2005). Scientific thinking and reasoning. In K. J. Holyoak & R. Morrison (Eds.), Cambridge handbook of thinking (pp. 705–725). New York: Cambridge University Press.

    Google Scholar 

  • Dunlop, B. W., & Nemeroff, C. B. (2007). The role of dopamine in the pathophysiology of depression. Archives of General Psychiatry, 64(3), 327–337.

    PubMed  Google Scholar 

  • Elliott, R. (2010). Psychotherapy change process research: Realizing the promise. Psychotherapy Research, 20(2), 123–135. https://doi.org/10.1080/10503300903470743.

    Article  PubMed  Google Scholar 

  • Etkin, A., & Wager, T. D. (2007). Functional neuroimaging of anxiety: A meta-ana lysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. American Journal of Psychiatry, 164(10), 1476–1488. https://doi.org/10.1176/appi.ajp.2007.07030504.

    Article  PubMed  Google Scholar 

  • Feyerabend, P. K. (1981). Realism, rationalism and scientific method: Vol. 1. Philosophical papers. New York: Cambridge University Press.

    Google Scholar 

  • Flückiger, C., Grosse Holtforth, M., Znoj, H. J., Caspar, F., & Wampold, B. E. (2013). Is the relation between early post-session reports and treatment outcome an epiphenomenon of intake distress and early response? A multi-predictor analysis in outpatient psychotherapy. Psychotherapy Research, 23(1), 1–13. https://doi.org/10.1080/10503307.2012.693773.

    Article  PubMed  Google Scholar 

  • Frommer, J., & Rennie, D. L. (Eds.). (2001). Qualitative psychotherapy research: Methods and methodology. Lengerich: Pabst Science Publishers.

    Google Scholar 

  • Frommer, J., Langenbach, M., & Streeck, U. (2004). Qualitative psychotherapy research in German-speaking countries. Psychotherapy Research, 14(1), 57–75. https://doi.org/10.1093/ptr/kph004.

    Article  PubMed  Google Scholar 

  • Gadamer, H. G. (1989). Truth and method. New York: Continuum.

    Google Scholar 

  • Gao, Z. (2014). Methodologism/methodological imperative. In T. Teo (Ed.), Encyclopedia of Critical Psychology (pp. 1176–1179). https://doi.org/10.1007/978-1-4614-5583-7_614.

    Chapter  Google Scholar 

  • Gelo, O. C. G. (2012). On research methods and their philosophical assumptions: “Raising the consciousness of researchers” again. Psychotherapie & Sozialwissenschaft, 14(2), 111–130.

    Google Scholar 

  • Gelo, O. C. G., & Gallucci, M. (2016). Quantitative researchh. In H. L. Miller (Ed.), The SAGE encyclopedia of theory in psychology (pp. 780–783). Thousand Oaks: Sage.

    Google Scholar 

  • Gelo, O.C.G., & Gelo, C. (2012). Dialogical relationship among scientific paradigms in psychotherapy research. In S. for P. R. (SPR) (Ed.), 3rd Joint Meeting of the European and the United Kingdom Chapters of the Society for Psychotherapy Research (p. 46). Ulm, Germany: Ulmer Textbank.

  • Gelo, O. C. G., & Manzo, S. (2015). Quantitative approaches to treatment process, change process, and process-outcome research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 247–277). Vienna: Springer.

    Google Scholar 

  • Gelo, O., Braakmann, D., & Benetka, G. (2008). Quantitative and qualitative research: Beyond the debate. Integrative Psychological & Behavioral Science, 42(3), 266–290. https://doi.org/10.1007/s12124-008-9078-3.

    PubMed  Google Scholar 

  • Gelo, O., Braakmann, D., & Benetka, G. (2009). ERRATUM: Quantitative and qualitative research: Beyond the debate. Integrative Psychological and Behavioral Science, 43(4), 406–407. https://doi.org/10.1007/s12124-009-9107-x.

    Article  Google Scholar 

  • Gelo, O.C.G, Auletta, A. F., & Braakmann, D. (2010). Aspetti teorico-metodologici e analisi dei dati nella ricerca in psicoterapia . Parte I: La ricerca sull’esito e la ricerca sul proces processo – dagli anni ’ 50 agli anni ’ 80 [Theoretical-methodological issues and data analysis in psychotherapy research. Ricerca in Psicoterapia/Research in Psychotherapy, 13(1), 61–91.

  • Gelo, Omar C.G., Pritz, A., & Rieken, B. (Eds.). (2015). Psychotherapy Research: Foundations, Process and Outcome. https://doi.org/10.1007/978-3-7091-1382-0.

    Google Scholar 

  • Gelso, C. J., Betz, N. E., Friedlander, M. L., Helms, J. E., Hill, C. E., Patton, M. J., Super, D. E., & Wampold, B. E. (1988). Research in counseling psychology: Prospects and recommendations. The Counseling Psychologist, 16(3), 385–406. https://doi.org/10.1177/0011000088163006.

    Article  Google Scholar 

  • Gergen, K. J. (1994). Realities and relationships: Soundings in social construction. Cambridge: Harvard University Press.

    Google Scholar 

  • Gigerenzer, G. (2010). Personal reflections on theory and psychology. Theory & Psychology, 20(6), 733–743. https://doi.org/10.1177/0959354310378184.

    Article  Google Scholar 

  • Gold, C. (2015). Quantitative psychotherapy outcome research: Methodological issues. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 537–558). Vienna: Springer.

    Google Scholar 

  • Goldfried, M. R., & Wolfe, B. E. (1998). Toward a more clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology, 66(1), 143–150. https://doi.org/10.1037//0022-006X.66.1.143.

    Article  PubMed  Google Scholar 

  • Gone, J. P. (2009). A community-based treatment for native American historical trauma: Prospects for evidence-based practice. Journal of Consulting and Clinical Psychology, 77(4), 751–762. https://doi.org/10.1037/a0015390.

    Article  PubMed  Google Scholar 

  • Gottesman, I. I., & Gould, T. D. (2003). The endophenotype concept in psychiatry: Etymology and strategic intentions. American Journal of Psychiatry, 160(4), 636–645. https://doi.org/10.1176/appi.ajp.160.4.636.

    Article  PubMed  Google Scholar 

  • Grant, B. F., Stinson, F. S., Dawson, D. A., Chou, P., Dufour, M. C., Compton, W., Pickering, R. P., & Kaplan, K. (2004). Prevalence and co-occurrence of substance use disorders and independentmood and anxiety disorders: Results from the national epidemiologic survey on alcohol and relatedconditions. Archives of General Psychiatry, 61(8), 807–816. https://doi.org/10.1001/archpsyc.61.8.807.

    Article  PubMed  Google Scholar 

  • Green, D., & Latchford, G. (2012). Maximising the benefits of psychotherapy: A practice-based evidence approach. https://doi.org/10.1002/9781119967590.

    Google Scholar 

  • Grol, R., & Grimshaw, J. (2003). From best evidence to best practice: Effective implementation of change in patients’ care. The Lancet, 362(9391), 1225–1230. https://doi.org/10.1016/S0140-6736(03)14546-1.

    Article  Google Scholar 

  • Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105–117). Thousand Oaks: Sage.

    Google Scholar 

  • Hanson, W. E., Creswell, J. W., Clark, V. L. P., Petska, K. S., & Creswell, J. D. (2005). Mixed methods research designs in counseling psychology. Journal of Counseling Psychology, 52(2), 224–235. https://doi.org/10.1037/0022-0167.52.2.224.

    Article  Google Scholar 

  • Harris, K., Collinson, C., & das Nair, R. (2012). Service-users’ experiences of an early intervention in psychosis service: An interpretative phenomenological analysis. Psychology and Psychotherapy: Theory, Research and Practice, 85(4), 456–469. https://doi.org/10.1111/j.2044-8341.2011.02043.x.

    Article  Google Scholar 

  • Hasin, D. S., Stinson, F. S., Ogburn, E., & Grant, B. F. (2007). Prevalence, correlates, disability, and comorbidity of DSM-IV alcohol abuse and dependence in the United States: Results from the national epidemiologic survey on alcohol and related conditions. Archives of General Psychiatry, 64(7), 830–842. https://doi.org/10.1001/archpsyc.64.7.830.

    Article  PubMed  Google Scholar 

  • Hill, C. E., & Lambert, M. J. (2004). Methodological issues in studying psychotherapy process and outcomes. In Bergin and Garfield’s handbook of sychotherapy and behavior change (5th ed., pp. 84–135). New York: Wiley.

    Google Scholar 

  • Hill, C. E., Nutt, E. A., & Jackson, S. (1994). Trends in psychotherapy process research: Samples, measures, researchers, and classic publications. Journal of Counseling Psychology, 41(3), 364–377. https://doi.org/10.1037/0022-0167.41.3.364.

    Article  Google Scholar 

  • Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., Hess, S. A., & Ladany, N. (2005). Consensual qualitative research: An update. Journal of Counseling Psychology, 52(2), 196–205. https://doi.org/10.1037/0022-0167.52.2.196.

    Article  Google Scholar 

  • Hofmann, S., Sawyer, A., Witt, A., & Oh, D. (2010). The effect of mindfulness- based therapy on anxiety and depression: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 169–183. https://doi.org/10.1037/a0018555.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hostetler, K. (1994). Community and neutrality in critical thought: A nonobjectivist view on the conduct and teaching of critical thinking. In K. S. Walters (Ed.), Re-thinking reason: New perspectives in critical thinking (pp. 135–154). Albany: State University of New York Press.

    Google Scholar 

  • Jasso, G. (1988). Principles of theoretical analysis. Sociological Theory, 6(1), 1. https://doi.org/10.2307/201910.

    Article  Google Scholar 

  • Joiner, T. E., Sheldon, K. M., Williams, G., & Pettit, J. (2006). The integration of self-determination principles and scientifically informed treatments is the next tier. Clinical Psychology: Science and Practice, 10(3), 318–319. https://doi.org/10.1093/clipsy.bpg028.

    Article  Google Scholar 

  • Kalivas, P. W., & Volkow, N. D. (2005). The neural basis of addiction:A pathology of motivation and choice. American Journal of Psychiatry, 162(8), 1403–1413. https://doi.org/10.1176/appi.ajp.162.8.1403.

    PubMed  Google Scholar 

  • Kapur, S. (2003). Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia. American Journal of Psychiatry, 160(1), 13–23. https://doi.org/10.1176/appi.ajp.160.1.13.

    Article  PubMed  Google Scholar 

  • Karg, K., Burmeister, M., Shedden, K., & Sen, S. (2011). The serotonin transporter promoter variant (5-HTTLPR), stress, and depression meta-analysis revisited: Evidence of genetic moderation. Archives of General Psychiatry, 68(5), 444–454. https://doi.org/10.1001/archgenpsychiatry.2010.189.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kazdin, A. E. (2009). Understanding how and why psychotherapy leads to change. Psychotherapy Research, 19, 418–428. https://doi.org/10.1080/10503300802448899.

    Article  PubMed  Google Scholar 

  • Kellert, S. H., Longino, H. E., & Waters, C. K. (Eds.). (2006). Scientific pluralism. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Kendell, R., & Jablensky, A. (2003). Distinguishing between the validity and utility of psychiatric diagnoses. American Journal of Psychiatry, 160(1), 4–12. https://doi.org/10.1176/appi.ajp.160.1.4.

    Article  PubMed  Google Scholar 

  • Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005a). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62, 593–602. https://doi.org/10.1001/archpsyc.62.6.593.

    Article  PubMed  Google Scholar 

  • Kessler, R. C., Wai, T. C., Demler, O., & Walters, E. E. (2005b). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 617–627. https://doi.org/10.1001/archpsyc.62.6.617.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kessler, R. C., Adler, L., Barkley, R., Biederman, J., Conners, C. K., Demler, O., Faraone, S. V., Greenhill, L. L., Howes, M. J., Secnik, K., Spencer, T., Ustun, T. B., Walters, E. E., & Zaslavsky, A. M. (2006). The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. American Journal of Psychiatry, 163(4), 716–723. https://doi.org/10.1176/appi.ajp.163.4.716.

    Article  PubMed  Google Scholar 

  • Kukla, A. (1982). Logical incoherence of value-free science. Journal of Personality and Social Psychology, 43(5), 1014–1017. https://doi.org/10.1037/0022-3514.43.5.1014.

    Article  Google Scholar 

  • Kukla, A. (1989). Nonempirical issues in psychology. American Psychologist, 44(5), 785–794. https://doi.org/10.1037/0003-066X.44.5.785.

    Article  Google Scholar 

  • Kukla, A. (2001). Methods of theoretical psychology. Cambridge: Massachusetts Institute of Technology.

    Google Scholar 

  • Lakatos, I. (1978). The methodology of scientific research programs. Cambridge: Cambridge University Press.

    Google Scholar 

  • Lambert, M. J. (Ed.). (2004). Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed.). New York: Wiley.

    Google Scholar 

  • Lambert, M. J. (Ed.). (2013a). Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed.). New York: Wiley.

    Google Scholar 

  • Lambert, M. J. (2013b). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 169–218). Hoboken: Wiley.

    Google Scholar 

  • Lambert, M. J. (2015). Outcome research: Methods for improving outcome in routine care. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 593–610). Vienna: Springer.

    Google Scholar 

  • Lanius, R. A., Vermetten, E., Loewenstein, R. J., Brand, B., Christian, S., Bremner, J. D., & Spiegel, D. (2010). Emotion modulation in PTSD: Clinical and neurobiological evidence for a dissociative subtype. American Journal of Psychiatry, 167(6), 640–647. https://doi.org/10.1176/appi.ajp.2009.09081168.

    Article  PubMed  Google Scholar 

  • Laudan, L. (1977). Progress and its progress. Berkeley: University of California Press.

    Google Scholar 

  • Laudan, L. (1984). Science and values. Berkeley, CA: University of California Press.

  • Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2013). Why many clinical psychologists are resistant to evidence-based practice: Root causes and constructive remedies. Clinical Psychology Review, 33(7), 883–900. https://doi.org/10.1016/j.cpr.2012.09.008.

    Article  PubMed  Google Scholar 

  • Luppino, F. S., De Wit, L. M., Bouvy, P. F., Stijnen, T., Cuijpers, P., Penninx, B. W. J. H., & Zitman, F. G. (2010). Overweight, obesity, and depression: A systematic review and meta-analysis of longitudinal studies. Archives of General Psychiatry, 67, 220–229. https://doi.org/10.1001/archgenpsychiatry.2010.2.

    Article  PubMed  Google Scholar 

  • Lutz, W., Ehrlich, T., Rubel, J., Hallwachs, N., Röttger, M.-A., Jorasz, C., Mocanu, S., Vocks, S., Schulte, D., & Tschitsaz-Stucki, A. (2013). The ups and downs of psychotherapy: Sudden gains and sudden losses identified with session reports. Psychotherapy Research, 23(1), 14–24. https://doi.org/10.1080/10503307.2012.693837.

    Article  PubMed  Google Scholar 

  • Marder, S. R., Essock, S. M., Miller, A. L., Buchanan, R. W., Casey, D. E., Davis, J. M., Kane, J. M., Lieberman, J. A., Schooler, N. R., Covell, N., Stroup, S., Weissman, E. M., Wirshing, D. A., Hall, C. S., Pogach, L., Pi-Sunyer, X., Bigger, J. T., Jr., Friedman, A., Kleinberg, D., Yevich, S. J., Davis, B., & Shon, S. (2004). Physical health monitoring of patients with schizophrenia. The American Journal of Psychiatry, 161(8), 1334–1349. https://doi.org/10.1176/appi.ajp.161.8.1334.

    Article  PubMed  Google Scholar 

  • McLeod, J. (2011). Qualitative research in counselling and psychotherapy (2nd ed.). London: Sage.

    Google Scholar 

  • Merikangas, K. R., Akiskal, H. S., Angst, J., Greenberg, P. E., Hirschfeld, R. M., Petukhova, M., & Kessler, R. C. (2007). Lifetime and 12-month prevalence of bipolar Spectrum disorder in the National Comorbidity Survey Replication. Archives of General Psychiatry, 64(5), 543–552.

    PubMed  PubMed Central  Google Scholar 

  • Miklowitz, D. J. (2008). Adjunctive psychotherapy for bipolar disorder: State of the evidence. American Journal of Psychiatry, 165, 1408–1419. https://doi.org/10.1176/appi.ajp.2008.08040488.

    Article  PubMed  Google Scholar 

  • Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

    PubMed  PubMed Central  Google Scholar 

  • Mörtl, K., & Gelo, O. C. G. (2015). Qualitative methods in psychotherapy process research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 381–428). Vienna: Springer.

    Google Scholar 

  • Najavits, L. M., & Hien, D. (2013). Helping vulnerable populations: A comprehensive review of the treatment outcome literature on substance use disorder and PTSD. Journal of Clinical Psychology, 69(5), 433–479. https://doi.org/10.1002/jclp.21980.

    Article  PubMed  Google Scholar 

  • Oddli, H. W., & Rønnestad, M. H. (2012). How experienced therapists introduce the technical aspects in the initial alliance formation: Powerful decision makers supporting clients’ agency. Psychotherapy Research, 22(2), 176–193. https://doi.org/10.1080/10503307.2011.633280.

    Article  PubMed  Google Scholar 

  • Orlinsky, D. R., & Russell, R. L. (1994). Tradition and change in psychotherapy research. Notes on the fourth generation. In R. L. Russell (Ed.), Reassessing psychotherapy research (pp. 185–214). New York: Guilford Press.

    Google Scholar 

  • Orlinsky, D. E., Grawe, K., & Parks, B. K. (1994). Process and outcome in psychotherapy – Noch Einmal. In M. J. Lambert (Ed.), Handbook of psychotherapy and behavior change (4th ed., pp. 270–376). New York: NY.

    Google Scholar 

  • Pacchiarotti, I., Bond, D. J., Baldessarini, R. J., Nolen, W. A., Grunze, H., Licht, R. W., Post, R. M., Berk, M., Goodwin, G. M., Sachs, G. S., Tondo, L., Findling, R. L., Youngstrom, E. A., Tohen, M., Undurraga, J., González-Pinto, A., Goldberg, J. F., Yildiz, A., Altshuler, L. L., Calabrese, J. R., Mitchell, P. B., Thase, M. E., Koukopoulos, A., Colom, F., Frye, M. A., Malhi, G. S., Fountoulakis, K. N., Vázquez, G., Perlis, R. H., Ketter, T. A., Cassidy, F., Akiskal, H., Azorin, J. M., Valentí, M., Mazzei, D. H., Lafer, B., Kato, T., Mazzarini, L., Martínez-Aran, A., Parker, G., Souery, D., Özerdem, A., McElroy, S. L., Girardi, P., Bauer, M., Yatham, L. N., Zarate, C. A., Nierenberg, A. A., Birmaher, B., Kanba, S., el-Mallakh, R. S., Serretti, A., Rihmer, Z., Young, A. H., Kotzalidis, G. D., MacQueen, G. M., Bowden, C. L., Ghaemi, S. N., Lopez-Jaramillo, C., Rybakowski, J., Ha, K., Perugi, G., Kasper, S., Amsterdam, J. D., Hirschfeld, R. M., Kapczinski, F., & Vieta, E. (2013). The international society for bipolar disorders (ISBD) task force report on antidepressant use in bipolar disorders. American Journal of Psychiatry, 170(11), 1249–1262. https://doi.org/10.1176/appi.ajp.2013.13020185.

    Article  PubMed  Google Scholar 

  • Petrowski, K., Pokorny, D., Nowacki, K., & Buchheim, A. (2013). The therapist’s attachment representation and the patient’s attachment to the therapist. Psychotherapy Research, 23(1), 25–34. https://doi.org/10.1080/10503307.2012.717307.

    Article  PubMed  Google Scholar 

  • Phillips, J. C., Ingram, K. M., Smith, N. G., & Mindes, E. J. (2003). Methodological and content review of lesbian-, gay-, and bisexual-related articles in counseling journals: 1990-1999. The Counseling Psychologist, 31(1), 25–62. https://doi.org/10.1177/0011000002239398.

    Article  Google Scholar 

  • Phillips, M. L., Travis, M. J., Fagiolini, A., & Kupfer, D. J. (2008). Medication effects in neuroimaging studies of bipolar disorder. American Journal of Psychiatry, 165(3), 313–320. https://doi.org/10.1176/appi.ajp.2007.07071066.

    Article  PubMed  Google Scholar 

  • Pokorny, D. (2015). Quantitative data analysis in psychotherapy process research: Structures and procedures. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process and outcome (pp. 229–245). Vienna: Springer.

    Google Scholar 

  • Polanczyk, G., De Lima, M. S., Horta, B. L., Biederman, J., & Rohde, L. A. (2007). The worldwide prevalence of ADHD: A systematic review and metaregression analysis. American Journal of Psychiatry, 164(6), 942–948. https://doi.org/10.1176/appi.ajp.164.6.942.

    Article  PubMed  Google Scholar 

  • Polkinghorne, D. E. (1983). Methodology for the human sciences: Systems of inquiry. Albany: State University of New York Press.

    Google Scholar 

  • Polkinghorne, D. E. (1991). Two conflicting calls for methodological reform. The Counseling Psychologist, 19(1), 103–114. https://doi.org/10.1177/0011000091191008.

    Article  Google Scholar 

  • Polkinghorne, D. E. (2005). Language and meaning: Data collection in qualitative research. Journal of Counseling Psychology, 52(2), 137–145. https://doi.org/10.1037/0022-0167.52.2.137.

    Article  Google Scholar 

  • Ponterotto, J. G. (2005a). Integrating qualitative research requirements into professional psychology training programs in North America: Rationale and curriculum model. Qualitative Research in Psychology, 2(2), 97–116. https://doi.org/10.1191/1478088705qp035oa.

    Article  Google Scholar 

  • Ponterotto, J. G. (2005b). Qualitative research in counseling psychology: A primer on research paradigms and philosophy of science. Journal of Counseling Psychology, 52(2), 126–136. https://doi.org/10.1037/0022-0167.52.2.126.

    Article  Google Scholar 

  • Raison, C. L., & Miller, A. H. (2003). When not enough is too much: The role of insufficient glucocorticoid signaling in the pathophysiology of stress-related disorders. American Journal of Psychiatry, 160(9), 1554–1565.

    PubMed  Google Scholar 

  • Raison, C. L., Lowry, C. A., & Rook, G. A. W. (2010). Inflammation, sanitation, and consternation: Loss of contact with coevolved, tolerogenic microorganisms and the pathophysiology and treatment of major depression. Archives of General Psychiatry, 67(12), 1211–1224. https://doi.org/10.1001/archgenpsychiatry.2010.161.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rennie, D. L. (2004). Anglo-north American qualitative counseling and psychotherapy research. Psychotherapy Research, 14(1), 37–55. https://doi.org/10.1093/ptr/kph003.

    Article  PubMed  Google Scholar 

  • Rennie, D. L. (2012). Qualitative research as methodical hermeneutics. Psychological Methods, 17(3), 385–398. https://doi.org/10.1037/a0029250.

    Article  PubMed  Google Scholar 

  • Rennie, D. L., & Frommer, J. (2015). Applications of qualitative and mixed-methods counseling and psychotherapy research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 429–454). Vienna: Springer.

    Google Scholar 

  • Richardson, L. K., Frueh, B. C., Grubaugh, A. L., Egede, L., & Elhai, J. D. (2009). Current directions in videoconferencing tele-mental Health Research. Clinical Psychology: Science and Practice, 16(3), 323–338. https://doi.org/10.1111/j.1468-2850.2009.01170.x.

    Article  Google Scholar 

  • Rieken, B., & Gelo, O. C. G. (2015). The philosophy of psychotherapy science: Mainstream and alternative views. Psychotherapy Research: Foundations, process, and outcome, 67–92. Vienna: Springer.

  • Rodgers, B., & Elliott, R. (2015). Qualitative methods in psychotherapy outcome research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 559–578). Vienna: Springer.

    Google Scholar 

  • Roth, A., & Fonagy, P. (2005). What works for whom? A critical review of psychotherapy research (2nd ed.). New York: Guilford Press.

    Google Scholar 

  • Rush, A. J., Trivedi, M. H., Wisniewski, S. R., Nierenberg, A. A., Stewart, J. W., Warden, D., et al. (2006). Acute and longer- term outcomes in depressed outpatients requiring one or several treatment steps: A STAR * D report. American Journal of Psychiatry, 163(11), 1905–1917.

    PubMed  Google Scholar 

  • Salvatore, S. (2011). Psychotherapy research needs theory: Outline for an epistemology of the clinical exchange. Integrative Psychological & Behavioral Science, 45(3), 366–388. https://doi.org/10.1007/s12124-011-9180-9.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental for generalized designs causal inference. Boston: Houghton-Mifflin.

    Google Scholar 

  • Shapiro, S. L., Carlson, L. E., Astin, J. A., & Freedman, B. (2006). Mechanisms of mindfulness. Journal of Clinical Psychology, 62(3), 373–386. https://doi.org/10.1002/jclp.

    Article  PubMed  Google Scholar 

  • Shelton, K., & Delgado-Romero, E. A. (2011). Sexual orientation microaggressions: The experience of lesbian, gay, bisexual, and queer clients in psychotherapy. Journal of Counseling Psychology, 58(2), 210–221. https://doi.org/10.1037/a0022251.

    Article  PubMed  Google Scholar 

  • Singh, A. A., & Shelton, K. (2011). A content analysis of LGBTQ qualitative research in counseling: A ten-year review. Journal of Counseling & Development, 89(2), 217–226. https://doi.org/10.1002/j.1556-6678.2011.tb00080.x.

    Article  Google Scholar 

  • Slife, B. D. (1998). Raising the consciousness of researchers: Hidden assumptions in the behavioral sciences. Adapted Physical Activity Quarterly, 15(3), 208–221. https://doi.org/10.1123/apaq.15.3.208.

    Article  Google Scholar 

  • Slife, B. D. (2004). Theoretical challenges to therapy practice and research: The constraint of naturalism. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behaviour change (5th ed.). New York: Wiley.

    Google Scholar 

  • Slife, B. D., & Gantt, E. E. (1999). Methodological pluralism: A framework for psychotherapy research. Journal of Clinical Psychology, 55(12), 1453–1465. https://doi.org/10.1002/(SICI)1097-4679(199912)55:12<1453::AID-JCLP4>3.0.CO;2-C.

    Article  PubMed  Google Scholar 

  • Slife, B. D., & Williams, R. N. (1995). What’s behind the research: Discovering the hidden assumptions in the behavioral sciences. Thousand Oaks: Sage.

    Google Scholar 

  • Slife, B. D., & Williams, R. N. (1997). Toward a theoretical psychology: Should a subdiscipline be formally recognized? American Psychologist, 52(2), 117–129. https://doi.org/10.1037//0003-066X.52.2.117.

    Article  Google Scholar 

  • Slife, B. D., Wiggins, B. J., & Graham, J. T. (2005). Avoiding an EST monopoly: Toward a pluralism of philosophies and methods. Journal of Contemporary Psychotherapy, 35(1), 83–97. https://doi.org/10.1007/s10879-005-0805-5.

    Article  Google Scholar 

  • Smaling, A. (2000). What kind of dialogue should paradigm-dialogues be? Quality and Quantity, 34(1), 51–63. https://doi.org/10.1023/A:1004747524463.

    Article  Google Scholar 

  • Stegenga, J. (2014). Down with the hierarchies. Topoi, 33(2), 313–322. https://doi.org/10.1007/s11245-013-9189-4.

    Article  Google Scholar 

  • Stein, M. B., & McAllister, T. W. (2009). Exploring the convergence of posttraumatic stress disorder and mild traumatic brain injury. The American Journal of Psychiatry, 166(7), 768–776. https://doi.org/10.1176/appi.ajp.2009.08101604.

    Article  PubMed  Google Scholar 

  • Stiles, W. B. (2015). Theory building, enriching, and fact gathering: Alternative purposes of psychotherapy research. In O. C. G. Gelo, A. Pritz, & B. Rieken (Eds.), Psychotherapy research: Foundations, process, and outcome (pp. 159–179). Vienna: Springer.

    Google Scholar 

  • Suppe, F. (1977). The structure of scientific theories. Urbana: University of Illinois Press.

    Google Scholar 

  • Timulak, L. (2009). Meta-analysis of qualitative studies: A tool for reviewing qualitative research findings in psychotherapy. Psychotherapy Research, 19(4–5), 591–600. https://doi.org/10.1080/10503300802477989.

    Article  PubMed  Google Scholar 

  • Trivedi, M. H., Rush, A. J., Wisniewski, S. R., Nierenberg, A. A., Warden, D., Ritz, L., et al. (2006). Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: Implications for clinical practice. American Journal of Psychiatry, 163(1), 28–40. https://doi.org/10.1176/appi.ajp.163.1.28.

    Article  PubMed  Google Scholar 

  • Weisz, J. R., Doss, A. J., & Hawley, K. M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology, 56(1), 337–363. https://doi.org/10.1146/annurev.psych.55.090902.141449.

    Article  PubMed  Google Scholar 

  • Westerman, M. A. (2006). Quantitative research as an interpretive enterprise: The mostly unacknowledged role of interpretation in research efforts and suggestions for explicitly interpretive quantitative investigations. New Ideas in Psychology, 24(3), 189–211. https://doi.org/10.1016/j.newideapsych.2006.09.004.

    Article  Google Scholar 

  • Wood, R. T. A., & Griffiths, M. D. (2007). A qualitative investigation of problem gambling as an escape-based coping strategy. Psychology and Psychotherapy: Theory, Research and Practice, 80(1), 107–125. https://doi.org/10.1348/147608306X107881.

    Article  Google Scholar 

  • Yanchar, S. C., & Slife, B. D. (2004). Teaching critical thinking by examining assumptions. Teaching of Psychology, 31(2), 85–90. https://doi.org/10.1207/s15328023top3102_2.

    Article  Google Scholar 

  • Yanchar, S. C., Slife, B. D., & Warne, R. (2008). Critical thinking as disciplinary practice. Review of General Psychology, 12(3), 265–281. https://doi.org/10.1037/1089-2680.12.3.265.

    Article  Google Scholar 

  • Zarate, C. A., Singh, J. B., Carlson, P. J., Brutsche, N. E., Ameli, R., Luckenbaugh, D. A., Charney, D. S., & Manji, H. K. (2006). A randomized trial of an N-methyl-D-aspartate antagonist in treatment-resistant major depression. Archives of General Psychiatry, 63(8), 856–864. https://doi.org/10.1001/archpsyc.63.8.856.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank the following SPR past presidents who replied to our survey: Lynne Angus, Lorna Smith Benjamin, Larry E. Beutler, Louis G. Castonguay, John Clarkin, Paul Crits-Christoph, Guillermo de la Parra, Irene Elkin, Robert Elliot, Marvin R. Goldfried, Leslie S. Greenberg, Alan S. Gurman, Clara E. Hill, Mardi J. Horowitz, Horst Kächele, Michael J. Lambert, Erhard Mergenthaler, David E. Orlinksy, William E. Piper, George Silberschatz, William B. Stiles, Bernhard Strauß, Jeanne Watson, Hadas Wiseman.

Author information

Authors and Affiliations

  1. Department of History, Social Sciences and Human Studies, Via di Valesio – Complesso Studium 2000, Ed. 5, 73100, Lecce, Italy

    Omar Carlo Gioacchino Gelo, Gloria Lagetto, Chiara Dinoi & Enrico Ciavolino

  2. Faculty of Psychotherapy Science, Sigmund Freud University Vienna, Wien, Austria

    Omar Carlo Gioacchino Gelo

  3. Department of Humanistic Studies, University of Urbino Carlo Bo, Urbino, Italy

    Erika Belfiore, Elisa Lombi & Stefano Blasi

  4. Department of Economics and Statistics, University of Naples Federico II, Naples, Italy

    Massimo Aria

Authors
  1. Omar Carlo Gioacchino Gelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gloria Lagetto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chiara Dinoi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erika Belfiore
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elisa Lombi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stefano Blasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Massimo Aria
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Enrico Ciavolino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omar Carlo Gioacchino Gelo.

Ethics declarations

Conflict of Interest

Omar C. G. Gelo declares that he has no conflict of interest. Gloria Lagetto declares that she has no conflict of interest. Chiara Dinoi declares that she has no conflict of interest. Erika Belfiore declares that she has no conflict of interest. Elisa Lombi declares that she has no conflict of interest. Stefano Blasi declares that she has no conflict of interest. Massimo Aria declares that she has no conflict of interest. Enrico Ciavolino declares that she has no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gelo, O.C.G., Lagetto, G., Dinoi, C. et al. Which Methodological Practice(s) for Psychotherapy Science? A Systematic Review and a Proposal. Integr. psych. behav. 54, 215–248 (2020). https://doi.org/10.1007/s12124-019-09494-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12124-019-09494-3

Keywords

  • Psychotherapy science
  • Methodological practice
  • Research methods
  • Scientific method
  • Methodological pluralism
  • Scientific pluralism
  • Scientific monism
  • Quantitative monopoly