Abstract
There is a critical need to identify strategies for financing the implementation of evidence-based practices. We illustrate the potential of pay-for-success financing (PFS)—a strategy in which private investors fund implementation and receive a return on investment from a government payer—using multisystemic therapy as an example. We argue that standard multisystemic therapy (for serious juvenile offenders) and several of its adaptations (for other complex behavioral problems in youth) would be good candidates for PFS in the right contexts. Despite some challenges for policymakers and administrators, PFS has significant potential as a financing strategy for evidence-based practices.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
There is a critical need to increase availability of evidence-based practices in community mental health systems (Beidas and Powell 2016; Powell et al. 2015). Significant progress has been made in defining criteria for high-quality evidence (e.g., systematically collected data, methods that establish causality, replicable findings) and identifying practices for which such evidence is available (see e.g., Blueprints for Healthy Youth Development, California Evidence-Based Clearinghouse), yet identification does not guarantee implementation. Among numerous remaining challenges, cost is a critical barrier to implementation of evidence-based practices in these systems. Well-specified and feasible financing strategies are therefore needed to support implementation efforts (Bond et al. 2014; Proctor et al. 2011; Roundfield and Lang 2017).
One innovative implementation financing strategy is pay-for-success (PFS), also known as social impact bonds. Under a PFS contract, private-sector investors fund the implementation of an intervention targeting an important social welfare problem. If the intervention provides value to the public sector, as determined by an independent evaluation, the investors will receive a payout from the government (Segal et al. 2016; Social Finance, Inc. 2012). Since the first PFS contract was launched in 2010, this private-public partnership strategy has been applied to an increasingly diverse array of interventions that promote social welfare, including supportive housing, early childhood education, and nurse home-visiting for pregnant women. The United States currently represents approximately 20% of the estimated 108 PFS contracts globally and 50% of the total funds invested (Iovan et al. 2018; Social Finance, Inc. 2018).
A recent review of PFS contracts concluded that the approach has considerable potential to impact population health, but also emphasized that PFS contracts should finance implementation of interventions with a strong evidence base (Lantz et al. 2016). Interventions without a strong evidence base may not produce interim or longer-term results and there will not be a public sector payout to the investors. To address this and other challenges, Lantz and Iovan proposed seven criteria—listed in Table 1—that can be used to identify interventions that are appropriate for PFS financing (Lantz and Iovan 2017). Of the many possible approaches that could be taken, these criteria were most appropriate for our research given that they provide a generalizable evaluation of a practice’s suitability for PFS, while still acknowledging administrative, stakeholder, and political considerations. They are also consistent with complimentary sets of criteria (e.g., Urban Institute 2016) that are more suitable for subsequent, context-specific assessment of the feasibility of a particular PFS project, using a particular evidence-based practice, in a specific governmental and population context.
The purpose of this brief is to illustrate the potential of PFS as a financing strategy for evidence-based practices in mental health by applying Lantz and Iovan’s (2017) criteria to a group of related interventions based on the multisystemic therapy (MST) model (Henggeler et al. 2009). MST was initially developed as an intervention for serious antisocial behavior in youth. It has well-established clinical effectiveness and economic benefits in that population (Dopp et al. 2017; McCart and Sheidow 2016) and has been widely disseminated by a purveyor organization, MST Services. Furthermore, the MST model has been adapted to address other complex behavioral problems in youth and families (MST Services 2017), with varied levels of evidence available for each adaptation. However, MST is challenging for state agencies to implement due to its complexity and costs (Dopp et al. 2018a), especially because many costs accrue up front (e.g., site assessments, initial training, administrative changes) before the intervention starts affecting outcomes. Traditional government budgets have often prioritized remediation of the highest-risk individuals in a population (even when it relies on expensive interventions such as secure confinement) over preventive or rehabilitative approaches like MST (National Research Council 2013; see also Iovan et al. 2018). The combination of high initial costs and high potential returns on investment has generated considerable interest in PFS contracts as an alternative funding mechanism for MST (Roberts and Cameron 2014; University of Denver 2018).
We applied the first six PFS intervention selection criteria to the full range of MST adaptations, to illustrate the process of evaluating and comparing among interventions with varied evidence bases. We did not examine Criterion #7 (no significant political or stakeholder challenges) in our analysis because it is context-dependent; however, we discuss later how that criterion could be applied to our findings. We based our conclusions about the remaining six criteria on information from a number of sources, including an annual report by MST Services (2018b) that summarizes all published research on MST; relevant studies that were not included in that report because they were not published [e.g., results from the Washington State Institute of Public Policy (WSIPP 2017) cost-benefit model] or published too late; and lists of licensed teams for MST and its variants (MST Services 2018a). We specifically used the lists of licensed teams to evaluate Criterion #6, i.e., the ability of provider organizations to implement MST and its adaptations without significant administrative challenges. Given that MST licensure involves continuous participation in a comprehensive quality assurance/improvement system (e.g., training, expert consultation, outcome monitoring) designed to promote successful implementation, we considered the presence of licensed teams a key indicator for that criterion.
Finally, when making determinations about whether the PFS intervention criteria were met for a given MST adaptation, we incorporated a recently proposed framework for “scaling out” interventions (Aarons et al. 2017). Drawing on well-established external validity theories (e.g., Campbell 1957) and multilevel mediation modeling, those scholars describe how evidence-based practices can “borrow strength” by considering evidence from previous effectiveness trials alongside new evidence for the practice’s effectiveness with a different population and/or delivery system—but only to the extent that core intervention processes are maintained during scale-out. Consistent with that framework, we considered MST adaptations to borrow moderate strength from standard MST when they involved changes in population (Type I Scale-Out) or delivery system (Type II Scale-Out) only, but borrow minimal strength when they involved changes in both population and delivery system (Type III Scale-Out).
Our conclusions regarding how well MST and its adaptations meet key PFS intervention criteria are described below and outlined in Table 2. We found that standard MST and three of its adaptations [MST for serious conduct problems, problematic sexual behaviors (MST-PSB), and child abuse and neglect (MST-CAN)] have significant potential for PFS financing. Seven additional MST adaptations failed to meet one or more of the criteria considered.
1. MST Addresses Problems of Interest to the Public Sector
MST was designed to address serious, complex problems related to youth behavioral health. In recent years, the nascent field of PFS financing has moved beyond its early focus on “cost savings” to increasingly emphasize practices that provide benefits to recipients, taxpayers, and society at large (i.e., high-value practices; Iovan et al. 2018). Thus, investors and government entities that are dedicated to increasing community well-being and addressing widespread societal problems (e.g., criminality, child maltreatment) would likely view MST and its adaptations as targeting problems that are worth paying to address.
2. MST and Some Adaptations have a Strong Evidence Base
MST adaptations vary widely in terms of their stage of development on the continuum from pilot studies to large-scale transport (see MST Services 2017) and, thus, there is also considerable variability in the amount and quality of studies examining each adaptation. Both standard MST and MST for serious conduct problems clearly meet Criterion #2, with a total of 23 trials between them. This represents a considerable body of evidence from which other adaptations can “borrow strength.” MST-PSB, MST-CAN, and MST Health Care all had a much smaller number of published studies, but still met Criterion #2 because those studies had generally favorable results and/or borrowed strength. The remaining MST adaptations either lacked sufficient research evidence or the existing research evidence was not sufficiently favorable.
3. MST and Some Adaptations are Economically Attractive to the Public Sector
Far fewer studies have examined the economic impact (i.e., savings or value produced) of MST and its adaptations, even in cases where the evidence base for clinical outcomes is robust. Standard MST and MST for serious conduct problems again had the most evidence regarding economic impact; eight of these studies showed economic benefits of MST (e.g., returns of up to $5.04 per dollar spent; Dopp et al. 2014) and the remaining two that were unfavorable were limited in scope. In keeping with the emphasis on value over budgetary cost savings, most of the benefits produced by MST accrued through avoided crime victim expenses, particularly intangible benefits (i.e., reduced pain and suffering), though a significant minority of benefits were taxpayer savings in the juvenile/criminal justice and Medicaid behavioral health sectors (see e.g., Dopp et al. 2014; Dopp et al. 2018a). It is also notable that MST generated most of these benefits within 2 years of intervention delivery.
The other adaptations that met this criterion were MST-PSB and MST-CAN, and both of those were based on the results of one or two cost-benefit analyses (with the analysis for MST-CAN having just been published this past year; Dopp et al. 2018b) but again could borrow strength. For the other adaptations with evidence for economic impact, that evidence was either too limited to meet this criterion (i.e., only examined a narrow range of cost offsets) or was difficult to interpret because of limitations in the overall evidence base for clinical effectiveness of that adaptation.
4. Key MST Clinical Outcomes are Clearly Defined and Quantifiable; and 5. Those Outcomes are Attained in a Reasonable Time Period
The research base for MST exhibited a number of strengths with respect to these two criteria. With the exception of MST-Prevention, every MST adaptation had expected clinical outcomes that were: for Criterion #4, well-defined (e.g., recidivism, out-of-home placement); and for Criterion #5, reasonable in their timeframes (i.e., can be achieved within 1–2 years of beginning intervention).
6. MST and Some Adaptations can be Implemented Without Significant Administrative Challenges
Most licensed MST teams are for MST with serious juvenile offenders/conduct problems (MST Services 2018a does not distinguish between these teams), MST-PSB, or MST-CAN. There are also a limited number of licensed teams for MST-Psychiatric and MST-Family Integrated Transitions (MST Services 2018a). Thus, we considered standard MST and those five adaptations capable of being implemented without significant administrative challenges.
In conclusion, we argue that standard MST for serious juvenile offenders and three adaptations (MST for serious conduct problems, MST-PSB, and MST-CAN) meet the six selection criteria we evaluated, and thus are strong candidates for PFS financing, a novel public-private financing approach to social welfare interventions. Our analysis illustrates the potential of PFS financing to help governments implement MST, a complex intervention that has considerable up-front costs yet also provides cost savings and value (i.e. the behavioral or mental health outcomes that are worth the costs) within a reasonable time period.
One limitation of our analysis is that governments will also need to consider Criterion #7, “No significant stakeholder or political opposition,” before investing in the implementation of MST through a PFS contract. It was not possible for us to globally evaluate this criterion, but we suggest that the extensive dissemination of MST thus far—with over 500 licensed teams in 15 countries (MST Services 2018a)—indicates that it can be met across a wide variety of settings and cultural contexts. In addition, more economic evaluations of MST are also needed, particularly those that examine the impact of large-scale implementation and/or MST adaptations. Even the extensive pool of economic studies for standard MST was limited by a preponderance of studies conducted by MST developers and their direct collaborators, except for the WSIPP (2017) evaluations that—while rigorous—had not undergone peer review.
Overall, PFS shows considerable promise as a financing strategy for implementation of a complex, expensive evidence-based practice (i.e., MST) through private-public partnerships. These findings suggest that other intensive services for high-risk populations in mental and behavioral health might be good candidates for PFS contracts, pending review of the relevant evidence. For example, the Nurse–Family Partnership (Karoly et al. 1998) and the Perry Preschool Program (Nores et al. 2005) have produced favorable clinical and economic outcomes, whereas findings on the economic attractiveness of wraparound services (Swenson et al. 2000) and the Fast Track program (Foster et al. 2006) were less convincing. Evaluation using the PFS intervention selection criteria (Lantz and Iovan 2017) would be a useful first step to identify which practices are the best candidates for using PFS financing to fund their implementation.
Finally, it is important to bear in mind that numerous factors beyond intervention selection (e.g., legal and policy challenges in PFS payouts, knowledge and technical skills, level of local support) affect the capacity of a given government to execute the PFS financing model (Lantz et al. 2016; Segal et al. 2016; Social Finance, Inc. 2012). If a particular government were interested in using PFS to implement MST, they would need to determine how various government entities would be involved in each aspect of the contract based on what makes sense for their context. For example, a particular government entity might be designated to provide the PFS payout, though some of the most successful administrative structures for PFS contracts have involved states dedicating centralized funds for these payouts. This helps avoid budgetary conflicts of interest when savings accrue across different government sectors or agencies (Lantz et al. 2016). The promise of PFS as a financing strategy for any given evidence-based practice—MST or otherwise—is dependent on local capacity to execute a PFS contract, which should be established prior to the intervention selection process illustrated in this brief.
References
Aarons, G. A., Sklar, M., Mustanski, B., Benbow, N., & Brown, C. H. (2017). “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implementation Science, 12, 111. https://doi.org/10.1186/s13012-017-0640-6.
Beidas, R. S., & Powell, B. J. (2016). Advancing implementation research and practice in behavioral health systems. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 825–833. https://doi.org/10.1007/s10488-016-0762-1.
Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 228–236. https://doi.org/10.1007/s10488-012-0461-5.
Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312. https://doi.org/10.1037/h0040950.
Dopp, A. R., Borduin, C. M., Wagner, D. V., & Sawyer, A. M. (2014). The economic impact of multisystemic therapy through midlife: A cost–benefit analysis with serious juvenile offenders and their siblings. Journal of Consulting and Clinical Psychology, 82(4), 694–705. https://doi.org/10.1037/a0036415.
Dopp, A. R., Borduin, C. M., White, M. H., II, & Kuppens, S. (2017). Family-based treatments for serious juvenile offenders: A multilevel meta-analysis. Journal of Consulting and Clinical Psychology, 85(4), 335–354. https://doi.org/10.1037/ccp0000183.
Dopp, A. R., Coen, A. S., Smith, A. B., Reno, J., Bernstein, D. H., Kerns, S. E. U., … Altschul, D. (2018a). Economic impact of the statewide implementation of an evidence-based treatment: Multisystemic therapy in New Mexico. Behavior Therapy, 49(4), 551–566. https://doi.org/10.1016/j.beth.2017.12.003.
Dopp, A. R., Schaeffer, C. M., Swenson, C. C., & Powell, J. S. (2018b). Economic impact of multisystemic therapy for child abuse and neglect. Administration and Policy in Mental Health and Mental Health Services Research. https://doi.org/10.1007/s10488-018-0870-1.
Foster, E. M., Jones, D., & Conduct Problems Prevention Research Group. (2006). Can a costly intervention be cost-effective?: An analysis of violence prevention. Archives of General Psychiatry, 63(11), 1284–1291. https://doi.org/10.1001/archpsyc.63.11.1284.
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (2009). Multisystemic therapy for antisocial behavior in children and adolescents. London: Guilford Press.
Iovan, S., Lantz, P. M., & Shapiro, S. (2018). “Pay for Success” projects: Financing interventions that address social determinants of health in 20 countries. American Journal of Public Health, 108(11), 1473–1477. https://doi.org/10.2105/AJPH.2018.304651.
Karoly, L. A., Greenwood, P. W., Everingham, S. S., Hoube, J., Kilburn, M. R., Rydell, C. P., … Chiesa, J. (1998). Investing in our children: What we know and don’t know about the costs and benefits of early childhood interventions. Santa Monica, CA: RAND Corporation.
Lantz, P. M., & Iovan, S. (2017). When does pay-for-success make sense? Stanford Social Innovation Review. Retrieved February 2, 2018, from https://ssir.org/articles/entry/when_does_pay_for_success_make_sense.
Lantz, P. M., Rosenbaum, S., Ku, L., & Iovan, S. (2016). Pay for success and population health: Early results from eleven projects reveal challenges and promise. Health Affairs, 35(11), 2053–2061. https://doi.org/10.1377/hlthaff.2016.0713.
McCart, M. R., & Sheidow, A. J. (2016). Evidence-based psychosocial treatments for adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 45(5), 529–563. https://doi.org/10.1080/15374416.2016.1146990.
MST Services. (2017). Multisystemic Therapy adaptations: Pilot studies to large-scale dissemination. Mount Pleasant, SC: Author.
MST Services. (2018a). MST licensed organizations. Retrieved April 11, 2018, from http://www.mstservices.com/licensed-organizations.
MST Services. (2018b). Multisystemic Therapy research at a glance: Published MST outcome, implementation and benchmarking studies. Mount Pleasant, SC: Author.
National Research Council. (2013). Reforming juvenile justice: A developmental approach. Washington, DC: The National Academies Press. https://doi.org/10.17226/14685.
Nores, M., Belfield, C. R., Barnett, W. S., & Schweinhart, L. (2005). Updating the economic impacts of the High/Scope Perry Preschool Program. Educational Evaluation and Policy Analysis, 27, 245–261. https://doi.org/10.3102/01623737027003245.
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L.J., Smith, J. L., … Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10:21. https://doi.org/10.1186/s13012-015-0209-1.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., … Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7.
Roberts, L., & Cameron, G. (2014). Evaluation of the Essex multi-systemic therapy social impact bond: Interim report. London: OPM Group.
Roundfield, K. D., & Lang, J. M. (2017). Costs to community mental health agencies to sustain an evidence-based practice. Psychiatric Services, 68(9), 876–882. https://doi.org/10.1176/appi.ps.201600193.
Segal, J., Shumway, J., & Bianchi, J. P. (2016). New tools to amplify impact: A pay for success guide to building nonprofit capacity. Boston: Social Finance Inc.
Social Finance, Inc. (2018). Social Impact Bonds reach global mass: 108 projects launched in 24 countries. Retrieved February 2, 2018, from http://socialfinance.org/news/social-impact-bonds-reach-global-mass-108-projects-launched-24-countries/.
Social Finance, Inc. (2012). A new tool for scaling impact: How social impact bonds can mobilize private capital to advance social good. Boston: Author.
Swenson, C. C., Randall, J., Henggeler, S. W., & Ward, D. (2000). The outcomes and costs of an interagency partnership to serve maltreated children in state custody. Children’s Services: Social Policy, Research, and Practice, 3(4), 191–209. https://doi.org/10.1207/S15326918CS0304_1.
University of Denver, Graduate School of Social Work. (2018). GSSW Center for Effective Interventions to expand multisystemic therapy in underserved Colorado communities. https://socialwork.du.edu/news/reducing-juvenile-crime-colorado?futm_source=scene@gssw&utm_medium=email&utm_campaign=scene@gssw%3EMay18.
Urban Institute. (2016). Pay-for-Success project assessment tool. Washington, DC: Author. Retrieved March 6, 2019, from https://www.urban.org/research/publication/pay-success-project-assessment-tool.
Washington State Institute for Public Policy. (2017). Benefit-cost results. Olympia, WA: Author. Retrieved April 9, 2018, from http://www.wsipp.wa.gov/BenefitCost.
Funding
This study was funded by the Robert Wood Johnson Foundation through a grant supporting the University of Michigan Policies for Action Research Hub (PI: Lantz, #73217).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Alex R. Dopp, Cameron M. Perrine, Samantha Iovan, and Paula M. Lantz declare that they have no conflicts of interest.
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed Consent
This article does not contain any studies with human participants performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Dopp, A.R., Perrine, C.M., Iovan, S. et al. The Potential of Pay-for-Success as a Financing Strategy for Evidence-Based Practices: An Illustration with Multisystemic Therapy. Adm Policy Ment Health 46, 629–635 (2019). https://doi.org/10.1007/s10488-019-00940-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-019-00940-3