Abstract
High-quality implementation is important for preventive intervention effectiveness. Although this implies fidelity to a practice model, some adaptation may be inevitable or even advantageous in routine practice settings. In order to organize the study of adaptation and its effect on intervention outcomes, scholars have proposed various adaptation taxonomies. This paper examines how four published taxonomies retrospectively classify adaptations: the Ecological Validity Framework (EVF; Bernal et al. in J Abnorm Child Psychol 23(1):67–82, 1995), the Hybrid Prevention Program Model (HPPM; Castro et al. in Prev Sci 5(1):41–45, 2004. https://doi.org/10.1023/B:PREV.0000013980.12412.cd), the Moore et al. (J Prim Prev 34(3):147–161, 2013. https://doi.org/10.1007/s10935-013-0303-6) taxonomy, and the Stirman et al. (Implement Sci 8:65, 2013. https://doi.org/10.1186/1748-5908-8-65) taxonomy. We used these taxonomies to classify teacher-reported adaptations made during the implementation of TOOLBOX™, a social emotional learning program implemented in 11 elementary schools during the 2014–2015 academic year. Post-implementation, 271 teachers and staff responded to an online survey that included questions about adaptation, yielding 98 adaptation descriptions provided by 42 respondents. Four raters used each taxonomy to try to classify these descriptions. We assessed the extent to which raters agreed they could classify the descriptions using each taxonomy (coverage), as well as the extent to which raters agreed on the subcategory they assigned (clarity). Results indicated variance among taxonomies, and tensions between the ideals of coverage and clarity emerged. Further studies of adaptation taxonomies as coding instruments may improve their performance, helping scholars more consistently assess adaptations and their effects on preventive intervention outcomes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., et al. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7(1), 32. https://doi.org/10.1186/1748-5908-7-32.
Baumann, A. A., Powell, B. J., Kohl, P. L., Tabak, R. G., Penalba, V., Proctor, E. K., et al. (2015). Cultural adaptation and implementation of evidence-based parent-training: A systematic review and critique of guiding evidence. Children and Youth Services Review, 53, 113–120. https://doi.org/10.1016/j.childyouth.2015.03.025.
Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with hispanics. Journal of Abnormal Child Psychology, 23(1), 67–82.
Bernal, G., & Domenech Rodríguez, M. M. (2012). Cultural adaptations: Tools for evidence-based practice with diverse populations. Washington, DC: American Psychological Association.
Castro, F. G., Barrera, M., Jr., & Martinez, C. R., Jr. (2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5(1), 41–45. https://doi.org/10.1023/B:PREV.0000013980.12412.cd.
Center for Prevention Implementation Methodology. (n.d.). Retrieved July 13, 2018, from http://cepim.northwestern.edu/.
Chambers, D. A., & Norton, W. E. (2016). The adaptome. American Journal of Preventive Medicine, 51(4), S124–S131. https://doi.org/10.1016/j.amepre.2016.05.011.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.
Colby, M., Hecht, M. L., Miller-Day, M., Krieger, J. L., Syvertsen, A. K., Graham, J. W., et al. (2013). Adapting school-based substance use prevention curriculum through cultural grounding: A review and exemplar of adaptation processes for rural schools. American Journal of Community Psychology, 51, 190–205. https://doi.org/10.1007/s10464-012-9524-8.
Collin, M. A. (2015). TOOLBOX™ Primer. Sebastopol, CA: Dovetail Learning Inc.
Cooper, B. R., Shrestha, G., Hyman, L., & Hill, L. (2016). Adaptations in a community-based family intervention: Replication of two coding schemes. The Journal of Primary Prevention, 37(1), 33–52. https://doi.org/10.1007/s10935-015-0413-4.
Dusenbury, L., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20(3), 308–313. https://doi.org/10.1093/her/cyg134.
Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science. https://doi.org/10.1023/b:prev.0000013981.28071.52.
Ferrer-Wreder, L., Sundell, K., & Mansoory, S. (2012). Tinkering with perfection: Theory development in the intervention cultural adaptation field. Child and Youth Care Forum, 41(2), 149–171. https://doi.org/10.1007/s10566-011-9162-6.
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the oregon model of parent management training. Behavior Therapy, 36(1), 3–13.
Glasgow, R. E., & Emmons, K. M. (2007). How can we increase translation of research into practice? Types of evidence needed. Annual Review of Public Health, 28(1), 413–433. https://doi.org/10.1146/annurev.publhealth.28.021406.144145.
Goncy, E. A., Sutherland, K. S., Farrell, A. D., Sullivan, T. N., & Doyle, S. T. (2015). Measuring teacher implementation in delivery of a bullying prevention program: The impact of instructional and procedural adherence and competence on student responsiveness. Prevention Science, 16(3), 440–450. https://doi.org/10.1007/s11121-014-0508-9.
Gould, S. J. (2011). Full house. Cambridge: Harvard University Press.
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23.
Hansen, W. B., Pankratz, M. M., Dusenbury, L., Giles, S. M., Bishop, D. C., Albritton, J., et al. (2013). Styles of adaptation: The impact of frequency and valence of adaptation on preventing substance use. Health Education, 113(4), 345–363.
Keith, R. E., Hopp, F. P., Subramanian, U., Wiitala, W., & Lowery, J. C. (2010). Fidelity of implementation: Development and testing of a measure. Implementation Science, 5(1), 99. https://doi.org/10.1186/1748-5908-5-99.
Kemp, L. (2016). Adaptation and fidelity: A recipe analogy for achieving both in population scale implementation. Prevention Science, 17(4), 429–438. https://doi.org/10.1007/s11121-016-0642-7.
Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 33(2), 363–374.
Leong, F. T., & Lee, S.-H. (2006). A cultural accommodation model for cross-cultural psychotherapy: Illustrated with the case of Asian Americans. Psychotherapy: Theory, Research, Practice, Training, 43(4), 410.
Lewis, C. C., Stanick, C. F., Martinez, R. G., Weiner, B. J., Kim, M., Barwick, M., et al. (2015). The society for implementation research collaboration instrument review project: A methodology to promote rigorous evaluation. Implementation Science, 10(1), 2. https://doi.org/10.1186/s13012-014-0193-x.
Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31(2), 242–257. https://doi.org/10.1177/1090198103260514.
Moore, J. E., Bumbarger, B. K., & Cooper, B. R. (2013). Examining adaptations of evidence-based programs in natural contexts. The Journal of Primary Prevention, 34(3), 147–161. https://doi.org/10.1007/s10935-013-0303-6.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3), 315–340. https://doi.org/10.1177/109821400302400303.
O’Connell, M. E. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academies Press.
Ogden, T., Bjørnebekk, G., Kjøbli, J., Patras, J., Christiansen, T., Taraldsen, K., et al. (2012). Measurement of implementation components ten years after a nationwide introduction of empirically supported programs: A pilot study. Implementation Science, 7(1), 49. https://doi.org/10.1186/1748-5908-7-49.
Ozer, E. J., Wanis, M. G., & Bazell, N. (2010). Diffusion of school-based prevention programs in two urban districts: Adaptations, rationales, and suggestions for change. Prevention Science, 11(1), 42–55. https://doi.org/10.1007/s11121-009-0148-7.
Rabin, B. A., Purcell, P., Naveed, S., Moser, R. P., Henton, M. D., Proctor, E. K., et al. (2012). Advancing the application, quality and harmonization of implementation science measures. Implementation Science: IS, 7, 119. https://doi.org/10.1186/1748-5908-7-119.
Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., et al. (1998). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13(2), 239–250.
Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67(6), 463–476. https://doi.org/10.1037/a0028180.
Schober, I., Sharpe, H., & Schmidt, U. (2013). The reporting of fidelity measures in primary prevention programmes for eating disorders in schools. European Eating Disorders Review. https://doi.org/10.1002/erv.2243.
Shapiro, V. B., Kim, B. K. E., Accomazzo, S., & Roscoe, J. N. (2016). Predictors of rater bias in the assessment of social-emotional competence. International Journal of Emotional Education, 8(2), 25.
Spoth, R., Rohrbach, L. A., Greenberg, M., Leaf, P., Brown, C. H., Fagan, A., et al. (2013). Addressing core challenges for the next generation of type 2 translation research and systems: The translation science to population impact (tsci impact) framework. Prevention Science, 14(4), 319–351. https://doi.org/10.1007/s11121-012-0362-6.
SPR MAPS II Task Force. (2008). Type 2 translational research: Overview and definitions. Retrieved June 13, 2017, from http://www.preventionresearch.org/SPR_Type2TranslationResearch_OverviewandDefinition.pdf.
Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8, 65. https://doi.org/10.1186/1748-5908-8-65.
Wiltsey Stirman, S., Gutner, C. A., Crits-Christoph, P., Edmunds, J., Evans, A. C., & Beidas, R. S. (2015). Relationships between clinician-level attributes and fidelity-consistent and fidelity-inconsistent modifications to an evidence-based psychotherapy. Implementation Science, 10(1), 115. https://doi.org/10.1186/s13012-015-0308-z.
Acknowledgements
This research was funded by the Stuart Foundation and a Hellman Foundation Graduate Fellow Award. We thank Mark Collin, Dr. Chuck Fisher, Pamela McVeagh-Lally, and our colleagues at the UC Berkeley Center for Prevention Research in Social Welfare (especially Dr. Sarah Accomazzo and Kimberly Knodel) for their contributions to this work. We also thank Dr. Stacey Alexeeff for statistical consultation. Finally, we thank the administrators, teachers, and staff who participated in this research, and Catherine Rodecker and Dr. Kathryn Mapps for their implementation and evaluation leadership. Aspects of this paper were previously presented at the 2016 Society for Prevention Research conference in San Francisco and the 2017 Society for Social Work and Research conference in New Orleans. All research protocols were approved by the Committee for the Protection of Human Subjects (CPHS) at the University of California, Berkeley.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare they have no conflicts of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
Taxonomy coverage by adaptation question
Taxonomy | Overall | Favorite lesson | Least favorite lesson | Outside lesson | ||||
|---|---|---|---|---|---|---|---|---|
% | κ | % | κ | % | κ | % | κ | |
EVF | 57 | 0.06 | 63 | − 0.07 | 48 | 0.08 | 57 | 0.21 |
HPPM—Form | 19 | 0.12 | 16 | 0.10 | 8 | 0.05 | 33 | 0.18 |
HPPM—Dimension | 16 | 0.26 | 16 | 0.17 | 8 | 0.16 | 23 | 0.35 |
Moore—Fit | 36 | 0.26 | 33 | 0.26 | 28 | 0.10 | 47 | 0.39 |
Moore—Timing | 5 | − 0.07 | 7 | − 0.01 | 0 | − 0.15 | 7 | − 0.09 |
Moore—Valence | 66 | 0.02 | 77 | − 0.10 | 56 | − 0.01 | 60 | 0.18 |
Stirman—Decision | 90 | 0.10 | 95 | 0.22 | 84 | − 0.04 | 87 | 0.11 |
Stirman—Adaptation Type | 82 | 0.13 | 84 | 0.21 | 84 | 0.03 | 77 | 0.13 |
Stirman—Context | 0 | 1.00 | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 |
Stirman—Level | 81 | − 0.12 | 81 | − 0.13 | 78 | − 0.17 | 86 | − 0.09 |
Stirman—Nature | 87 | 0.33 | 92 | 0.02 | 78 | 0.13 | 86 | 0.46 |
Appendix 2
Taxonomy clarity by adaptation question
Taxonomy | Overall | Favorite lesson | Least favorite lesson | Outside lesson | ||||
|---|---|---|---|---|---|---|---|---|
% | κ | % | κ | % | κ | % | κ | |
EVF | 32 | 0.31 | 22 | 0.23 | 58 | 0.46 | 29 | 0.24 |
Language | 2 | 0 | 8 | 0 | ||||
Persons | 4 | 0 | 8 | 6 | ||||
Metaphors | 0 | 0 | 0 | 0 | ||||
Content | 4 | 7 | 0 | 0 | ||||
Concepts | 0 | 0 | 0 | 0 | ||||
Goals | 0 | 0 | 0 | 0 | ||||
Methods | 14 | 11 | 42 | 0 | ||||
Context | 9 | 4 | 0 | 24 | ||||
HPPM—Form | 47 | 0.11 | 29 | 0.08 | 50 | − 0.13 | 60 | 0.19 |
Content | 37 | 29 | 0 | 50 | ||||
Delivery | 11 | 0 | 50 | 10 | ||||
HPPM—Dimension | 94 | 0.45 | 86 | 0.32 | 100 | 1.00 | 100 | 0.24 |
Cognitive-informational | 69 | 43 | 50 | 100 | ||||
Affective-motivational | 25 | 43 | 50 | 0 | ||||
Environmental | 0 | 0 | 0 | 0 | ||||
Moore—Fit | 77 | 0.03 | 79 | 0.33 | 86 | − 0.12 | 71 | − 0.43 |
Logistical | 20 | 43 | 14 | 0 | ||||
Philosophical | 57 | 36 | 71 | 71 | ||||
Moore—Timing | 100 | 0.25 | 100 | 0.26 | 0 | 0.00 | 100 | − 0.15 |
Proactive | 60 | 100 | 0 | 0 | ||||
Reactive | 40 | 0 | 0 | 100 | ||||
Moore—Valence | 43 | 0.13 | 42 | 0.01 | 57 | 0.32 | 33 | 0.08 |
Positive | 28 | 33 | 21 | 22 | ||||
Neutral | 14 | 9 | 36 | 6 | ||||
Negative | 2 | 0 | 0 | 6 | ||||
Stirman—Decision Maker | 98 | 0.17 | 100 | − 0.04 | 95 | 0.26 | 96 | 0.29 |
Individual practitioner | 98 | 100 | 95 | 96 | ||||
Team or group of practitioners | 0 | 0 | 0 | 0 | ||||
Administrator | 0 | 0 | 0 | 0 | ||||
Researcher | 0 | 0 | 0 | 0 | ||||
Intervention developer or purveyor | 0 | 0 | 0 | 0 | ||||
Coalition of stakeholders | 0 | 0 | 0 | 0 | ||||
Stirman—Adaptation Type | 94 | − 0.01 | 100 | − 0.09 | 86 | 0.13 | 91 | 0.06 |
Training and evaluation processes | 0 | 0 | 0 | 0 | ||||
Context | 0 | 0 | 0 | 0 | ||||
Content | 94 | 100 | 86 | 91 | ||||
Stirman—Context | 0 | – | 0 | – | 0 | – | 0 | – |
Format | 0 | 0 | 0 | 0 | ||||
Setting | 0 | 0 | 0 | 0 | ||||
Personnel | 0 | 0 | 0 | 0 | ||||
Population | 0 | 0 | 0 | 0 | ||||
Stirman—Level | 25 | − 0.15 | 21 | − 0.17 | 14 | − 0.10 | 39 | − 0.21 |
Individual recipient | 0 | 0 | 0 | 0 | ||||
Cohort | 23 | 21 | 7 | 39 | ||||
Population | 2 | 0 | 7 | 0 | ||||
Provider/facilitator | 0 | 0 | 0 | 0 | ||||
Unit | 0 | 0 | 0 | 0 | ||||
Hospital/organization | 0 | 0 | 0 | 0 | ||||
Network/community | 0 | 0 | 0 | 0 | ||||
Stirman—Nature | 78 | 0.62 | 79 | 0.59 | 71 | 0.74 | 83 | 0.45 |
Tailoring/tweaking/refining | 42 | 33 | 36 | 61 | ||||
Adding elements | 25 | 33 | 21 | 11 | ||||
Removing elements | 9 | 9 | 7 | 11 | ||||
Shortening/condensing | 6 | 3 | 7 | 0 | ||||
Lengthening/extending | 0 | 0 | 0 | 0 | ||||
Substituting elements | 0 | 0 | 0 | 0 | ||||
Re-ordering elements | 0 | 0 | 0 | 0 | ||||
Integrating another approach into the intervention | 0 | 0 | 0 | 0 | ||||
Integrating the intervention into another approach | 0 | 0 | 0 | 0 | ||||
Repeating elements | 0 | 0 | 0 | 0 | ||||
Loosening structure | 0 | 0 | 0 | 0 | ||||
Departing from the intervention (“drift”) | 0 | 0 | 0 | 0 | ||||
Rights and permissions
About this article
Cite this article
Roscoe, J.N., Shapiro, V.B., Whitaker, K. et al. Classifying Changes to Preventive Interventions: Applying Adaptation Taxonomies. J Primary Prevent 40, 89–109 (2019). https://doi.org/10.1007/s10935-018-00531-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10935-018-00531-2