Abstract
Despite the utility of formative evaluation in implementation research, few published projects in low- and middle-income countries have used this approach and incorporating qualitative data into implementation projects can be challenging. Implementation science and qualitative formative evaluation can help inform the delivery of evidence-based practices, specifically identifying gaps in service delivery and implementation fidelity. Including stakeholders in formative evaluation facilitates actionable feedback and practical problem-solving of real-time implementation challenges. This paper presents a real-world example of qualitative formative evaluation that was used to inform the implementation of an integrated healthcare intervention in Togo, West Africa. We organize our results around the five main advantages of implementation-focused formative evaluation that can provide context-specific data useful for implementation teams: (a) highlighting actual versus planned interventions; (b) enabling implementation through identification of modifiable barriers; (c) facilitating any needed refinements in the original implementation intervention; (d) enhancing interpretation of project results; and (e) identifying critical details and guidance necessary for replication of results in other clinical settings. We provide specific examples of how these qualitative findings provide context-specific data and result in actionable steps, with a specific focus on including program implementers and stakeholders in the process. This paper describes lessons learned from the implementation of qualitative implementation-focused formative evaluation.
Clinical Trial Registration ClinicalTrials.gov, NCT03694366 (registered 10/03/2018).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- LMIC:
-
Low- and middle-income country
- SDG:
-
Sustainable development goal
- CHW:
-
Community Health Worker
- IMCI:
-
Integrated Management of Childhood Illness
- QUERI:
-
Quality Enhancement Research Initiative
- CFIR:
-
Consolidated Framework for Implementation Research
- MOH:
-
Ministry of Health and Public Hygiene
References
Arnold, J., Samson, M., Schechter, J., Goodwin, A. S., Braganza, S., Sesso, G. C., Lopez, A., & Fiori, K. (2016). Getting there: Overcoming barriers to reproductive and maternal health services access in Northern Togo—A qualitative study. World Medical & Health Policy, 8(3), 223–244. https://doi.org/10.1002/wmh3.195
Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), 32. https://doi.org/10.1186/s40359-015-0089-9
Clark, L., Birkhead, A. S., Fernandez, C., & Egger, M. J. (2017). A transcription and translation protocol for sensitive cross-cultural team research. Qualitative Health Research, 27(12), 1751–1764. https://doi.org/10.1177/1049732317726761
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50
Dubois, M., Faro, E. Z., Lee, D. S., Katin, V., Kenkou, K., & Fiori, K. P. (2020). Integrating childhood TB: Applying the care delivery value chain to improve pediatric HIV/TB services in Togo, West Africa. AIDS Care, 32(11), 1445–1450. https://doi.org/10.1080/09540121.2020.1770672
Dyke, E., Pénicaud, S., Hatchard, J., Dawson, A.-M., Munishi, O., & Jalal, C. (2021). Girl-Powered Nutrition program: Key themes from a formative evaluation of a nutrition program co-designed and implemented by adolescent girls in low- and middle-income countries. Current Developments in Nutrition, 5(7), nzab083. https://doi.org/10.1093/cdn/nzab083
Eccles, M. P., & Mittman, B. S. (2006). Welcome to Implementation Science. Implementation Science, 1(1), 1. https://doi.org/10.1186/1748-5908-1-1
El Bcheraoui, C., Mimche, H., Miangotar, Y., Krish, V. S., Ziegeweid, F., Krohn, K. J., Ekat, M. H., Nansseu, J. R., Dimbuene, Z. T., Olsen, H. E., Tine, R. C. K., Odell, C. M., Troeger, C. E., Kassebaum, N. J., Farag, T., Hay, S. I., & Mokdad, A. H. (2020). Burden of disease in francophone Africa, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. The Lancet Global Health, 8(3), e341–e351. https://doi.org/10.1016/s2214-109x(20)30024-3
Elwy, A. R., Wasan, A. D., Gillman, A. G., Johnston, K. L., Dodds, N., McFarland, C., & Greco, C. M. (2020). Using formative evaluation methods to improve clinical implementation efforts: Description and an example. Psychiatry Research, 283, 112532. https://doi.org/10.1016/j.psychres.2019.112532
Escoffery, C., Lebow-Skelley, E., Haardoerfer, R., Boing, E., Udelson, H., Wood, R., Hartman, M., Fernandez, M., & Mullen, P. (2018). A systematic review of adaptations of evidence-based public health interventions globally. Implementation Science, 13(1), 1–21.
Fiori, K., Schechter, J., Dey, M., Braganza, S., Rhatigan, J., Houndenou, S., Gbeleou, C., Palerbo, E., Tchangani, E., Lopez, A., Bensen, E., & Hirschhorn, L. R. (2016). Closing the delivery gaps in pediatric HIV care in Togo, West Africa: Using the care delivery value chain framework to direct quality improvement. AIDS Care, 28(Suppl 2), 29–33. https://doi.org/10.1080/09540121.2016.1176678
Fiori, K. P., Belli, H. M., Lauria, M. E., Hirschhorn, L. R., Schechter, J., Hansman, E., Rajshekhar, N., Katin, V., Gbeleou, S., Grunitsky-Bekele, M., & Pitche, V. P. (2020). Implementing an integrated community based health systems strengthening approach to improve HIV survival in Northern Togo. AIDS Care, 32(6), 705–713. https://doi.org/10.1080/09540121.2019.1626342
Fiori, K. P., Lauria, M. E., Singer, A. W., Jones, H. E., Belli, H. M., Aylward, P. T., Agoro, S., Gbeleou, S., Sowu, E., Grunitzky-Bekele, M., Singham Goodwin, A., Morrison, M., Ekouevi, D. K., & Hirschhorn, L. R. (2021). An integrated primary care initiative for child health in Northern Togo. Pediatrics, 148(3), e2020035493. https://doi.org/10.1542/peds.2020-035493
Garchitorena, A., Miller, A. C., Cordier, L. F., Rabeza, V. R., Randriamanambintsoa, M., Razanadrakato, H. R., Hall, L., Gikic, D., Haruna, J., McCarty, M., Randrianambinina, A., Thomson, D. R., Atwood, S., Rich, M. L., Murray, M. B., Ratsirarson, J., Ouenzar, M. A., & Bonds, M. H. (2018). Early changes in intervention coverage and mortality rates following the implementation of an integrated health system intervention in Madagascar. BMJ Global Health, 3(3), e000762. https://doi.org/10.1136/bmjgh-2018-000762
Golding, N., Burstein, R., Longbottom, J., Browne, A. J., Fullman, N., Osgood-Zimmerman, A., Earl, L., Bhatt, S., Cameron, E., Casey, D. C., Dwyer-Lindgren, L., Farag, T. H., Flaxman, A. D., Fraser, M. S., Gething, P. W., Gibson, H. S., Graetz, N., Krause, L. K., Kulikoff, X. R., & Hay, S. I. (2017). Mapping under-5 and neonatal mortality in Africa, 2000–15: A baseline analysis for the Sustainable Development Goals. Lancet, 390(10108), 2171–2182. https://doi.org/10.1016/s0140-6736(17)31758-0
Guest, G., Bunce, A., & Johnson, L. (2016). How many interviews are enough? Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822x05279903
Hagedorn, H., Hogan, M., Smith, J. L., Bowman, C., Curran, G. M., Espadas, D., Kimmel, B., Kochevar, L., Legro, M. W., & Sales, A. E. (2006). Lessons learned about implementing research evidence into clinical practice. Experiences from VA QUERI. Journal of General Internal Medicine, 21(Suppl 2), S21–S24. https://doi.org/10.1111/j.1525-1497.2006.00358.x
Hamilton, A. B., Brunner, J., Cain, C., Chuang, E., Luger, T. M., Canelo, I., Rubenstein, L., & Yano, E. M. (2017). Engaging multilevel stakeholders in an implementation trial of evidence-based quality improvement in VA women’s health primary care. Translational Behavioral Medicine, 7(3), 478–485. https://doi.org/10.1007/s13142-017-0501-5
Hamilton, A. B., & Finley, E. P. (2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, 112516. https://doi.org/10.1016/j.psychres.2019.112516
Healy, E. A., Kaiser, B. N., & Puffer, E. S. (2018). Family-based youth mental health interventions delivered by nonspecialist providers in low- and middle-income countries: A systematic review. Families, Systems & Health, 36(2), 182–197. https://doi.org/10.1037/fsh0000334
Hug, L., Sharrow, D., & You, D. (2019). Levels & trends in child mortality: Report 2019. UNICEF. Retrieved from https://www.unicef.org/media/60561/file/UN-IGME-child-mortality-report-2019.pdf.
Kc, A., Singh, D. R., Upadhyaya, M. K., Budhathoki, S. S., Gurung, A., & Målqvist, M. (2020). Quality of care for maternal and newborn health in health facilities in Nepal. Maternal and Child Health Journal, 24(Suppl 1), 31–38. https://doi.org/10.1007/s10995-019-02846-w
King, J. A., King, M. J., Edwards, N., Hair, S. A., Cheang, S., Pearson, A., & Coelho, S. (2018). Addressing transport safety and accessibility for people with a disability in developing countries: A formative evaluation of the Journey Access Tool in Cambodia. Global Health Action, 11(1), 1538658.
Kirchner, J. E., Parker, L. E., Bonner, L. M., Fickel, J. J., Yano, E. M., & Ritchie, M. J. (2012). Roles of managers, frontline staff and local champions, in implementing quality improvement: Stakeholders’ perspectives. Journal of Evaluation in Clinical Practice, 18(1), 63–69. https://doi.org/10.1111/j.1365-2753.2010.01518.x
Lauria, M. E., Fiori, K. P., Jones, H. E., Gbeleou, S., Kenkou, K., Agoro, S., Agbèrè, A. D., Lue, K. D., & Hirschhorn, L. R. (2019). Assessing the Integrated Community-Based Health Systems Strengthening initiative in northern Togo: A pragmatic effectiveness-implementation study protocol. Implementation Science, 14(1), 92. https://doi.org/10.1186/s13012-019-0921-3
McCarthy, K. J., Braganza, S., Fiori, K., Gbeleou, C., Kpakpo, V., Lopez, A., Schechter, J., Singham Goodwin, A., & Jones, H. E. (2017). Identifying inequities in maternal and child health through risk stratification to inform health systems strengthening in Northern Togo. PLoS ONE, 12(3), e0173445. https://doi.org/10.1371/journal.pone.0173445
McIntosh, S., Pérez-Ramos, J., Demment, M. M., Vega, C. V., Avendaño, E., Ossip, D. J., & Dye, T. D. (2016). Development and implementation of culturally tailored offline mobile health surveys. JMIR Public Health and Surveillance, 2(1), e5408.
Ministère de la Planification, Ministère de la Santé Togo, & ICF International. (2015). Enquête Démographique et de Santé au Togo 2013–2014. Retrieved from http://dhsprogram.com/pubs/pdf/FR301/FR301.pdf.
Ministère de la Santé et de la Protection Sociale. (2017). Plan National de Developpement Sanitaire.
Namazzi, G., Waiswa, P., Nakakeeto, M., Nakibuuka, V. K., Namutamba, S., Najjemba, M., Namusaabi, R., Tagoola, A., Nakate, G., Ajeani, J., Peterson, S., & Byaruhanga, R. N. (2015). Strengthening health facilities for maternal and newborn care: Experiences from rural eastern Uganda. Global Health Action, 8, 24271. https://doi.org/10.3402/gha.v8.24271
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y
Perrin, K. M., Burke, S. G., O’Connor, D., Walby, G., Shippey, C., Pitt, S., McDermott, R. J., & Forthofer, M. S. (2006). Factors contributing to intervention fidelity in a multi-site chronic disease self-management program. Implementation Science, 1(1), 1–6.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
Safaeinili, N., Brown-Johnson, C., Shaw, J. G., Mahoney, M., & Winget, M. (2020). CFIR simplified: Pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learning Health Systems, 4(1), e10201. https://doi.org/10.1002/lrh2.10201
Sorensen, G., Nagler, E. M., Pawar, P., Gupta, P. C., Pednekar, M. S., & Wagner, G. R. (2017). Lost in translation: The challenge of adapting integrated approaches for worker health and safety for low- and middle-income countries. PLoS ONE, 12(8), e0182607. https://doi.org/10.1371/journal.pone.0182607
Stetler, C. B., Legro, M. W., Wallace, C. M., Bowman, C., Guihan, M., Hagedorn, H., Kimmel, B., Sharp, N. D., & Smith, J. L. (2006). The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine, 21(Suppl 2), S1–S8. https://doi.org/10.1111/j.1525-1497.2006.00355.x
Theobald, S., Brandes, N., Gyapong, M., El-Saharty, S., Proctor, E., Diaz, T., Wanji, S., Elloker, S., Raven, J., Elsey, H., Bharal, S., Pelletier, D., & Peters, D. H. (2018). Implementation research: New imperatives and opportunities in global health. Lancet, 392(10160), 2214–2228. https://doi.org/10.1016/s0140-6736(18)32205-0
Vaismoradi, M., & Snelgrove, S. (2019). Theme in qualitative content analysis and thematic analysis. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 20(3), 3376. https://doi.org/10.17169/fqs-20.3.3376
World Health Organization. (2015). Service Availability and Readiness Assessment (SARA): An annual monitoring system for service delivery. Retrieved from https://www.who.int/healthinfo/systems/sara_reference_manual/en/.
World Health Organization. (2018). Maternal mortality in 2000–2017. Retrieved from https://www.who.int/gho/maternal_health/countries/tgo.pdf?ua=1.
Acknowledgements
The authors wish to acknowledge all those who participated in the interviews conducted for this study. Without their work and their words, this paper would not have been possible.
Funding
Study funded by Integrate Health through generous philanthropic support. No funders had any role in the design or analysis of this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors have indicated that they have no conflicts of interest to disclose.
Ethical Approval
Ethics approvals for this study were obtained from the institutional review. Boards of the Togolese Ministry of Health in Lomé, Togo and the Albert Einstein College of Medicine in New York, USA (ref: 072744).
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Faro, E.Z., Haughton, J., Lauria, M.E. et al. A Pragmatic Approach to Qualitative Formative Evaluation of an Integrated Primary Care Program in Togo, West Africa. Glob Implement Res Appl 2, 209–218 (2022). https://doi.org/10.1007/s43477-022-00055-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43477-022-00055-2