WASH Benefits Bangladesh trial: system for monitoring coverage and quality in an efficacy trial
- 500 Downloads
Researchers typically report more on the impact of public health interventions and less on the degree to which interventions were followed implementation fidelity. We developed and measured fidelity indicators for the WASH Benefits Bangladesh study, a large-scale efficacy trial, in order to identify gaps between intended and actual implementation.
Community health workers (CHWs) delivered individual and combined water, sanitation, handwashing (WSH) and child nutrition interventions to 4169 enrolled households in geographically matched clusters. Households received free enabling technologies (insulated water storage container; sani-scoop, potty, double-pit, pour-flush latrine; handwashing station, soapy-water storage bottle), and supplies (chlorine tablets, lipid-based nutrient supplements, laundry detergent sachets) integrated with parallel behavior-change promotion. Behavioral objectives were drinking treated, safely stored water, safe feces disposal, handwashing with soap at key times, and age-appropriate nutrition behaviors. We administered monthly surveys and spot-checks to households from randomly selected clusters for 6 months early in the trial. If any fidelity measures fell below set benchmarks, a rapid response mechanism was triggered.
In the first 3 months, functional water seals were detected in 33% (14/42) of latrines in the sanitation only arm; 35% (14/40) for the combined WSH arm; and 60% (34/57) for the combined WSH and Nutrition arm, all falling below the pre-set benchmark of 80%. Other fidelity indicators met the 65 to 80% uptake benchmarks. Rapid qualitative investigations determined that households concurrently used their own latrines with broken water seals in parallel with those provided by the trial. In consultation with the households, we closed pre-existing latrines without water seals, increased the CHWs’ visit frequency to encourage correct maintenance of latrines with water seals, and discouraged water-seal removal or breakage. At the sixth assessment, 86% (51/59) of households were in sanitation only; 92% (72/78) in the combined WSH; and 93% (71/76) in the combined WSH and Nutrition arms had latrines with functional water seals.
An intensive implementation fidelity monitoring and rapid response system proved beneficial for this efficacy trial. To implement a routine program at scale requires further research into an adaptation of fidelity monitoring that supports program effectiveness.
WASH Benefits Bangladesh: ClinicalTrials.gov, ID: NCT01590095. Registered on 30 April 2012.
KeywordsWASH Benefits Implementation fidelity Water quality Sanitation Handwashing Child nutrition Efficacy Behavior change Cluster randomized controlled trial Bangladesh
Community health worker
Village education resource center
Water, sanitation, and hygiene
Researchers typically report on the health impact and/or behavior change of large-scale, often complex public health interventions that include methods to persuade target groups to adopt an innovation such as a health-related technology or behavior . Intervention impact may vary depending on programmatic efficiency, and political and social contexts. Contextual factors in both implementation and operation may moderate the impact of a planned intervention . Very few community trials have reported on implementation fidelity, i.e., the degree to which an intervention is delivered as intended [3, 4, 5].
Implementation fidelity is an important fundamental tool for assessing the implementation process [6, 7, 8] which can help to explain the association between intervention and outcomes . When limited or no impact is detected in an intervention trial, it is difficult to determine whether this occurred because the intervention was ineffective, or intervention uptake and implementation quality was poor . Fidelity measurement can additionally enhance the credibility of intervention impact and transferability to other settings . Conceptually, implementation fidelity has been defined using five elements: adherence to the planned intervention, exposure or dose, quality of delivery, participant responsiveness and program differentiation [4, 5]. Of these, quality of delivery can be measured against benchmarks set by the intervention implementers, to relate to adequate participant exposure [4, 11]. Determining whether fidelity indicators meet benchmarks early in the implementation period can guide corrective action through investigations of shortfalls in intervention delivery or uptake.
Among components of water, sanitation, and hygiene (WASH) interventions, there are some that can be initiated at one time point; e.g., water treatment or handwashing. However, sanitation interventions that include latrine construction require a series of steps to be completed; for example, negotiating location with the householders, digging the pit, producing the cement rings and slab, installing rings and slabs, and installing the above-ground (super) structure. Not all steps are necessarily completed as indicated; latrines may be half-constructed, or constructed and never used, or modified post construction. These conditions can limit the ability of latrines to separate feces from the environment, resulting in low implementation fidelity. WASH Benefits was a large-scale, cluster-randomized efficacy trial that aimed to assess the impact of improvements in water quality, sanitation, handwashing, and child nutrition and feeding – alone and in combination, on child health . Therefore, it was necessary to achieve high levels of uptake of both enabling technologies (latrines, handwashing stations, etc.) and the related behaviors. To this end, we put in place an intensive system for monitoring intervention delivery, described in detail elsewhere . Here, we describe development and use of fidelity measure indicators for the WASH Benefits Bangladesh study, as part of a three-paper series on WASH Benefits Intervention Delivery and Performance [13, 14]. We illustrate the benefits of early review of a rigorous, real-time fidelity assessment for this trial that required complex hardware delivery and installation, as well as parallel behavior-change promotion. We describe the use of benchmarks to signal early implementation problems, particularly for latrine installation that required qualitative investigation and course correction. Monitoring fidelity indicators throughout the intervention period is described in the third paper of this series .
WASH Benefits intervention design
Description of WASH Benefits trial interventions, fidelity measures and benchmarks, Bangladesh, 2012–2015
Provided to eligible householdsb
Fidelity measuresc (benchmark for uptakec)
Free supply of chlorine tablets (Aquatabs® brand, sodium dichloroisocyanurate) with a 10-L insulated water storage container (Fig. 1a)
Storage container, N = 2160
Presence of chlorine in stored water detected using a HACH color-wheel colorimeter (> 0.2 mg/L) (HACH LANGE GmbH, USA) (65%)
Double-pit improved latrine; sani- scoop, a locally developed tool for child and animal feces handling and disposal (Sultana et al.); potties (Faruqe) (Fig. 1b)
Latrine installed: 4533
Child potty: 4384
Reported location and method of disposal for child’s last bowel motion (65%)
Spot check for sani-scoop that easily accessible to mother (80%)
Spot check for visible potty (easily accessible to mother) (80%)
Spot check of designated household latrines; a functional water seal (80%)
A handwashing station comprised of a bucket with a fitted tap (40 L near the latrine and 16 L near the cooking area), a soapy-water bottle (Hulland et al. ) and free bi-weekly supply of detergent for soapy-water preparation (Fig. 1c)
Station for latrine and kitchen: 4320
Spot check for handwashing location near the toilet and the cooking area, and for the presence of water and soap (65% with at least one with soap and water present)
Promotion of exclusive breastfeeding up to 6 months of age, complementary feeding and supplementation using lipid-based nutrient supplements (LNS; Nutriset, France) for children aged 6–24 months (Fig. 1d).
Total sachets distributed: 1,579,900
Reported messages on infant/child nutrition and/or lipid- based nutrient supplements (LNS) local name; Sonamoni (80%)
Inspection of number of sachets of Sonamoni for children > 6 months of age, consistent with use of 2 sachets per day (70%)
Among households in the individual and combined sanitation arms, a sanitation team (comprising a research scientist, sanitation engineers from the implementing agency, village education resource center (VERC) and field attendants) installed study-provided latrines, independent of other intervention hardware delivery, with the first latrine installed in December, 2012 and the final in February, 2014. The latrine model, referred to as “double pit” (alternatively named dual pit, twin pit, two-pit, Fossa Alterna) is a traditional improved pit latrine that comprises two unconnected pits; one in use and one for which households transfer the above-ground (super) structure when the first pit is full . This model uses a water seal (siphon), leading from the latrine to the pit that prevents odors from coming from the pit below into the toilet area (Fig. 2). The installation process involved multiple quality assurance steps. Latrine components were manufactured by pre-approved suppliers that met project-set standards of production; a subset of components was checked for compliance by breaking 5% of the concrete rings, slab and lid purchased for the project with a hammer. We checked the metal reinforcement of the same components by using metal detectors. If any evaluated component failed the quality check, the entire lot was rejected. The sanitation team used a checklist to verify complete latrine installation.
Development of fidelity criteria and critical benchmarks
Data collection and analysis
We administered surveys and spot checks during unannounced fidelity assessment visits every month, from November, 2012, 2 months after intervention delivery commenced, and for the next 6 months. We surveyed more intensely in the first 4 months, all intervention households from four randomly selected blocks per month, of the 10 initial trial blocks where the interventions had already been delivered (n = 8 × 6 × 4 households). We surveyed four randomly selected households per intervention cluster from eight randomly selected blocks per month, of 20 trial blocks where the interventions had already been delivered (n = 4 × 6 × 8 households), in the remaining months. The sample size for assessing intervention fidelity was calculated based on feasibility and convenience, no statistical method was applied. We randomly selected the households each month to ensure representativeness. Measures were made more intensely in the early period because WASH Benefits was a large complex intervention with inevitable deviations from optimal implementation. We aimed for early identification of problem to guide corrective measures to avoid deviation in the subsequent intervention blocks.
We conducted both surveys and spot checks; the survey provided a snapshot of the attitudes and behaviors – including thoughts, meaningful opinions, and comments among the target population. Spot checks, conducted in real time, provided objective measures of hardware functionality and use indicating on the ground program operation, and any unexpected developments affecting the program and its objectives.
Field workers collected data on smartphones, using a survey and spot-check form on the presence (double-pit latrine with super structure, slab/pan, sani-scoop and potty for three or more children), functionality (double-pit latrine: superstructure is in good condition, slab/pan is in good condition, feces have not flowed out, functional water seal; potty and sani-scoop: not broken and easily accessible) condition and signs of use (double-pit latrine: wet floor and pan, water in water seal, feces in pan/water seal, pathway to latrine suggest regular use – is clear, without grass or any barrier, water container present; potty: visible sign of feces inside/on the pot/removable pot, sani-scoop: visible sign of feces on the sani-scoop, not broken)of provided technologies, and on reported corresponding behaviors (use of double-pit latrine, use of potty and sani-scoop, disposal practices for child and animal feces) (Table 1). Data collectors checked for completeness and consistency of forms after each visit. Data were stored in a secure server in a SQL database, and checked for validity and data consistency. We calculated frequencies of fidelity measures for each intervention component (Table 1) using STATA 13 (StataCorp LP, College Station, TX, USA).
The data were rapidly assessed and circulated as standardized reports within 30 days of data collection. If any of the monthly fidelity measures were below critical benchmarks, then a rapid response mechanism was triggered. Investigators including anthropologists trained in qualitative investigation and not involved in project implementation reviewed monitoring and process documentation previously filed from routine monthly field staff and CHW meetings in the low-performing areas. They also visited sites, conducted informal discussions with CHWs, their supervisors, field staff, and study participants to identify the underlying problem(s). The findings were summarized and discussed to develop appropriate actions in response. These included additional classroom training and on the job training for CHWs, their supervisors, and field staff, on how to problem-solve to achieve the pre-set benchmark.
Intervention fidelity achievements
Investigation of shortfall in sanitation implementation benchmarks: Latrine maintenance and use
The reasons for which < 80% of household latrines were found with a functional water seal were explored from the third assessment month by staff responsible for latrine installation, the qualitative team, CHWs and their supervisors. The qualitative investigation found that several households intentionally removed or broke the water seals in the newly installed, trial provided latrines (Figs. 2 and 3b). This is because they perceived that the water seals required extra water to flush the fecal contents around the S-shaped bend into the pit below and might fill the pits rapidly, blocking easy flow of feces into the pit, or causing water to splash during defecation. The recipients did not pour an adequate volume of water into the toilet during post-defecation cleansing, and thus failed to dislodge the fecal material completely from latrine pans, leading to low usage rates and poor maintenance (breaking/removing water seals). The new latrines were, therefore, less convenient than previously owned unimproved pit latrines. Furthermore, household members reported that they continued to use their existing latrines out of habit, kept them for visitors, outsiders and neighboring non-recipient households, many lacking a functional water seal (Fig. 3c) despite the availability of trial-provided improved latrines.
The qualitative team worked with CHWs and the latrine installation team to address these issues. In home visits and courtyard meetings, they demonstrated how the siphon/water seal functioned, and stressed the need to pour additional water with force to flush the feces through the water seal. They also discussed the advantage of having a functional water seal, citing reduced bad odors, and fewer flies and mosquitoes coming from the pit while a person was using the toilet. To discourage continued use of previously existing “unhygienic” (lacking functioning water seal) latrines, they were structurally closed with the household’s consent to prevent ongoing environmental contamination.
Investigation of shortfall in sanitation implementation benchmarks: Unsafe child feces disposal
Prior to potty distribution, > 90% children aged < 3 years were habitually defecating in the courtyard (Mahfuza Islam, manuscript in preparation). Qualitative investigation revealed that although potties were provided and promoted according to protocol guidelines, additional support was required to encourage and teach rural mothers how to train their children to consistently use the potty. In addition, CHWs encouraged the mothers to store potties in a place that was accessible to the trained toddler when they need to use it; they encouraged household members other than the mother to participate in the potty training and habit formation. In addition, we provided refresher training to the CHWs to address safe feces disposal into latrines or covered pits, instead of the prevalent practices of disposing feces into the bushes just out of sight.
We have summarized the implementation fidelity system put in place for the WASH Benefits Bangladesh efficacy trial, and provided an example of how systematic fidelity measures can detect compliance and deviations and trigger a rapid course correction. Such careful monitoring is especially essential in complex community-based efficacy studies where high uptake of technologies and behaviors is necessary to achieve the study aims . Deviations in fidelity for large-scale interventions can also be detected through internal monitoring and evaluation, but this often suffers from slow report turnaround and insufficient review for critical feedback. An objective fidelity review with rapid assessment of initial rollout can be done in parallel to implementation process documentation and monitoring, to provide critical input and focus resources for increased efficiency.
Complex interventions tend to have lower fidelity because they are comparatively more difficult to implement than simpler interventions [1, 19]. This is consistent with our findings for the sanitation intervention in the WASH Benefits Bangladesh study that included labor-intensive, multi-stepped latrine installation, close supervision, and quality assurance. The sanitation team installed 4533 latrines, for enrolled households and their neighbors in 21 sub-districts across four districts over 15 months. In our case, fidelity measurements demonstrated that the intervention was implemented as intended by the sixth assessment month; however, this required qualitative investigation and adjustments to the respective behavioral-change package.
Providing enabling products can encourage target behaviors; product quality influences uptake or response among intervention recipients . In our case, households found the project-provided latrine of good quality but less convenient when supplied with a water seal, which some subsequently broke or removed. Breaking or removing water seals has been observed in Bangladesh  and India  and is believed to be common; however, there are few published data on the frequency of this practice. Determining how commonly this occurs could benefit behavior-change efforts in future latrine interventions in South Asia and among low-income countries where this practice may occur. Latrines with broken or missing water seals have a reduced ability to keep excreta separate from the environment and as such are categorized as unimproved .
We did not develop an implementation fidelity score or index to communicate the results in the monthly reports. A numerical index or score calculated for each assessment round might be useful in demonstrating the level of fidelity across combined intervention arms compared to individual intervention arms in a multi-factorial design as in WASH Benefits. However, the specific objective of early fidelity measures was to trigger investigation to address low uptake. A fidelity or implementation index may be relevant for explaining intervention uptake over the 24-month intervention period, to assist interpreting health impact.
In summary, fidelity indicators were efficient, quick to collect, easily observable and indicated the presence, functionality of hardware, condition and sign of use, and corresponding behaviors, which focused on technology accessibility, behavior-change communication crucial for sustained intervention uptake. Only the indicators for child feces disposal and CHW-delivered nutrition messages were reported and not directly observed and were, therefore, potentially overestimated. Fidelity assessment was conducted by only three research assistants who were not directly involved in intervention delivery thus data were generated at relatively low cost. Fidelity indicators described here can be used in other similar intervention trials.
An intensive implementation fidelity monitoring and response system proved beneficial for the WASH Benefits efficacy trial. Timely exploration of benchmark shortfalls enabled corrective measures to ensure that implementation adhered to the intended protocol. The indicators used were derived from easily observable, easy-to-collect data that can be used in other similar WASH and nutrition trials. Flexible adaptations to behavioral interventions at the community level can be directed by time-sensitive fidelity assessments and need to be documented to inform future large-scale studies in similar settings.
Although fidelity assessments are often conducted during the pilot or developmental stages of a study, the integrity of research studies of all designs and stages would benefit from a structured fidelity assessment as a key component [19, 24]. To implement a routine program at scale requires further research into an adaptation of fidelity monitoring that supports program effectiveness.
The authors acknowledge the WASH Benefits study collaborators and acknowledge the cooperation of the study households and communities that were visited several times for intervention delivery and data collection.
This research study was funded by the Bill & Melinda Gates Foundation grant OPPGD759 through the University of California Berkeley. icddr,b acknowledges with gratitude the commitment of the Bill &Melinda Gates Foundation to its research efforts. Icddr,b is also grateful to the Governments of Bangladesh, Canada, Sweden and the UK for providing core/unrestricted support.
Availability of data and materials
Data can be made available through the authors.
MR drafted the manuscript with the listed co-authors. SPL, the principal investigator, drafted the research protocol with input from co-authors and some members of the Technical Advisory Group, and coordinated the study team. The intervention technology and behavior components, and the delivery methods were developed by PW, EL, FB, FH, SA, LU, TC, SMP, AMN, AKMM, and KKD with input from the Technical Advisory Group. All authors provided feedback, read and approved the manuscript.
Ethics approval and consent to participate
All households provided written informed consent at enrollment. The protocol was reviewed and approved by human subjects review committees at the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b) and at the University of California, Berkeley.
Consent for publication
All co-authors have reviewed this version of the manuscript and provided consent for manuscript submission.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 11.Mihalic S, Fertman CI, Snyder SM, Jensen PS. Emotional & Behavioral Disorders in Youth. 2004.Google Scholar
- 12.Arnold BF, Null C, Luby SP, Unicomb L, Stewart CP, Dewey KG, Ahmed T, Ashraf S, Christensen G, Clasen T, et al. Cluster-randomised controlled trials of individual and combined water, sanitation, hygiene and nutritional interventions in rural Bangladesh and Kenya: the WASH benefits study design and rationale. BMJ Open. 2013;3:e003476.CrossRefPubMedPubMedCentralGoogle Scholar
- 13.Unicomb L, Begum F, Leontsini E, Rahman M, Ashraf S, Naser AM, Nizama F, Kannat K, Hussain F, Parvez SM, et al. WASH Benefits Study Bangladesh; management structure for achieving high coverage in an efficacy trial. 2018 (accepted)Google Scholar
- 14.P arvez SM, Azad R, Rahman R, Unicomb L, Ram PK, Naser AM, Stewart C, Jannat K, Rahman MJ, Leontsini E, et al. Achieving optimal technology and behavioral uptake of single and combined interventions in an efficacy trial (WASH Benefits) in rural Bangladesh. 2018. (accepted)Google Scholar
- 15.Hussain F, Clasen T, Akter S, Bawel V, Luby SP, Leontsini E, Unicomb L, Kanti BM, Brittany T, Winch PJ. Advantages and limitations for users of double pit pour-flush latrines in rural Bangladesh. BMC Public Health. 2017;17(1):515.Google Scholar
- 16.Hussain F, Luby SP, Unicomb L, Leontsini E, Naushin T, Buckland AJ, Winch PJ. Assessment of the acceptability and feasibility of child potties for safe child feces disposal in rural Bangladesh. Am J Trop Med Hygiene. 2017;97(2):469-76.Google Scholar
- 18.Hulland KR, Leontsini E, Dreibelbis R, Unicomb L, Afroz A, Dutta NC, Nizame FA, Luby SP, Ram PK, Winch PJ. Designing a handwashing station for infrastructure-restricted communities in Bangladesh using the integrated behavioural model for water, sanitation and hygiene interventions (IBM-WASH). BMC Public Health. 2013;13:877.CrossRefPubMedPubMedCentralGoogle Scholar
- 20.Dreibelbis R, Winch PJ, Leontsini E, Hulland KR, Ram PK, Unicomb L, Luby SP. The integrated behavioural model for water, sanitation, and hygiene: a systematic review of behavioural models and a framework for designing and evaluating behaviour change interventions in infrastructure-restricted settings. BMC Public Health. 2013;13:1015.CrossRefPubMedPubMedCentralGoogle Scholar
- 23.World Health Organization, & UNICEF. Progress on drinking water, sanitation and hygiene: 2017 update and SDG baselines. World Health Organization; 2017. http://www.who.int/mediacentre/news/releases/2017/launch-version-report-jmp-water-sanitationhygiene.pdf. Accessed 19 June 2018.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.