Abstract
Background
Child diarrhea persists as a leading public health problem in India despite evidence supporting zinc and low osmolarity oral rehydration salts as effective treatments. Across 2 years in 2010–2013, the Diarrhea Alleviation using Zinc and Oral Rehydration Salts Therapy (DAZT) program was implemented to operationalize delivery of these interventions at scale through private and public sector providers in rural Gujarat and Uttar Pradesh, India.
Methods/Design
This study evaluates the cost-effectiveness of DAZT program activities relative to status quo conditions existing before the study, comparing a Monte Carlo simulation method with net-benefit regression, discussing the strengths and weaknesses of each approach. A control group was not included in the ‘before and after’ study design as zinc has proven effectiveness for diarrhea treatment. Costs will be calculated using a societal perspective including program implementation and household out-of-pocket payments for care seeking, as well as estimates of wages lost. Outcomes will be measured in terms of episodes averted in net-benefit regression and in terms of the years of life lost component of disability-adjusted life years in the method based on Monte Carlo simulation. The Lives Saved Tool will be used to model anticipated changes in mortality over time and deaths averted based on incremental changes in coverage of oral rehydration salts and zinc. Data will derive from cross-sectional surveys at the start, midpoint, and endpoint of the program. In addition, Lives Saved Tool (LiST) projections will be used to define the reference case value for the ceiling ratio in terms of natural units.
Discussion
This study will be useful both in its application to an economic evaluation of a public health program in its implementation phase but also in its comparison of two methodological approaches to cost-effectiveness analysis. Both policy recommendations and methodological lessons learned will be discussed, recognizing the limitations in drawing strong policy conclusions due to the uncontrolled study design. It is expected that this protocol will be useful to researchers planning what method to use for the evaluation of similar before and after studies.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Background
Worldwide, diarrhea is the fourth leading cause of mortality among children under 5, accounting for 9% of total deaths [1]. In 2011, over 700,000 children died due to diarrhea [2], with eighty percent of cases occurring in East Asia and the Pacific, South Asia, and Africa, and 33% in South Asia alone [3]. However, diarrhea mortality is a solvable health problem, with this number of deaths having fallen from 4.6 million in 1980 [4]. India is a priority area for addressing the remaining burden, recognized as one of 15 countries that account for 53% of total episodes worldwide, with 312.22 million episodes and 205,600 deaths each year nationwide [2].
Since becoming widely used for diarrhea treatment in the 1980s [5], oral rehydration salts (ORS) have been instrumental in contributing to declines in prevalence. ORS prevents mortality by reducing the loss of fluids and electrolytes and death due to dehydration [6]. However, coverage of ORS in India remains low at less than 30%, and one out of ten children nationwide continues to experience diarrhea in any 2-week period [7].
The 2004 United Nations Children’s Fund (UNICEF)/World Health Organization (WHO) Joint Statement for the Clinical Management of Acute Diarrhea revised the global standards for acute diarrhea management to include ‘20 mg per day of zinc supplementation for 10–14 days (10 mg per day for infants under 6 months old)’ [8]. The therapeutic effect of zinc is to strengthen the immune system, improve absorption of water and electrolytes in the intestines, enhance the regeneration of the gut epithelium, increase levels of enzymes in the epithelium, and help the body clear pathogens from the intestines [9]. As a complement to ORS, zinc has been shown to reduce incidence [8], prevalence [10], and duration of diarrhea episodes [8],[9],[11]-[13]. Evidence on whether zinc reduces all-cause mortality is scarce, with one trial using non-injury mortality as a proxy for diarrhea deaths [9], not testing mortality as an outcome [11], or not having sufficient power to detect a significant difference in mortality [13]. Baqui et al. (2002) showed a non-significant difference in non-injury deaths, controlling for other factors, and authors concluded that the effect on mortality was due to zinc [9]. Applying the Child Health Epidemiology Reference Group (CHERG) Rules for Evidence Review indicates that mortality reduction could be as much as 23%, and ideal data, from randomized controlled trials (RCTs), is unlikely to emerge as the strength of evidence in support of zinc makes these trials unethical [14].
In India, evidence on the effectiveness of zinc for the management of acute diarrhea is mixed between studies showing an effect [10],[15],[16], and those with no or marginal effects [17]-[19]. Of these, one was conducted in a rural community setting [10]. This randomized controlled trial was carried out in six primary health care (approximately 30,000 population each centers) in Haryana [10], which provided an intervention that included training and supply of zinc and ORS to Anganwadi Workers and primary health center staff. This intervention nearly doubled the proportion of patients that received treatment between 3 and 6 months. In addition, the intervention led to significant reductions in diarrhea prevalence and the rate of hospitalizations due to diarrhea.
Building upon the success of effectiveness trial activities in Haryana, programmatic efforts to introduce zinc and ORS have been initiated throughout the last decade in India. Between 2005–10, the Point-of-Use Water Disinfection and Zinc Treatment (POUZN) project was implemented and found to be effective in improving both supply and demand for zinc. This program worked with pharmaceutical companies, channeled zinc through both the public and private sectors, collaborated with non-governmental organizations (NGOs), recruited support from key opinion leaders, provided training to detailers, stimulated prescribing practices among rural medical providers (RMPs), and promoted products with social marketing.
In 2010, the Diarrhea Alleviation using Zinc and ORS Therapy (DAZT) program expanded upon POUZN activities in scale and scope to make zinc and ORS available for the management of diarrhea through private and public sector providers in 6 districts of Gujarat and 12 districts of Uttar Pradesh (UP) states of northern India. DAZT is similar to POUZN in the characteristics mentioned above, but differed in that DAZT did not provide point-of-use water interventions, monitored zinc purchase and sales with short messaging service (SMS) in the private sector, and promoted zinc through informational booths in private clinics and hospitals. A full description of DAZT program activities is presented in Table 1.
The objective of this study is to evaluate the cost-effectiveness of the DAZT program under real-world conditions compared to the status quo existing before the intervention was introduced in the study area. Two analytical approaches will be used to meet this objective: calculation of cost-effectiveness using a Monte Carlo Simulation method with patient level data [20], and calculation of cost-effectiveness using a net-benefit regression approach to control for covariates [21].
Both community-based [Lefevre et al. forthcoming, Bishai et al. forthcoming] and model-based evidence [4],[22] suggest that zinc supplementation to treat diarrhea is cost-effective in low- and middle-income countries (LMICs), although hospital-based studies do not find a significant difference in cost [23], effect [19], or both [24] between children receiving zinc and those that do not. These studies were conducted across a wide range of settings; however, none of them can be classified as implementation research. This study will evaluate the cost-effectiveness of the DAZT program and further discourse on approaches to economic evaluation of health programs in low- and middle-income countries. Methodologically, examples of cost-effectiveness analyses evaluating ‘before and after’ studies designs are emerging (for example [25],[26]); however, these types of studies are not well established in the literature. Thought is needed to identify special considerations that may be relevant to before and after study evaluations. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has developed guidelines for using ‘real-world data’ [27], but guidelines for conducting cost-effectiveness analysis (CEA) alongside ‘before and after’ studies do not exist.
Alternative methods for conducting CEA are sometimes used to evaluate a single dataset to make methodological points, assess validity, and test robustness of results according to structural uncertainty [21],[28],[29]. Two of several analytical approaches available for CEA include Monte Carlo Simulation methods with patient level data [20] and net-benefit regression [21]. The latter framework combines established methodologies from economic evaluation and econometrics and has the advantage of being able to adjust for imbalances in confounding variables. Net-benefit regression methodology can be used for randomized [21],[30] and non-randomized studies [31],[32], although the case can be made that it is even more appropriate for non-randomized study designs as covariates are more likely to be unevenly distributed across study arms [33]. Bootstrapping confidence intervals presents the problem of ambiguity in incremental cost-effectiveness ratios (ICERs), although this problem can be circumvented in both methods using the net-benefit statistic, and both net-benefit regression and bootstrapping can be used to generate cost-effectiveness acceptability curves (CEACs). The bootstrap approach offers one advantage in that uncertainty around program costs can be simulated by resampling from parametric distributions fit to the data, while program costs are divided evenly across patients within each arm in the net-benefit regression approach [21] or are assumed to cancel out across study arms or phases [31]. Deterministic calculations and net-benefit regression are commonly evaluated in tandem in the literature [21],[31],[32], but scope exists to add thought on comparisons between bootstrapping and net-benefit regression. It is anticipated that documenting thought behind choosing between these analytic frameworks will be useful to researchers facing similar questions about evaluation of programs with similar study designs.
Policy recommendations from the completed research will add to the growing literature evaluating the cost-effectiveness of zinc for diarrhea treatment in low- and middle-income countries. Findings will be timely as India is developing a Health Technology Assessment (HTA) program in collaboration with the UK National Institute of Health and Care Excellence [34], in which the importance of CEA in policy making can be expected to increase. In addition, local capacity to produce zinc is rapidly expanding including Bharat Immunologicals and Biologicals Corporation Limited (BIBCOL), which is a government-owned corporation with the capacity to produce 240 million of 20 mg scored tablets of zinc sulfate per year [35]. Currently, about 40 private companies are producing zinc for local and international markets.
This article presents the DAZT program in India. Analytic methods for evaluating cost-effectiveness are described and compared, in addition to methods for assessing wealth quintile using principal components analysis. Strengths and limitations of the study are discussed, followed by arguments against targeting zinc according to subgroup, and considerations necessary for drawing conclusions about generalizability.
Methods
Setting and study design
Gujarat and Uttar Pradesh are states in northwest and north India, respectively, which, along with Madhya Pradesh, contribute to half of the diarrhea burden in India. The majority of these states are rural, access to basic sanitation is limited, and most adults have only primary education or less; although signs exist that these states are progressing through the ‘demographic transition’ [36],[37]. In 2010, the DAZT program was introduced through both the public and private sectors in 6 districts of Gujarat and 12 districts of Uttar Pradesh. Care seeking for diarrhea was mostly from private doctors, and shifts in use of the health system were seen from private hospitals to community health workers. Data were collected on the last episode affecting the youngest child in each household surveyed Given that the intervention is implemented on a system-wide level, and the effectiveness of zinc has been established for diarrhea treatment [14], a ‘before and after’ study design was chosen [38]. This evaluation compares the costs and health outcomes associated with the DAZT intervention after 2 years of implementation (Table 1) with those that preceded the intervention. This time frame was considered appropriate as it represented conditions after which the program was fully implemented. Outcome indicators include caregiver knowledge, incidence of diarrhea, treatment seeking, source of care, use of zinc, cost of treatment, and cost-effectiveness.
Sample size and power calculation
The sample size for the main effects study was calculated according to standard methods for rates calculated through cluster randomized designs. Calculations used ORS coverage rates in the initial survey since very few children received zinc before the start of the program, and delivery of ORS and zinc were assumed to be linked. Actual sample sizes achieved for each survey are presented in Table 2. Using sample sizes from the main study, power to detect meaningful differences will be calculated for our cost-effectiveness estimate. Two formulas from Glick et al. [39] will be merged to determine power for cost-effectiveness using a novel approach to calculating the reference case ceiling ratio from an estimate based on per capita gross national income (GNI) and Lives Saved Tool (LiST) outputs of deaths averted/episodes averted.
Sampling and data collection
Data were collected through closed-ended survey questionnaires administered to a sample of caregivers of children 2–59 months old. In each survey, information was elicited on caregiver’s knowledge of diarrhea management, illnesses in the past 2 weeks, care seeking, treatment received, wealth quintile, and demographic characteristics of the household. Data on economic costs will be collected prospectively throughout implementation to capture program and incremental users' costs.
Program costs will be based on the financial records of the international NGOs working with public and private sector partners. To enhance generalizability, capital costs of furniture and equipment will be based on the original price paid inflated to 2014 and annualized using the full lifespan of the item and a 3% discount rate [20]. Donated labor, equipment, and facilities will be valued according to market prices; transfer payments such as taxes and subsidies will be removed; and rent costs will be used to represent building costs [40]. Since results will be presented in US$, it will not be necessary to adjust non-traded items to correct for price distortions [41]. Users' cost will be derived from reported out-of-pocket payments for care seeking and wages lost.
Conceptual framework
A conceptual framework has been developed to identify important model variables based on categories from Andersen and Newman [42]. This framework consists of three main categories of interacting factors that are associated with treatment seeking including predisposing, enabling, and need factors; as indicated in Figure 1. The model was limited to only children that sought care for diarrhea, although other symptoms associated with diarrhea were assessed in caregiver surveys. If treatment was sought and given, economic costs could result, which are an important component of cost-effectiveness. In our framework, knowledge differs from the concept described in Andersen and Newman [43] in that our data describes awareness of the treatments, where they consider values and beliefs about treatment. All of these variables were justified by literature, and will be included regardless of whether their coefficients in our models are significant to ensure that variances are correctly estimated.
Conceptual framework based on Andersen and Newman [[42]].
Descriptive statistics
Descriptive statistics of the parameters (Table 3) will be calculated according to their means, confidence intervals, and p-values reflecting the extent of difference between arms. Comparisons will be made at the start and endpoint of the program. Stata svy commands will be used to account for clustering with F-tests used to evaluate the significance of differences between groups. As costs are not normally distributed, significance of differences between components listed in Table 4 at the start and end of the program will be calculated using confidence intervals with bootstrapped standard errors.
Costing
The cost of the DAZT program from 2011 to 2013 (Gujarat) and 2014 (Uttar Pradesh) will be calculated from the societal perspective according to Saving Newborn Lives guidelines and standard textbooks [20],[40],[43]. The societal perspective reflects the incremental costs incurred by implementing agencies, government providers, and households. Program costs will be divided into capital costs and recurrent costs, with data derived from program records, primary sources, and interviews with individuals responsible for implementation. Capital costs will be annualized and discounted according to a rate of 3% according to the standard rate for economic evaluation in International Health [44]. Consumer price indices from the International Monetary Fund (IMF) will be used to inflate costs to 2014. Costs will be converted to US$ using exchange rates from OANDA.com. Incremental government provider costs will be derived using reported estimates of time spent on the provision of diarrhea treatment services by public sector providers. Economic costs to caregivers for diarrhea treatment of the episodes in the last 2 weeks will be evaluated according to data collected through household questionnaire and consist of direct medical costs of treatment, direct non-medical costs (transportation), and indirect costs (wages lost by caregivers) [45]. No adjustments will be made to collected data to approximate opportunity costs.
Health outcomes
In the standard cost-effectiveness calculation, health outcomes will be derived from incremental changes in coverage measured in cross-sectional household surveys and inputted into LiST. LiST will estimate the incremental number of lives saved from the study’s start to its completion. The years of life lost (YLL) component of disability-adjusted life years (DALYs) will be calculated using the standard formula from the Global Burden of Disease study [46]. A life expectancy estimate will be taken from WHO life tables representing the midpoint between 1–4 years old. A discount rate of 3% will be used in reference case calculations, and age weighting will not be used to be consistent with the Disease Control Priorities Project second edition (DCP2) [41]. Long-term sequelae such as acute cognitive and psychomotor development effects in young children, the effects of diarrhea on stunting [4], and obesity and related conditions that can occur later in life (cardiovascular disease and diabetes) [47] were not measured and will not be considered. The net-benefit regression approach will be limited to episodes averted as it is impossible to know the specific long-term health outcomes for individual patients. Limitations of drawing data from a single uncontrolled study are recognized [48], although cost-effectiveness analysis is often conducted on data from a single effectiveness study, and conditions in the study sites are not atypical of other relevant settings in low- and middle-income countries.
Cost-effectiveness analysis
Cost-effectiveness will be calculated using two different approaches including 1) Monte Carlo Simulation with patient level data [20] and 2) a net-benefit regression-based approach [21].
Monte Carlo Simulation with patient level data approach to cost-effectiveness analysis
Initial cost-effectiveness calculations will be generated through Monte Carlo simulation to evaluate uncertainty in economic costs to caregivers and health outcomes, using an additional simulation technique to account for the uncertainty in program costs. Samples of economic costs to caregivers and health outcomes will be drawn from the data equal in size to the number of data points, with replacement. Uncertainty around program costs will be simulated by drawing data from gamma distributions derived from most likely, low and high estimates for each component. This process will be repeated 10,000 times to ensure that tails of standard error distributions are filled and increase stability in results since available computing power is sufficient. ICERs will be calculated, and uncertainty will be quantified using a bootstrap approach, choosing the values representing the upper and lower 2.5% of data. To disambiguate negative ICERs and show the probability of cost-effectiveness according to different values of the ceiling ratio, CEACs will be calculated, using per capita GNI as the reference case threshold [49] (US$1,740 in Gujarat, $571 in Uttar Pradesh). Calculations will be performed using Microsoft Excel with the Palisade at-risk add-in.
Net-benefit regression-based approach to cost-effectiveness analysis
A second approach to calculating cost-effectiveness will be taken according to net-benefit regression methods to control for imbalances in covariates arising from the non-randomized study design [21],[31],[32]. This method uses patient-level data of economic costs to caregivers and health outcomes, distributing program costs from the last year of the program evenly across children in the endpoint survey. A modification of the net-benefit statistic will serve as the dependent variable, given that outcomes are measured in terms of a health gap measure instead of health expectancy measure [50]. This formulation adds costs to outcomes expressed in monetary terms instead of taking the difference as is done in standard net-benefit calculations. The coefficient on a treatment variable that distinguishes between the starting point and endpoint of the program will be used to represent incremental net benefit. One sided p-values from the treatment variable will be used to construct CEACs, using results from a series of regressions that use different values for the ceiling ratio. Due to the non-normality in the distribution of residual values, a generalized linear model using a gamma distribution will be used. Huber-White robust standard errors will be used to address heteroskedasticity, and clustering will be accounted for consistent with study design. Data will be checked for non-linearities to assess whether use of a spline term improves model fit. The importance of influential data points will be checked using DFBETAs, excluding caregivers with the top ten DFBETA scores in sensitivity analysis. Calculations will be performed in Stata 13, using standards for good practice for using statistical regression models in economic evaluations [51].
Principal components analysis
Principal components analysis will be used to categorize caregivers according to wealth quintile using established methods [52],[53]. While no established rules exist for selecting variables, broad categories of assets will include durable asset ownership, housing characteristics, and access to basic services based on precedent [52]. To avoid illogical rankings [54], only those variables with a prevalence between 5%–95% will be retained. While these thresholds are arbitrary, they are consistent with rules of general inference to define a value with low probability. Eigenvalues will be generated in Stata, which represent linear combinations of variables that capture the maximum amount of remaining uncertainty in the data for each component. The component with the greatest eigenvalue will be selected for creating a wealth index, as higher order components have been shown not to be important in a previous study and convention has become to create wealth indices on only one component [52]. Factor loadings will be assessed to determine what variables align the most closely with the principal component and evaluate whether its representation of wealth has face validity. Caregivers will be divided into quintiles, and variables for regression analyses will be based on these quintiles. The scale will be validated according to a Cronbach’s alpha greater than 0.6.
Ethical approval and study status
For the main study, ethical approval was obtained from SAS and the Johns Hopkins Bloomberg School of Public Health (JHSPH) Institutional Review Board (IRB). DAZT project activities commenced in late 2010 and will span through 2014.
Discussion
Economic evaluation provides insights into the appropriate allocation of resources, and yet the number of cost-effectiveness and cost-utility analyses alongside programs at scale - where large amounts of resources are invested - remains limited. When planning these studies, differences in methods adopted have implications on the validity of findings and on their generalizability. Economic evaluation has traditionally focused on the cost of an intervention to produce a desired health outcome in studies based on modeled evidence or data collected alongside randomized controlled trials. As the number of interventions with known impact on health increases, there is an increased need for evidence on ways of operationalizing delivery of these health services to ensure high coverage, quality, and cost-effectiveness.
The aims of the DAZT intervention are to improve the supply, demand, and prescribing practices for treatment of diarrhea in children with zinc and ORS. These objectives are consistent with goals of The Indian Academy of Pediatrics [55], WHO [56], and UNICEF [3] who have all endorsed zinc supplementation to ORS for the treatment of diarrhea in children. In addition, the Government of India has established the Oral Rehydration Therapy Program to increase awareness about causes of diarrhea and its treatment among mothers and communities [7], and results of this study may be used to support this program.
This study will provide an evaluation of a zinc delivery strategy in a resource-constrained setting, in addition to a comparison of methods for cost-effectiveness analysis. The calculation of results using two different methods will provide evidence about the robustness of results, while informing broader discourse on methods for carrying out economic evaluations alongside programs delivered at scale. Each method has its strengths and limitations. The Monte Carlo Simulation with patient level data approach will allow for the incorporation of uncertainty around program costs and is a more widely used technique than the net-benefit regression approach. The net-benefit regression framework will allow for cost-effectiveness to be calculated controlling for other factors, which are unbalanced across arms in this study. Comparing the two analytical approaches will stimulate discussion about when analysts should choose one over the other when planning their analyses, particularly in the context of evaluation of programs at scale.
Strengths and limitations
The main strength of this study is that it provides two different analytic perspectives on cost-effectiveness, allowing assessment of the robustness of results. It’s time frame is comparable to other community-based studies, although the full effect may not be captured as knowledge takes time to fully proliferate, supply systems mature through time, and changes in prescribing patterns can be a gradual process. A heterogeneous mix of patients will be included, enhancing the ability to generalize findings to other areas in India. Recall bias is minimized by asking only about episodes occurring in the last 2 weeks, in keeping with precedent from a previous study [57]. Overall, the quality of the data is high, with interviewers returning to households to fill gaps in missing data.
The main limitation of this study is that the study design of uncontrolled before and after studies is vulnerable to bias, such as secular trends, the Hawthorne effect [48], or regression to the mean [58]. However, the case for clinical effectiveness can be made based on evidence from randomized studies that have found reductions in the prevalence of diarrhea when providers were enabled to give zinc to children with diarrhea [10]. A CHERG review concluded that zinc causes a 23% reduction in diarrhea-specific mortality based on reductions in hospitalization rates, reasoning that this estimate was realistic as it was more conservative than reductions in all-cause or diarrhea-specific mortality found in other studies [14]. This evidence is consistent with the 2004 WHO/UNICEF endorsement of zinc for treating child diarrhea, which concluded that zinc reduces the severity and duration of acute diarrhea, and the number of episodes in the 2–3 months after treatment [8]. Since equipoise existed for the DAZT delivery strategy, but not for the effectiveness of zinc, the before and after study design was deemed to be the appropriate form of evaluation [59]. The intervention has since been rolled out statewide in Gujarat, and the final survey is underway in Uttar Pradesh.
Secular trends such as improvements in living conditions, access to safe water, and improved sanitation may have contributed to the decline in diarrhea prevalence in the DAZT study area [60]. However, secular trends may be less relevant for costs as economic growth is unlikely to drive prices for diarrhea treatment down. In addition, this study design is sensitive to sudden changes in the conditions of the study area [61]. The Hawthorne effect is a potential confounder, which would overstate the magnitude of the effect [48]. Regression towards the mean may be problematic [58], which means that if a study had high diarrhea prevalence at beginning, future measurements will be likely to be less extreme. The DAZT study found a 14% prevalence of diarrhea in 2011, while levels were 13% in 2005/6 in Gujarat [36] and were 9% nationwide [7]. Regression to the mean predicts that the prevalence would be lower in future surveys as downward changes would be more frequent than upward changes.
Limitations exist in that household data is based on self-report, reporting bias could have occurred, and long-term outcomes and programmatic outcomes such as quality are not measured. Some researchers argue that only children who receive the recommended 10–14 day course should be included in analysis. However, in the DAZT program, most children only received partial courses of zinc in addition to antibiotics and anti-diarrheal medications. Seasonal effects are possible given that each survey was given at a different point in the monsoon cycle across the years of the study. Limitations of principal components analysis are recognized, such as not evaluating the quality of assets, assuming the first principal component to be an adequate indicator of socioeconomic status, being a relative measure, and producing quintiles with variable arrays of asset ownership [53],[62],[63]. Finally, outcomes will not be evaluated according to severity, which could lead to outcomes such as children with fewer symptoms or shorter episodes having more favorable cost-effectiveness.
Targeting interventions according to subgroup
Cost-effectiveness according to subgroup can be assessed in the net-benefit regression framework as the coefficients on interaction terms between specific variables and the treatment variable. However, the case exists that zinc for diarrhea should not be targeted as it does not pose the threat of microbial resistance that is relevant to antibiotics and antimalarial medicines [64]. If widespread use of zinc reduces the use of antibiotics, microbial resistance to them may be slowed. In addition, zinc incurs only modest drug costs per child, and the costs of rationing treatment may outweigh the benefits of universal coverage [65]. In addition, universal coverage may be justified on equity grounds — while diarrhea affects people regardless of socioeconomic status, mortality preferentially affects the poor. Making treatments available according to patient subgroups is justified according to treatment effect, although is not justified according to demographic characteristics such as race [66]. However, information on key factors affecting effectiveness, such as or level of zinc status, were not collected. In addition, the data structure produces counterintuitive results as evaluating number of episodes treated at different service providers implies episodes averted when treatment seeking is reduced. For these reasons, cost-effectiveness according to subgroup will not be emphasized in this analysis.
Generalizability
Results may be transferable to countries with similar health systems and epidemiological profiles. Because of differences in costs and similarities in efficacy of zinc across settings, the case has been made that cost-effectiveness is likely to vary more than effectiveness [22]. However, considerable heterogeneity in effectiveness has been found across settings, even within Asia [13], and results may be limited in their generalizability in this dimension as well. Other factors that may affect generalizability include ages of children, duration of illness, health systems infrastructure, receptiveness of health workers to changing treatment practices, and use of other medicines to treat diarrhea. Methodological assumptions may need adjustment as well, including the discount rate and study perspective. Descriptive statistics about the study setting will be presented to facilitate discussion about generalizability.
Abbreviations
- BMGF:
-
Bill and Melinda Gates Foundation
- CEA:
-
cost-effectiveness analysis
- CEAC:
-
cost-effectiveness acceptability curves
- CHAI:
-
Clinton Health Access Initiative
- DALY:
-
disability-adjusted life years
- DAZT:
-
Diarrhea Alleviation using Zinc and ORS Therapy
- DCP2:
-
Disease Control Priorities Project second edition
- FHI-360:
-
Family Health International-360
- GNI:
-
gross national income
- HTA:
-
Health Technology Assessment
- ICERs:
-
incremental cost-effectiveness ratios
- IMF:
-
International Monetary Fund
- IRB:
-
Institutional Review Board
- ISPOR:
-
International Society for Pharmacoeconomics Research
- JHSPH:
-
Johns Hopkins Bloomberg School of Public Health
- MI:
-
micronutrients initiative
- ORS:
-
oral rehydration salts
- PHCs:
-
primary health centers
- POUZN:
-
Point-of-Use Water Disinfection and Zinc Treatment program
- UNICEF:
-
United Nations Children’s Fund
- UP:
-
Uttar Pradesh
- WHO:
-
World Health Organization
- YLL:
-
years of life lost
References
UN: Levels and trends in child mortality: Report 2014, Estimates developed by the UN inter-agency group for child mortality estimation. 2014, United Nations, Geneva
Fischer-Walker CL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, O'Brien KL, Campbell H, Black RE: Global burden of childhood pneumonia and diarrhoea. Lancet. 2013, 381: 1405-1416. 10.1016/S0140-6736(13)60222-6.
UNICEF: Diarrhoea: why children are still dying and what can be done. 2009, UNICEF/WHO, Geneva
Keusch GT, Fontaine O, Bhargava A, Boschi-Pinto C, Bhutta ZA, Gotuzzo E, Rivera J, Chow J, Shahid-Salles S, Laxminarayan R: Diarrheal diseases. Disease control priorities in developing countries . Edited by: Jamison D, Breman J, Measham A, Alleyne G, Cleason M, Evans DB, Jha P, Mills A, Musgrove P. 2006, The World Bank and Oxford University Press, Washington DC, 371-387. 2
Taylor CE, Greenough WB: Control of diarrheal diseases. Annu Rev Public Health. 1989, 10: 221-244. 10.1146/annurev.pu.10.050189.001253.
Munos MK, Walker CLF, Black RE: The effect of oral rehydration solution and recommended home fluids on diarrhoea mortality. Int J Epidemiol. 2010, 39: i75-i87. 10.1093/ije/dyq025.
NFHS-3: National family health survey 3, 2005-6. 2007, International Institute for Population Sciences, Mumbai
WHO: Clinical management of acute diarrhoea: WHO/UNICEF joint statement. 2004, World Health Organization, Geneva
Baqui AH, Black RE, El Arifeen S, Yunus M, Chakraborty J, Ahmed S, Vaughan JP: Effect of zinc supplementation started during diarrhea on morbidity and mortality in Bangladeshi children: community randomised trial. BMJ. 2002, 325: 1059-10.1136/bmj.325.7372.1059.
Bhandari N, Mazumder S, Taneja S, Dube B, Agarwal R, Mahalanabis D, Fontaine O, Black RE, Bhan MK: Effectiveness of zinc supplementation plus oral rehydration salts compared with oral rehydration salts alone as a treatment for acute diarrhea in a primary care setting: a cluster randomized trial. Pediatrics. 2008, 121: e1279-e1285. 10.1542/peds.2007-1939.
Lazzerini M, Ronfani L: Oral zinc for treating diarrhoea in children. Cochrane Database Syst Rev. 2008, 3: 1-65.
Haider BA, Bhutta ZA: The effect of therapeutic zinc supplementation among young children with selected infections: a review of the evidence. Food Nutr Bull. 2009, 30: 41S-59S.
Lazzerini M, Ronfani L: Oral zinc for treating diarrhoea in children. Cochrane Database Syst Rev. 2013, 3: 1-85.
Fischer-Walker CL, Black RE: Zinc for the treatment of diarrhoea: effect on diarrhoea morbidity, mortality and incidence of future episodes. Int J Epidemiol. 2010, 39: i63-i69. 10.1093/ije/dyq023.
Dutta P, Datta A, Niyogi S, Dutta S, Manna B, Basak M, Mahapatra T, Bhattacharaya S: Impact of zinc supplementation in malnourished children with acute watery diarrhoea. J Trop Pediatr. 2000, 46: 259-263. 10.1093/tropej/46.5.259.
Dutta P, Mitra U, Dutta S, Naik TN, Rajendran K, Chatterjee MK: Zinc, vitamin A, and micronutrient supplementation in children with diarrhea: a randomized controlled clinical trial of combination therapy versus monotherapy. J Pediatr. 2011, 159: 633-637. 10.1016/j.jpeds.2011.03.028.
Bhatnagar S, Bahl R, Sharma PK, Kumar GT, Saxena S, Bhan MK: Zinc with oral rehydration therapy reduces stool output and duration of diarrhea in hospitalized children: a randomized controlled trial. J Pediatr Gastroenterol Nutr. 2004, 38: 34-40. 10.1097/00005176-200401000-00010.
Patel A, Dibley MJ, Mamtani M, Badhoniya N, Kulkarni H: Zinc and copper supplementation in acute diarrhea in children: a double-blind randomized controlled trial. BMC Med. 2009, 5: 22-10.1186/1741-7015-7-22.
Patel AB, Badhoniya N, Dibley MJ: Zinc and copper supplementation are not cost-effective interventions in the treatment of acute diarrhea. J Clin Epidemiol. 2013, 66: 52-61. 10.1016/j.jclinepi.2012.06.012.
Drummond MF, Sculpher M, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the Economic Evaluation of Health Care Programs. 2005, Oxford University Press, Oxford, 3
Hoch JS, Briggs AH, Willan AR: Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ. 2002, 11: 415-430. 10.1002/hec.678.
Robberstad B, Strand T, Black RE, Sommerfelt H: Cost-effectiveness of zinc as adjunct therapy for acute childhood diarrhoea in developing countries. Bull WHO. 2004, 82: 523-531.
Gregorio GV, Dans LF, Cordero CP, Panelo CA: Zinc supplementation reduced cost and duration of acute diarrhea in children. J Clin Epidemiol. 2007, 60: 560-566. 10.1016/j.jclinepi.2006.08.004.
Patel AB, Dhande LA, Rawat MS: Economic evaluation of zinc and copper use in treating acute diarrhea in children: a randomized control trial. Cost Eff Resour Alloc. 2003, 1: 7-10.1186/1478-7547-1-7.
Tan-Torres Edejer T, Aikins M, Black R, Wolfson L, Hutubessy R, Evans DB: Cost effectiveness analysis of strategies for child health in developing countries. BMJ. 2005, 331: 1177-10.1136/bmj.38652.550278.7C.
Borghi J, Mayumana I, Mashasi I, Binyaruka P, Patouillard E, Njau I, Maestad O, Abdulla S, Mamdani M: Protocol for the evaluation of a pay for performance programme in Pwani region in Tanzania: a controlled before and after study. Implement Sci. 2013, 8: 1-12. 10.1186/1748-5908-8-80.
Garrison LP, Neumann PJ, Erickson P, Marshall D, Mullins CD: Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health. 2007, 10: 326-335. 10.1111/j.1524-4733.2007.00186.x.
Karnon J: Alternative decision modelling techniques for the evaluation of health care technologies: Markov processes versus discrete event simulation. Health Econ. 2003, 12: 837-848. 10.1002/hec.770.
Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD: Model parameter estimation and uncertainty: a report of the ISPOR-SMDM modeling good research practices task force-6. Value Health. 2012, 15: 835-842. 10.1016/j.jval.2012.04.014.
Hoch JS, Rockx MA, Krahn AD: Using the net benefit regression framework to construct cost-effectiveness acceptability curves: an example using data from a trial of external loop recorders versus Holter monitoring for ambulatory monitoring of. BMC Health Serv Res. 2006, 6: 68-10.1186/1472-6963-6-68.
Hounton S, Newlands D: Applying the net-benefit framework for analyzing and presenting cost-effectiveness analysis of a maternal and newborn health intervention. PLoS One. 2012, 7: e40995-10.1371/journal.pone.0040995.
Hounton S, Newlands D: Applying the net-benefit framework for assessing cost-effectiveness of interventions towards universal health coverage. Cost Eff Resour Alloc. 2012, 10: 8-10.1186/1478-7547-10-8.
Cousens S, Hargreaves J, Bonell C, Armstrong B, Thomas J, Kirkwood B, Hayes R: Alternatives to randomisation in the evaluation of public-health interventions: statistical analysis and causal inference. J Epidemiol Community Health. 2009, 65: 576-581. 10.1136/jech.2008.082610.
Hass B, Pooley J, Feuring M, Suvarna V, Harrington AE: Health technology assessment and its role in the future development of the Indian healthcare sector. Perspect Clin Res. 2012, 3: 66-10.4103/2229-3485.96449.
Gitanjali B, Weerasuriya K: The curious case of zinc for diarrhea: Unavailable, unprescribed, and unused. J Pharmacol Pharmacother. 2011, 2: 225-229. 10.4103/0976-500X.85933.
NFHS-3: National family health survey 3 (NFHS-3) India, 2005-6 Gujarat. 2007, International Institute for Population Sciences, Mumbai
NFHS-3: National family health survey 3 (NFHS-3) India, 2005-6 Uttar Pradesh. 2008, International Institute for Population Sciences, Mumbai
Sanders D, Haines A: Implementation research is needed to achieve international health goals. PLoS Med. 2006, 3: e186-10.1371/journal.pmed.0030186.
Glick HA: Sample size and power for cost-effectiveness analysis (Part 1). Pharmacoeconomics. 2011, 29: 189-198. 10.2165/11585070-000000000-00000.
SNL: Saving Newborn Lives Initiative: Project Costing Guidelines. 2004, Save the Children, Washington DC
Musgrove P, Fox-Rushby J: Cost-effectiveness analysis. Disease Control Priorities in Developing Countries. Edited by: Jamison D, Breman J, Measham A, Alleyne G, Cleason M, Evans D, Jha P, Mills A, Musgrove P. 2006, The World Bank and Oxford University Press, Washington, DC, 2
Andersen R, Newman JF: Societal and individual determinants of medical care utilization in the United States.The Milbank Memorial Fund Quarterly Health and Society 1973:95-124.,
Gold MR, Siegel JE, Russell LB, Weinstein MC: Cost Effectiveness in Health and Medicine. 1996, Oxford University Press, New York
WHO Guide to Cost-Effectiveness Analysis. 2003, World Health Organisation, Geneva
Rheingans R, Kukla M, Faruque ASG, Sur D, Zaidi AK, Nasrin D, Farag TH, Levine MM, Kotloff KL: Determinants of Household Costs Associated With Childhood Diarrhea in 3 South Asian Settings. Clin Infect Dis. 2012, 55: S327-S335. 10.1093/cid/cis764.
Murray CJ, Lopez AD: The global burden of disease: A comprehensive assessment of mortality and disability from diseases, injuries and risk factors in 1990 and projected to 2020. 1996, Harvard University Press, Cambridge
Guerrant RL, DeBoer MD, Moore SR, Scharf RJ, Lima AA: The impoverished gut—a triple burden of diarrhoea, stunting and chronic disease. Nat Rev Gastroenterol Hepatol. 2012, 10: 220-229. 10.1038/nrgastro.2012.239.
Grimshaw J, Campbell M, Eccles M, Steen N: Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract. 2000, 17: S11-10.1093/fampra/17.suppl_1.S11.
Shillcutt SD, Walker DG, Goodman CA, Mills AJ: Cost-effectiveness in low- and middle-income countries: a review of debates surrounding decision rules. Pharmacoeconomics. 2009, 27: 903-917. 10.2165/10899580-000000000-00000.
Hyder AA, Puvanachandra P, Morrow RH: Measures of health and disease in populations. International Public Health. Edited by: Merson MH, Black RE, Mills AJ. 2012, Jones and Bartlett Publishers, Sudbury, Massachusetts, Second
Kearns B, Ara R, Wailoo A, Manca A, Alava MH, Abrams K, Campbell M: Good practice guidelines for the use of statistical regression models in economic evaluations. Pharmacoeconomics. 2013, 31: 1-10. 10.1007/s40273-012-0001-x.
Filmer D, Pritchett LH: Estimating wealth effects without expenditure data - or tears: an application to educational enrolments in states of India. Demography. 2001, 38: 115-132.
Vyas S, Kumaranayake L: Constructing socio-economic status indices: how to use principal components analysis. Health Policy Plan. 2006, 21: 459-468. 10.1093/heapol/czl029.
Howe LD, Hargreaves J, R, Huttly SRA: Issues in the construction of wealth indices for the measurement of socio-economic position in low-income countries.Emerging Themes in Epidemiology 2008, 5:doi: 10.1186/1742-7622-1185-1183.,
Bhatnagar S, Bhandari N, Mouli U, Bhan M: Consensus statement of IAP National Task Force: Status report on management of acute diarrhea. Indian Pediatr. 2004, 41: 335-348.
WHO: Implementing the new recommendations on the clinical management of diarrhoea: guidelines for policy makers and programme managers. 2006, World Health Organization, Geneva
Mazumder S, Taneja S, Bhandari N, Dube B, Agarwal R, Mahalanabis D, Fontaine O, Black RE: Effectiveness of zinc supplementation plus oral rehydration salts for diarrhoea in infants aged less than 6 months in Haryana state, India. Bull World Health Organ. 2010, 88: 754-760. 10.2471/BLT.10.075986.
Bacchieri A, Cioppa GD: Experimental design: fallacy of “before-after” comparisons in uncontrolled studies. Fundamentals of clinical research: Bridging medicine, statistics and operations. 2007, Springer, New York, 10.1007/978-88-470-0492-4.
Hyder AA, Pratt B, Ali J, Kass N, Sewankambo N: The ethics of health systems research in low-and middle-income countries: a call to action. Glob Public Health. 2014, 9: 1008-1022. 10.1080/17441692.2014.931998.
Emina JB, Kandala N-B: Accounting for recent trends in the prevalence of diarrhoea in the Democratic Republic of Congo (DRC): results from consecutive cross-sectional surveys. BMJ open. 2012, 2: e001930-10.1136/bmjopen-2012-001930.
Eccles M, Grimshaw J, Campbell M, Ramsay C: Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual. Saf. Health Care. 2003, 12: 47-52. 10.1136/qhc.12.1.47.
Falkingham J, Namazie C: Measuring health and poverty: A review of approaches to identifying the poor. 2002, UK Department of International Development, London
Victora CG, Fenn B, Bryce J, Kirkwood BR: Co-coverage of preventive interventions and implications for child-survival strategies: evidence from national surveys. Lancet. 2005, 366: 1460-1466. 10.1016/S0140-6736(05)67599-X.
Sabot O, Schroder K, Yamey G, Montagu D: Scaling up oral rehydration salts and zinc for the treatment of diarrhoea. BMJ. 2012, 344: e940-10.1136/bmj.e940.
Jha P, Laxminarayan R: Choosing health: An entitlement for all Indians. Center for Global Health Research. Toronto, Canada; 2009.
Grutters JP, Sculpher M, Briggs AH, Severens JL, Candel MJ, Stahl JE, De Ruysscher D, Boer A, Ramaekers BL, Joore MA: Acknowledging Patient Heterogeneity in Economic Evaluation. Pharmacoeconomics. 2013, 31: 111-123. 10.1007/s40273-012-0015-4.
Acknowledgements
The DAZT Program is in partnership between the Micronutrients Initiative, Family Health International-360, UNICEF, Clinton Health Access Initiative (CHAI), the US Fund, and Johns Hopkins Bloomberg School of Public Health that is made possible only through the generous support of the Bill and Melinda Gate’s Foundation (BMGF). The BMGF has no role in the data collection or analysis of this study.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
CFW is the principal investigator of the DAZT Evaluation. AEL and SM are leading economic evaluation activities. CFW led efforts to determine DAZT effectiveness including household and provider surveys. AEL conceived the idea for this paper. SDS and AEL wrote the first draft of this manuscript with editing and proof reading from all other authors. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Shillcutt, S.D., LeFevre, A.E., Walker, C.L.F. et al. Protocol for the economic evaluation of the diarrhea alleviation through zinc and oral rehydration salt therapy at scale through private and public providers in rural Gujarat and Uttar Pradesh, India. Implementation Sci 9, 164 (2014). https://doi.org/10.1186/s13012-014-0164-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13012-014-0164-2