We conducted a systematic literature review to identify economic evaluations of rotavirus vaccine in LMICs. Our search was conducted in November 2015 and updated in February 2017. We included articles published between January 2000 and February 2017. Search terms were broken down into four components, which included terms relating to (1) costs/BIA/CEA/economic evaluation, (2) vaccines, (3) rotavirus and (4) LMICs [see Online Resource 1 of the Electronic Supplementary Material (ESM) for full details on search terms used].
Relevant Medical Subject Headings terms were included where appropriate. We developed and used an LMIC filter based on the most recent World Bank country income classification , expanded to include 25 countries that transitioned from low or middle income to high income from 2000 to 2017. We supplemented our LMIC filter with the Cochrane 2012 LMIC Filters  (Online Resource 2 of the ESM).
We searched the following electronic databases: EconLit, EMBASE (Ovid), MEDLINE (Ovid) and the National Health Service Economic Evaluation Database (through the Cochrane Library). Additionally, we ran basic searches in Research Papers in Economics and the Tufts CEA Registry . We also carried out searches of the first 150 hits in Google and Google Scholar using BIA-specific search terms to identify any unpublished BIAs (Online Resource 1 of the ESM). We supplemented the database searches by conducting manual bibliographic searches from recent and relevant rotavirus vaccine review papers [14, 15, 29, 33,34,35].
Duplicate citations were removed and all remaining papers were screened based on title and abstract. Two reviewers (NC and SC) screened titles and abstracts during the original search conducted in November 2015. One reviewer (NC) screened titles and abstracts identified in the updated search conducted in February 2017. We verified inclusion as a LMIC based on the World Bank’s fiscal year at the time of publication. Two reviewers (NC and SC) read the full text of papers identified as relevant. Non-English language papers were translated to English. Studies included consisted of those self-defined as a BIA or other economic evaluation including a CEA and cost-utility analysis (we refer to both types as CEA from now on). Studies classified as a cost-benefit, fiscal or revenue analysis, or costing studies that did not capture both the costs of rotavirus vaccination and the cost savings owing to reduced disease, were excluded, as were papers tagged as Review, Editorial, Perspective or Discussion pieces. Full inclusion and exclusion criteria are summarised in Table 1. We followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis guidelines and a checklist for the review .
Development of a Budget Impact Analysis (BIA) Checklist
Based on the ISPOR best-practice recommendations , we produced a framework to provide guidance on assessing the quality of BIAs. The BIA Checklist consisted of 15 items divided into four categories: Background, Interventions, Analytic Framework and Results (Table 2). From this framework, we developed a vaccine-specific scoring system to critically appraise papers. Each item was assigned a full (1), partial (0.5) or null (0) score, based on how closely the article met the relevant recommendation. Strict scoring rules were followed for each item, as specified in Online Resource 3 of the ESM, and described in the following sections.
For this category, relevant features of the healthcare system that may influence the budget must be considered. We identified five such features: financing available, budget for vaccines, the country’s decision to introduce the new vaccine, rotavirus disease burden and other relevant healthcare system factors such as availability of infrastructure. The recommended perspective is that of the decision maker or budget holder. Finally, the size of the eligible population must be described and data sources or approaches used to estimate population size explained.
Articles should describe the current mix of interventions and the expected mix after the introduction of the new vaccine. This includes identifying all cost categories relevant to the current mix of interventions, including outpatient cases and hospitalisation. To receive a full score, we specified that costs must be estimated using microcosting. For pragmatic reasons, a study that uses local reimbursement rates (or reference costs) from the country (for example, based on a basic benefits package) was considered equivalent to microcosting. Second, the anticipated uptake and coverage of the new vaccine must be considered. For a full score, the article must discuss where coverage estimates come from, why they are reasonable, and their reason for modelling or not modelling scale-up. Third, all cost categories included in estimating the cost of vaccine introduction should be identified. This includes microcosting of operational delivery and administration costs of the vaccination programme in addition to specifying the vaccine procurement cost. Finally, the impact on healthcare costs should be modelled, including a description of how this was done.
Within this category, we assessed aspects related to modelling choices and data inputs, including stating and justifying a time horizon appropriate to the budget holder, not discounting costs, and providing full details of the model used and input parameters. If an article makes no mention of discounting costs, we assumed no discounting was used. We generated a list of the six most relevant data inputs: demography; estimated vaccine coverage; burden of disease; vaccine efficacy; vaccine-related costs; and other health systems-related costs. Articles were scored based on how much local level data were used to inform these inputs.
Results must be presented as estimates of financial costs at each budget period after the new vaccine was introduced. This final category also assessed the determination of face validity, the presence of uncertainty and scenario analyses, and the article’s main conclusions from a budget impact perspective, including discussion of the main limitations. For these last three sub-sections within the results (validity, uncertainty and scenario analyses, and conclusions), scores do not take into account how well each of these items was done, but rather provides guidance on whether they were done at all.
Development of a Modified BIA Checklist for Cost-Effectiveness Analyses (CEAs)
We modified the BIA Checklist for CEAs by classifying each item as ‘Essential’ or ‘Desirable’ for estimating budget impact. Some items were subdivided into both an ‘Essential’ and ‘Desirable’ component. As a result, the Modified BIA Checklist for CEAs consisted of ten Essential components, and a further 12 Desirable components (Table 2). The scoring system was similarly modified to provide a ‘feasibility score’ to reflect a CEA’s suitability for adaptation for BIA purposes (Online Resource 4 of the ESM). Cost-Effectiveness Analyses were first scored on the Essential criteria, and only articles receiving a full score (6.5 points total) were assessed on the Desirable components (8.5 additional points).
All articles included in the final review were scored based on the BIA Checklist or the Modified BIA Checklist for CEA scoring systems. The maximum score for both checklists was 15 points. Two reviewers (NC and SC) scored the articles independently. Scoring disagreements across any of the items within the checklists were identified and discussed by the two reviewers prior to jointly agreeing on a score. If an agreement was not reached, the conflicting scores were discussed with the broader team (all authors) until a consensus was reached.