Background

Disease surveillance is an essential tool for the development of effective public health and patient care policies. During the last two years, Latin American countries have implemented surveillance of Zika virus (ZIKV) infection, newborn microcephaly, and Guillain-Barré syndrome (GBS) in response to ongoing outbreaks and concerns about possible causal associations between these diseases. In most countries, these systems follow guidelines from international health agencies [1, 2], and are commonly based on the passive detection and report of cases by health care personnel to surveillance units in Ministries of Health. Surveillance reports suggest substantial increases in microcephaly prevalence and GBS incidence following ZIKV outbreaks [3, 4], but increases could be explained by intensified surveillance [5].

ZIKV-microcephaly-GBS surveillance systems have been in place for a short time and there are no data on their performance. We present an assessment of the expected performance of these systems, based on current knowledge and assumptions about disease frequency, accuracy of diagnostic tests, and hypothetical effects of ZIKV infection on microcephaly and GBS risk. Specifically, we evaluated under what conditions would outbreaks of these diseases be identifiable, and what would be the impact of false positive cases detected through surveillance on health care burden. Findings from this study could inform the implementation of surveillance systems in Latin America.

Methods

Overall approach

We evaluated the expected performance of a hypothetical surveillance system in a population of 10 million with a birth rate of 17.3/1000 population. These figures are close to the median population size and fertility rate in Latin American countries. We conducted separate evaluations for ZIKV infection, newborn microcephaly, and GBS surveillance.

We assumed the risk of ZIKV infection during a baseline, non-outbreak period (RZ0), was 10%, and individuals who got infected were no longer susceptible in a subsequent outbreak period [6]. We calculated the ZIKV infection risk during an outbreak (RZ1) as RZ0 × RRZ0 → 1, where RRZ0 → 1 is a hypothetical relative increase in risk, and generated “true” 2 × 2 tables of period by ZIKV infection (Fig. 1 and Additional file 1, item 1). Then, we obtained 2 × 2 tables corrected for misclassification of ZIKV infection status and calculated the expected observed risk ratio (EORR) and the expected observed case ratio (EOCR) comparing the outbreak and non-outbreak period. By progressively increasing it from one to nine, we identified the minimum RRZ0 → 1 resulting in a detectable outbreak. We also calculated the proportion of false positive cases (PFP) identified through surveillance, an indicator of the added health care burden resulting from errors in ZIKV infection diagnosis.

Fig. 1
figure 1

Approach for comparing risk of Zika virus infection and related outcomes during outbreak and non-outbreak period

Full size image

We used the same approach to assess a possible GBS/microcephaly outbreak during a ZIKV infection outbreak with RZ1 = 2 × RZ0. However, we accounted for the number of new cases of ZIKV infection in our calculation of the risk of GBS/microcephaly during the outbreak. For instance, for GBS the risk during the outbreak period (RG1) was calculated as [(RG0 × NZ(−)) + (RG0 × NZ(+) × HRIZ → G)] / (NZ(−) + NZ(+))], where RG0 is the baseline risk of GBS, HRIZ → G is the relative increase in GBS risk among ZIKV infected individuals, and NZ(+) and NZ(−) are the numbers of ZIKV infected and non-infected individuals during the outbreak, respectively. We obtained misclassification corrected 2 × 2 tables and calculated the EORR, EOCR, and PFP for each condition. To find a minimum resulting in a detectable outbreak, we probed for values of HRIZ → G of 1, 5, and 10 for GBS and for values of HRIZ → M of 1, 5, 10 and 15 for microcephaly.

Outcome misclassification-corrected 2 × 2 tables were obtained using standard formulae for the case of non-differential errors [7]. Specifically, if A = number of exposed cases, B = number of non-exposed cases, C = number of exposed non-cases, and D = number of non-exposed non-cases from the “true” 2 × 2 table, then the expected observed number of exposed cases would be [(A × test sensitivity) + (C × (1 - test specificity))]. Also, the expected observed number of non-exposed non-cases would be [(D × test specificity) + (B × (1 - test sensitivity))]. We assumed no changes in case definition or surveillance procedures occurred during the outbreak.

Surveillance guidelines do not specify when to declare an outbreak [1, 8, 9]. We applied the standard criterion of an incidence beyond two standard deviations above the baseline average, the upper limit of the endemic channel, as the cut point to identify an outbreak [10]. In consequence, we declared an outbreak happened if the p-value from a chi square test for the EORR was below 0.0227.

Zika virus infection

The surveillance case definition used in Latin America: “Patient with rash with at least two or more of the following signs or symptoms: fever, usually <38.5 °C, conjunctivitis, arthralgia, myalgia, and peri-articular edema” [1], resembles the definition used by Duffy et al. in a study of an outbreak on Yap Island, Micronesia [11]. Duffy et al. reviewed medical records in all health centers to identify suspected cases and used ELISA tests for IgM antibodies and RT-PCR for ZIKV and dengue virus to confirm the diagnosis. They also conducted a serological survey in a random sample of the population. We used their data and formulae for screening studies with extreme verification bias [12, 13] to estimate the sensitivity and the specificity of the surveillance case definition in three scenarios (Additional file 1, item 2).

In the regular surveillance scenario only cases requesting medical care were detected, similar to Duffy et al.’s study [11], and sensitivity and specificity were 2 and 96%, respectively. The enhanced surveillance scenario was similar to regular surveillance, but we assumed the case detection probability was five times higher in Latin America than in Yap Island [11]. In this case, sensitivity was 9.8% and specificity was 79.7%. In the serological surveillance scenario all suspected cases of ZIKV infection were detected through a survey and infection was confirmed using the same tests used in the Yap Island study [11]. In this case, sensitivity was 37.7% and specificity was 81.1%.

Microcephaly

We conducted separate assessments for all and severe microcephaly, traditionally defined as a head circumference (HC) < 2 and < 3 standard deviations (SD) below the mean, respectively [14, 15]. Under these definitions the prevalences of all and severe microcephaly are 22.75 and 1.35/1000 newborn, respectively. We assumed ZIKV infection increased the risk of microcephaly in the baseline and outbreak period but only in newborn of women infected in that period.

We simulated HC values to estimate accuracy of a diagnosis of microcephaly (Additional file 1, item 3). First, we generated “true” HC values by randomly drawing one million observations from the HC distribution in Brazilian newborn (mean 34.2 cm, SD 1.2) [16]. We obtained random errors from a normal distribution with mean 0 and SD equal to the intra-observer technical error of HC measurements (TEM) [17] in the WHO Multicentre Growth Reference Study [18], and added them to the true values [19]. We cross-classified individuals by applying the definition of all and severe microcephaly to the true (gold standard) and the error augmented values, and calculated the sensitivity and specificity of HC measurements. We also used estimates of sensitivity and specificity for all microcephaly, among low birth weight newborn (≤2000 g) from Bhushan et al. [20], and their TEM values to estimate sensitivity and specificity for severe microcephaly (see Table 2 for values of sensitivity and specificity).

Guillain-Barré syndrome

We used a GBS incidence of 2/100,000 in our calculations. This value was taken from a published review in which incidence ranged from 0.4 to 4 cases/100,000 population in all but two out of 34 studies [21].

GBS diagnostic certainty could be very low, particularly in the early stages of the disease, and there is no clinical characteristic or biomarker that perfectly discriminates GBS from mimicking neurologic disorders [22]. The Brighton criteria is the standard tool to classify GBS cases by diagnostic certainty [23,24,25]. A certainty level ≤ 3, recommended for surveillance case definition, was used in this analysis [1]. We estimated an average sensitivity of 82.1% from three published studies [23, 24, 26] (Additional file 1, item 4) and used specificities of 91.7, 88.9, and 80.6%, from the sole study on this issue, as far as we know [27]. However, we applied this specificity only to individuals with incident peripheral neuropathy, the key clinical feature that most commonly leads to a suspicion of GBS [22, 28, 29], since only they could be falsely diagnosed as GBS cases. We used a random effects model to estimate the average incidence of peripheral neuropathy in published studies [30,31,32,33,34,35,36,37,38,39,40] and used the low and upper limits of its 95% confidence interval in the analysis (3.3 and 5.6/10,000; Additional file 1, item 5).

Results

Zika virus

A minimum RRZ0 → 1 resulting in a detectable outbreak was not identifiable for the scenarios of regular and enhanced surveillance. In both cases, the EORR and the EOCR were always less than one and decreased progressively with increasing RRZ0 → 1 (Table 1; Additional file 1, item 6). For instance, for enhanced surveillance the EORR and EOCR were 0.95 and 0.85 for RRZ0 → 1 = 2 and 0.78 and 0.70 for RRZ0 → 1 = 5, respectively. As expected, the PFP decreased with higher RRZ0 → 1. However, under the scenario of enhanced surveillance the PFP reached 92.3% for RRZ0 → 1 = 2 and 83.5% for RRZ0 → 1 = 5. Thus, for these scenarios, only 1 out of 13 and 1 out of 6 cases detected through enhanced surveillance were true ZIKV infection cases.

Table 1 Expected Observed Risk Ratios and Case Ratios and Proportion of False Positive Cases of Zika Virus Infection During an Outbreak, by Type of Surveillance, Increase in Infection Risk, and Case Definition Sensitivity and Specificity
Full size table

In the scenario of serologic based surveillance, an outbreak was detectable for RRZ0 → 1 ≥ 1.10. In that case, the EORR was 1.01, but the EOCR was 0.91. Moreover, the EOCR was > 1 only for RRZ0 → 1 ≥ 2.4. Even for larger values of the RRZ0 → 1 the EORR and the EOCR were strongly biased towards the null. For instance, for RRZ0 → 1 = 5 the EORR and the EOCR were 1.36 and 1.23, and the PFP was 55%. Thus, even when the risk of ZIKV increased 5-fold during the outbreak, less than half of all cases of ZIKV infection detected through serologic surveillance were true positive cases.

Microcephaly

Outbreaks of all microcephaly were detectable for HRIZ → M ≥ 2.00, regardless of the level of sensitivity and specificity of the diagnosis, but the expected observed prevalence ratio (EOPR) was less than 1.10 in all cases (Table 2; Additional file 1, item 7). In contrast, outbreaks of severe microcephaly were detectable only for HRIZ → M ≥ 24.00, but the EOPR was only 1.48. Even when the HRIZ → M was as high as 15, the EOPR was ≤1.41 for both all and severe microcephaly, regardless of sensitivity and specificity (Tables 3 and 4; Additional file 1, item 8). When ZIKV infection did not increase microcephaly prevalence (HRIZ → M = 1), the PFP for all microcephaly and severe microcephaly ranged from 22.6 to 56.6% and from 34.7 to 82.5%, respectively, depending on the sensitivity and specificity of HC measurements (Tables 3 and 4).

Table 2 Minimum Hypothetical and Expected Observed Prevalence Ratio to Detect an Outbreak of Microcephaly if Zika Virus Infection Risk Doubles, by Sensitivity and Specificity of Microcephaly Case Detection
Full size table
Table 3 Expected Observed Prevalence Ratio of All Microcephaly and Proportion of False Positive Cases During an Outbreak of Zika Virus Infection, by Increase in the Prevalence of Microcephaly in Newborn of Infected Mothers, and Sensitivity and Specificity of Microcephaly Case Detection
Full size table
Table 4 Expected Observed Prevalence Ratio of Severe Microcephaly and Proportion of False Positive Cases During an Outbreak of Zika Virus Infection, by Increase in the Prevalence of Microcephaly in Newborn of Infected Mothers, and Sensitivity and Specificity of Microcephaly Case Detection
Full size table

Guillain-Barré syndrome

Under ideal conditions, when the sensitivity and specificity for detecting GBS cases were both 100%, outbreaks of GBS were detected when HRIZ → G ≥ 4.0 (Table 5; Additional file 1, item 9). However, the minimum HRIZ → G resulting in a detectable outbreak increased for decreasing values of specificity. For the estimated sensitivity of 82.1%, outbreaks were detected if HRIZ → G ≥ 7, HRIZ → G ≥ 8, and HRIZ → G ≥ 9 for specificities of 91.7, 88.9, and 80.6%, respectively.

Table 5 Minimum Hypothetical Relative Increase in Risk (HRIZ → G) to Detect an Outbreak of Guillain-Barré Syndrome and Expected Observed Risk Ratios if the Risk if Zika Virus Infection Doubles During an Outbreaka
Full size table

When ZIKV infection was not associated with GBS risk (HRIZ → G = 1), the PFP varied with specificity, from 34 to 59% and from 50 to 72%, for low and high incidence of peripheral neuropathy, respectively (Table 6; Additional file 1, item 10). Regardless of HRIZ → G value, the misclassification-corrected EORR were small and decreased with decreasing specificity. Indeed, for specificity of 100% and HRIZ → G = 10, the EORR was only 1.47. For the lowest specificity the minimum PFP was 33.8% or higher in all scenarios. For higher values of specificity, the PFP was below 30%, regardless of the HRR, but only if the incidence of peripheral neuropathy was low. When the incidence of peripheral neuropathy was high, the PFP ranged from 24.5 to 61.0%, if HRIZ → G > 1.

Table 6 True and Expected Observed Risk Ratio of Guillain-Barré Syndrome (GBS) During an Outbreak of Zika Virus (ZIKV) Infection, by Hypothetical Increase in GBS Risk Among ZIKV Infected Individuals and by Specificity and Sensitivity of the GBS Case Definition Used for Surveillance
Full size table

Discussion

Our findings suggest surveillance systems for ZIKV, microcephaly, and GBS in Latin America have a limited capacity to detect outbreaks. ZIKV outbreaks were detectable only through serological surveillance. Outbreaks of all and severe microcephaly were detected only when ZIKV infection increased the frequency of these conditions at least two and 24 times, respectively. Outbreaks of GBS were detectable only when GBS risk was at least eight times higher among ZIKV infected individuals. Finally, under most scenarios, cases of ZIKV infection, microcephaly, and GBS were more likely false positive than true positive cases.

While judging these findings, one should carefully consider the validity of study assumptions.

ZIKV infection

We assumed a background risk of ZIKV infection of 10% and a doubling of the risk during an outbreak. A 10% risk corresponds to about half the risk in Brazil [41] and Puerto Rico [42]. A doubling of the risk is also consistent with a lowest limit of 10%, a most likely value of 25%, and an interquartile range of 19 to 33% suggested by Ellington et al. in a review of existing data [43]. Higher levels of baseline ZIKV infection risk would lower the chances of detecting an outbreak, due to a smaller pool of susceptible individuals [6, 44].

Values of sensitivity and specificity of ZIKV infection surveillance case definition were derived from the Yap Island study [11]. However, they are valid in Latin America, regardless of differences in prevalence, because sensitivity applies only to cases and specificity applies only to non-cases of a disease. The specificity of case definition for regular surveillance was high (95%), because it was applied to a self-selected sample of individuals who got medical care. This “pre-screening” improves specificity by increasing the prevalence of ZIKV infection among health care seekers. In contrast, the sensitivity was too low (2%). This is not surprising, because the definition only catches the 37.7% of all cases that are symptomatic [11] and only 11.5% of symptomatic cases seek medical care [45, 46]. Thus, the upper limit of sensitivity is 4.3%.

Due to the poor accuracy of ZIKV infection case definition, regular surveillance could show a decrease in incidence, even if the incidence has increased several fold. This could explain why incidence estimates from surveillance data has been orders of magnitude lower than those from serologic studies [11, 45], simulation studies [41, 47], and previous experience with viruses transmitted by the same vectors [43]. Also, this suggests regular ZIKV infection surveillance may be of little public health benefit.

Serologic surveillance seems the best option for ZIKV infection surveillance in Latin America. If serologic tests used in the Yap Island study were used, outbreaks would be detectable, though most cases would still be false positive. Yearly serologic surveys offer important advantages, compared to regular surveillance. They reduce the PFP; reflect trends in the whole population, instead of just health care seekers; provide needed data on background herd immunity [44]; and generate diagnostic accuracy data useful to correct risk estimates from regular surveillance [7]. Surveys could be conducted in random samples of a few hundred individuals (Additional file 1, item 12). Whether or not surveys are more cost-effective than regular surveillance is a moot point, since outbreaks cannot be detected using the latter. Well-planned serologic testing in random or haphazard samples of patients attending sentinel clinics could have similar benefits.

Unfortunately, better tests are needed for accurate diagnosis of ZIKV infection [48]. Moreover, the PFP cases would increase and outbreaks will be harder to detect if the Centers for Diseases Control’s (CDC) new guidelines to interpret serologic tests are used, as they improve sensitivity, in detriment of specificity [49].

In view of the uncertainty regarding the etiological role of ZIKV infection [5, 50,51,52], and the high PFP cases of ZIKV infection, even with serological testing, we should carefully ponder whether providing possibly infected patients with advice on conception attempts, changes in sexual behavior, and pregnancy outcomes is beneficial, cost-effective, or even ethical [53].

Microcephaly

Due to the lack of published data, we estimated the accuracy of the diagnosis of microcephaly in normal weight newborn by simulation. Large studies of child growth standards [16, 54], support our assumption of a normal distribution of HC values, and TEM values used for simulation came from previous studies [18, 20].

Current surveillance systems target “congenital syndrome associated with Zika virus infection”, instead of all and severe microcephaly [1, 8, 9]. We focused on microcephaly because this is the main component of the postulated syndrome and it was possible to assess the accuracy if its diagnosis. Moreover, most components of the proposed syndrome are common findings in newborns with microcephaly of genetic and infectious origins [55, 56], and there is no evidence they cluster more frequently than expected in newborn from ZIKV-infected mothers [57]. More important, attributing all cases of microcephaly to ZIKV makes little sense. Indeed, about 50% of severe cases have a genetic etiology [14, 58], and 44 to 59% of non-severe cases could be due to low birth weight (Additional file 1, item 12). Moreover, some 10 to 33% of all cases should have maternal ZIKV infection, based on the risk of ZIKV infection [41,42,43], even if there is no association between the two conditions.

In spite of striking differences in prevalence, etiology, prognosis, and management [55, 59], current surveillance systems do not differentiate between severe and non-severe microcephaly [1, 8, 60, 61]. Therefore, surveillance data are of limited value for characterizing disease burden, making decisions about case management, and developing and allocating health resources.

Outbreaks of all microcephaly during a ZIKV infection outbreak will be detectable through regular surveillance if the risk of microcephaly is at least twice higher among infected mothers, but outbreaks of severe microcephaly would be virtually undetectable. Moreover, the PFP for microcephaly will be high, even if maternal ZIKV infection increased microcephaly prevalence by at least 15 times. Such a high prevalence ratio is very unlikely, since evidence from observational and ecological studies in Brazil and Colombia does not support an increased risk of microcephaly in newborn of ZIKV-infected mothers [5, 57].

It is uncertain whether adding non-severe microcephaly to the traditional surveillance of preterm birth and low-birth-weight [14] is justified in terms of its incremental cost-effectiveness to prevent perinatal and infant mortality. Indeed, half the cases of non-severe microcephaly may be due to low-birth-weight, and postnatal nutritional interventions is the cornerstone for the management of both conditions [62,63,64,65]. On the other hand, it is unlikely low-income countries will keep severe microcephaly as a public health priority, in view of the high impact of competing causes of perinatal and infant mortality, such as low birth weight, preterm birth, and neonatal asphyxia. Therefore, strengthening existing perinatal surveillance hospital networks may be the best option for surveillance of severe microcephaly [66, 67].

Guillain-Barré syndrome

We used a GBS incidence of 2/100,000 in our analysis, which is higher than that in seven Latin American countries (1.41/100,000; Additional file 1, item 13) [3]. In addition, we evaluated scenarios where the risk of GBS among ZIKV-infected individuals increased five and 10 times. Such large increases in risk are unrealistic, since evidence from published studies [47, 68] do not support a causal ZIKV-GBS link [5, 50]. In addition, we used all published data to estimate the accuracy of a GBS diagnosis and the incidence of non-diabetic non-GBS peripheral neuropathy. Although these estimates may not strictly apply to Latin America, we believe they are robust enough for an informative assessment of GBS surveillance.

GBS surveillance data could be useful for health planning and program evaluation, but are of little use for outbreak control. In Latin America, GBS surveillance is compounded by the limited or non-existing capacity to perform motor nerve conduction tests, and to draw and analyze cerebrospinal fluid samples for cell count and protein concentration in most clinics. Without these tests, most GBS cases would have Brighton certainty levels ≤3, and a definite diagnosis may not be made until verifying the polyneuropathy was transient, something that could take several weeks [22, 69]. This compromises timely/accurate reporting of GBS cases. Thus, GBS surveillance may add little value to the analysis of data from existing sources [1, 2].

Assuming ongoing GBS surveillance is devoid of potential harm may be risky. As shown in this assessment, surveillance could result in large numbers of false positive cases. Premature declarations of GBS as a complication of ZIKV infection [1, 2], may contribute to misdiagnosing GBS-mimics as GBS cases. This could alter the distribution of scarce health resources in the region, distort health priorities and planning, increase the cost of surveillance activities, and add unnecessary testing, treatment, and morbidity in false positive GBS cases [22].

Conclusions

Our findings suggest it is unlikely that outbreaks of ZIKV infection and putative related outcomes in Latin America will be detected through surveillance systems based on current guidelines [1, 2]. In consequence, it is unlikely these systems would be useful in detecting and curtailing impending or ongoing outbreaks, quantifying burden of disease, identifying factors driving risk, assessing the effectiveness of control measures, or improving clinical care.

In spite of uncertainty in some assumptions and parameters, we believe our findings are sufficiently robust to inform public health policies that, so far, seem mostly supported by questionable causal links and hopes of potential benefits. Policies in response to the ZIKV outbreaks were timely implemented, in a context of limited knowledge about causal links. Unfortunately, knowledge and data about non-causal issues, such as disease burden, diagnostic accuracy, and cost-effectiveness of potential interventions were given little weight when formulating surveillance guidelines. Though they are undoubtedly intended to improve and save lives, ZIKV public health policies should be based on previous experience and scientific knowledge, lest they become irrelevant and harmful for those they meant to protect [70].