Background

Malaria mortality and morbidity have declined significantly in many parts of the world in the last 2 decades [1]. Despite this achievement, the World Health Organization estimated that there were still 438,000 deaths due to malaria in 2015, and that ~70 % of deaths occurred in children less than 5 years old [1]. Recently, Bhatt et al. estimated that 663 million clinical cases have been averted by malaria interventions from 2000 to 2015 in sub-Saharan Africa, largely as a result of insecticide-treated bed nets and indoor residual spraying [2]. However, the emergence of resistance in Anopheles vectors to current insecticides, and shifting behavioural patterns among vector groups are a great concern for future malaria control [3]. In addition, the spread of resistance to artemisinin-based combination therapy among Southeast Asian parasite strains have been reported [4]. Novel interventions are, therefore, important extensions to the current range of control tools, and may be necessary to achieve elimination in many currently endemic areas [5]. Transmission-blocking vaccines that interrupt human-to-mosquito transmission by targeting the sexual, sporogonic, or mosquito stages of the parasite are called SSM-VIMT. SSM-VIMT have the potential to reduce malaria transmission from humans to mosquitoes for whole populations, and could be an important supplement to traditional controls in countries striving for malaria elimination [6–8].

SSM-VIMT are designed to elicit anti-parasite or anti-mosquito antibodies in vaccinees, and the antibodies block parasite development in the mosquito vector when ingested with gametocytes: the sexual-stage, transmissible form of the malaria parasite. While several different assays can be applied for SSM-VIMT development [9], the standard membrane-feeding assay (SMFA) is considered the “gold standard” for determining the impact of test factors on gametocyte infectivity to mosquitoes (either measured by a reduction in oocyst intensity or in prevalence of infected mosquitoes). The SMFA has broad utility, and has been employed to evaluate the functionality of vaccine or whole-parasite induced antibodies in animal studies and human clinical trials [7, 8, 10–12], as well as antibodies induced by natural exposure to malaria infection in endemic settings [13–16]. Furthermore, increasing interest in transmission-blocking drugs has made the SMFA a useful assay for malaria drug development [17–19].

While there are variations in SMFA methodology among different investigators, the assay is generally conducted by feeding a blood meal containing a mixture of cultured Plasmodium falciparum gametocytes and test (or control) antibodies to Anopheles mosquitoes through a membrane-feeding apparatus. Approximately 1 week later mosquitoes from the test and control groups are examined to enumerate the oocyst-forms of parasites that, if present, can be visualized in the epithelium of the mosquito’s midgut by mercury-bromide staining. A recent study qualified SMFA following the International Conference on Harmonisation (ICH) Harmonised Tripartite Guideline Q2(R1) using an anti-Pfs25 monoclonal antibody (mAb) with a single protocol [20]. The study concluded that the range (the levels of transmission-blocking activity in which the analytical procedure has a suitable level of precision and linearity) of SMFA performed with their method was when there was more than ~80 % inhibition in oocyst intensity. However, there have been no direct studies, which assess inter-laboratory variation in % inhibition in SMFA.

Modern vaccine development relies heavily on collaborative research efforts and product development partnerships that involve multiple laboratories around the world [21]. Though highly standardized SMFAs have been performed in separate facilities, there is no assurance that SMFA data derived from experiments performed by different investigators are comparable. To enable such comparison, SMFA performance using the same test antibodies was assessed at two laboratories, TropIQ Health Sciences (TropIQ, using methodologies adapted from Radboud University Medical Center [Radboudumc], Nijmegen, The Netherlands) and the Laboratory of Malaria and Vector Research (LMVR, USA). Variation between the laboratories was assessed using a mouse mAb, a rat mAb, and human polyclonal antibody (pAb), all tested at several concentrations in two or three independent assays. When conducted under controlled conditions in different laboratories, the SMFA can provide informative comparable data for SSM-VIMT development.

Methods

Test materials

The details of mouse 4B7 (anti-Pfs25) mAb [22] and rat 85RF45.1 (anti-Pfs48/45) mAb [23] used in this study have been described previously. A protein G purified normal mouse IgG was used as the negative control for SMFA with 4B7 and 85RF45.1 mAbs, and the details of IgG preparation were described previously [20]. Human serum was sourced from a malaria-exposed Dutch expatriate with well documented antibody-mediated transmission-blocking activity, who gave verbal, but not written informed consent for his blood sample to be used for malaria research [24]. A pool of negative human sera was obtained from malaria-naive Dutch adults at Sanquin blood bank (Nijmegen, the Netherlands). From the positive and negative human sera, total IgGs were purified according to the manufacturer’s instructions using a GE Healthcare AB Spintrap purification kit (Little Chalfont, United Kingdom) at TropIQ/Radboudumc.

SMFA

The standardized methodology for performing the SMFA at LMVR has been described previously [20]. The methodology at TropIQ has also been described elsewhere and is based on that developed at Radboudumc and uses mosquitoes from Radboudumc facilities [25]. The TropIQ method differs from the Radboudumc method in that the quality of the gametocyte culture is assessed based on microscopic examination of male exflagellation instead of determination of ookinete formation in a sample that is fed to mosquitoes a day before the actual feed. In addition, the TropIQ method uses a different pipetting scheme that uses pooled reagents/parasites where possible as opposed to preparing each blood meal individually. The differences between the LMVR and TropIQ assays are summarized in Table 1. In this study, all SMFAs were conducted with P. falciparum NF54 line parasites and Anopheles stephensi mosquitoes that were propagated independently in the two laboratories. Oocyst counts were performed microscopically. Throughout the text, antibody concentration is expressed relative to the liquid phase in a feeder, i.e., serum and buffer, and does not take into consideration the volume taken up by the red blood cells.

Table 1 Details of SMFA methods
Full size table

Statistical analysis

Two different readouts have been widely used to express SMFA results, % inhibition in mean oocyst intensity (%TRA) and % inhibition in prevalence of infected mosquitoes (%TBA). It was decided to use %TRA for this analysis, since %TBA is dependent on the mean oocyst count in the control mosquitoes, and can only be reasonably estimated from the control mean and %TRA [26]. %TRA was calculated as: 100 × {1 − (mean number of oocysts in the test group)/(mean number of oocysts in the control groups)}. Percent inhibition of a test sample was always calculated against a control sample (either normal mouse antibody or normal human antibody) examined in the same feeding experiment. In each laboratory, the same samples were tested in 2 or 3 independent assays at the same concentrations. The best estimate and 95 % confidence intervals (95 % CIs) of %TRA from the multiple feeding experiments for each test antibody at each concentration in each laboratory were calculated using a negative binomial model with zero inflation model, as described previously [20]. To examine inter-laboratory variability, a multiple linear regression was utilized. Previous studies have shown that when the dose of antibody is plotted on a square root scale against inhibitory activity, specifically log-transformed mean oocyst ratio between test and control, SMFA data can be approximated to a linear regression [11, 20, 27]. Therefore, the same transformation was performed for the linear regression analysis. Since 100 %TRA data points were outside of linear range (i.e., log-transformed values became infinity), such points were excluded. The dose of antibody, laboratory of performing SMFA and the interaction of the two were used as explanatory variables. For the IC50 estimation, logistic regression was performed as described [19]. Briefly, oocyst counts from individual mosquitoes exposed to varying concentrations of test antibody were fitted to a Hill equation using Maximum Likelihood Estimation to find the best fit. All statistical tests were performed using R (R Foundation for Statistical Computing, Vienna, Austria), JMP12 (SAS Institute, Inc., Cary, NC) and p values <0.05 were considered significant.

Results

Similar correlations between mean oocysts and standard deviations or prevalence between the two laboratories

Several groups have reported that the distribution of oocyst numbers is explained well by a (zero-inflated) negative binomial model [20, 25, 28, 29]. TropIQ and LMVR use slightly different SMFA protocols (Table 1). To assess whether the data from the two laboratories show an overall similar distribution, the correlations: (1) between the mean number of oocysts in the mosquito and the standard deviation (SD) of the mean oocyst number, and (2) between mean oocyst number and the prevalence of infected mosquitoes (the fraction of mosquitoes containing any number of oocysts) were evaluated. For this purpose, LMVR historical data (including data generated outside of the present study) and TropIQ data (generated in the present study) were compared. For each “Container of Mosquitoes” (COM), the mean oocysts, the SD and the prevalence of infected mosquitoes were calculated. Throughout the paper, COM refers to a group of mosquitoes (~20 to 30 mosquitoes), which were housed in the same container and were fed the using same feeder with the same final mixture of gametocyte cultures and control/test antibodies. As shown in Fig. 1, the correlations observed in the TropIQ data generated in this study overlap with those observed in LMVR historical data. As there are no indications for a divergent data structure, the zero-inflated negative binomial model previously described by LMVR [20] was used to analyse the data sets from both laboratories. Interestingly, a similar correlation between mean oocysts and prevalence was also observed in SMFA with Plasmodium berghei parasites [29].

Fig. 1
figure 1

Strong correlations between mean oocysts and the standard deviation or the prevalence at both LMVR and TropIQ. For each COM, mean number of oocysts, the standard deviation a and the prevalence of infected mosquitoes b were calculated. Each point represents the data from a single COM. LMVR data include all COMs examined at LMVR in the last 5 years (n = 3014), and TropIQ data include only COMs used in this study (n = 33)

Full size image

SMFA comparison with mAbs

A mouse 4B7 (anti-Pfs25) mAb, which had been tested in many independent assays at LMVR, was tested at TropIQ at three different concentrations in two independent assays (Fig. 2). Inter-assay variation was calculated as the variation between different assay repeats of a single antibody condition within each laboratory, while inter-laboratory variation is given as the difference in inhibition for all assay repeats of a given condition between laboratories. At LMVR, the best estimates and the 95 % CIs of %TRA by 4B7 mAb were 25 (95 % CI −6 to 47: n = 9), 71 (65 to 76: n = 35) and 96 (96 to 97: n = 181) at 6, 23 and 94 μg/mL, respectively. The best estimates and the 95 % CI of TropIQ data were 20 (−76 to 65), 72 (38 to 89) and 98 (94 to 99) at 6, 23 and 94 μg/mL, respectively (n = 2 at each concentration). In agreement with previous work [20], inter-assay variation decreased with increasing activity of the antibody, as indicated by the smaller confidence intervals at higher %TRA, in both laboratories. Since 4B7 mAb was tested in many more feeding experiments at LMVR, the 95 % CIs at LMVR were smaller. In addition, fewer mean oocysts in the controls of the particular TropIQ feeds might partially contribute to the larger 95 % CIs. Nevertheless, the difference between the two laboratories in the best estimate of %TRA was less than 2 % points at 94 μg/mL.

Fig. 2
figure 2

SMFA comparison with 4B7 mAb. In LMVR, mouse 4B7 mAb was tested at 6 (n = 9), 23 (n = 35) and 94 (n = 181) μg/mL in multiple feeding experiments. At TropIQ, 4B7 mAb was tested at the same concentrations in two independent feeding assays. Individual feeding data and the best estimate of %TRA (with the 95 % CIs) from multiple feeds (LMVR-Comb or TropIQ-Comb) are shown. Average oocysts in a negative control antibody (normal mouse IgG) of TropIQ-feed 1(F1) and -feed 2(F2) experiments were 6.5 and 4.2, respectively. Average oocysts in negative control antibodies for LMVR experiments was 21.6 (SD = 18.5)

Full size image

Rat 85RF45.1 (anti-Pfs48/45) mAb was next tested at four different concentrations in two independent assays in both laboratories. The mAb dose-dependently inhibited oocyst formation. At higher concentrations of the mAb (i.e. 10.4 and 31.3 μg/mL), the %TRA observed for replicate experiments within labs and between labs were very comparable (Fig. 3). At lower concentrations, the observed %TRA diverged more between replicate experiments, both within and between laboratories (i.e. concentrations of 1.2 and 3.5 μg/mL). Since four different doses of mAb were tested in each feed, and the same number of feeds were conducted in each laboratory, further analysis was performed to determine the effects of dose and laboratory on inhibition. In a multiple linear regression analysis, dose of 85RF45.1 mAb, laboratory, and the interaction of the two were used as explanatory variables. The overall fit to the linear regression model was R2 = 0.73. While there was a significant effect of dose (p = 0.032), the laboratory (p = 0.427) and interaction term (dose and laboratory, p = 0.659) had no significant impact. The effect of dose on oocyst intensity was further investigated by fitting the data to a Hill equation (see Additional file 1). Resulting IC50 estimates (the concentration at which the mAb shows 50 % inhibition) were 1.2 and 8.8 µg/mL [feed 1 (F1) and feed 2 (F2) experiments] for data generated at LMVR and 3.4 and 2.5 µg/mL (F1 and F2) for data generated at TropIQ.

Fig. 3
figure 3

SMFA comparison with 85RF45.1 mAb. In both laboratories, rat 85RF45.1 mAb was tested at 1.2, 3.5, 10.4 and 31.3 μg/mL in two feeding experiments (F1 and F2). The individual data and the best estimate with 95 % CIs from the two feeds (LMVR-Comb or TropIQ-Comb) at each concentration of 85RF45.1 mAb are shown. Average oocysts in a negative control antibody (normal mouse IgG) of LMVR-F1, -F2, TropIQ-F1, and -F2 experiments were 7.8, 61.6, 6.5 and 4.2, respectively

Full size image

SMFA comparison with human polyclonal antibody

Lastly, a human pAb, which has shown a strong inhibitory activity in SMFA, was tested at four concentrations (23, 70, 210 and 630 μg/mL) in three independent assays in each laboratory (Fig. 4). At 630 μg/mL, the pAb showed a strong and reproducible  %TRA in both laboratories (best estimate of 96.6 %TRA at LMVR and 99.6 at TropIQ), and there was a dose-dependent reduction in oocyst numbers observed in each feeding experiment. As with the experiments using the rodent mAbs, inter-assay variation was larger at lower inhibitions. When a multiple linear regression analysis was performed using all four pAb concentrations (overall fit was R2 = 0.79), there was a significant effect of dose (p < 0.0001) on inhibition, but the effect of the laboratory (p = 0.232) and interaction term (dose and laboratory, p = 0.206) were not significant. Fitting the data to a Hill equation revealed IC50 estimates of 126, 329 and 158 μg/mL at LMVR, and 64, 66 and 198 μg/mL at TropIQ (see Additional file 2).

Fig. 4
figure 4

SMFA comparison with positive human pAb. In both laboratories, test human pAb was tested at 23, 70, 10.4, 210 and 630 μg/mL in three feeding experiments (F1, F2 and F3). The individual data and the best estimate with 95 % CIs from the three feeds (LMVR-Comb or TropIQ-Comb) at each concentration of test human pAb are shown. Average oocysts in the negative control (normal human IgG) of LMVR-F1, -F2, -F3, TropIQ-F1, -F2 and -F3 experiments were 3.9, 60.3, 14.0, 16.9, 4.3 and 5.9, respectively

Full size image

Discussion

This is the first study to evaluate inter-laboratory variation of the SMFA, the “gold standard” assay for Plasmodium gametocyte infectivity, using a range of transmission reducing test antibodies (mouse and rat mAbs, and human pAb). In this study, regardless of antibody type or the targets recognized by the antibodies tested, inter-assay and inter-laboratory variation decreased with increasing inhibition of oocyst formation. Conversely, variation became larger in both laboratories with decreasing percent inhibition. These results are in line with previous observations that precision of the SMFA increases with higher percentage reduction, and that levels of inhibition below 80 % need to be interpreted with caution when low numbers of replicate data are available [29]. Across the range of antibodies and concentration tested, inter-laboratory variation did not appear to be greater than the inter-assay variation observed within a laboratory. However, inter-assay variations were larger at lower levels of inhibition, therefore, it is possible that inter-laboratory variations could not be detected, if any, in the lower range. Since the error in %TRA estimate changes gradually (larger error with lower %TRA) as expected from the zero-inflated negative binomial model [20] and shown in this study, it may be difficult to establish a specific level of %TRA at which the inter-assay or inter-laboratory variations are acceptable in all situations. For example, in assessing samples from a clinical study it may be justifiable to set up more stringent criteria than that in a novel candidate discovery study. However, this study showed that at antibody concentrations that led to a >80 % reduction in oocyst numbers, the differences in best estimates of %TRA from multiple feeds between the two laboratories were less than 5 % points.

The SMFA is critical to SSM-VIMT development and there are preceding studies that provide several guidances to optimize intra-laboratory precision when performing SMFA. Medley et al. suggested that at least 50 mosquitoes (ideally 100) need to be dissected to accurately assess transmission-blocking activity [28]. Another study by van der Kolk et al. concluded that if all samples are not tested in a single experiment, data from different assays should be compared only when the mean intensity in the control is at least 35 oocysts per mosquito [13]. Churcher et al. simulated how many mosquitoes were required for dissection per feed to ensure that a reported percent inhibition value had within 10 % error from the true efficacy [29]. For example, if the mean oocyst number in the control is 5, it is estimated that ~200 mosquitoes need to be dissected to accurately report 80 % inhibition, whereas if the mean oocyst number is 100, approximately 50 mosquitoes need to be dissected.

At the present time, it is practically challenging and resource intensive to only regard SMFA experiments with a mean control oocyst intensity of 35 oocysts per mosquito as valid. In terms of the mosquito supply and maintenance, amount of test material required, and the labor intensiveness of dissection, it is also impractical to routinely perform the SMFA with 200 mosquitoes per COM (to be ready if the mean oocysts number in the control were to be 5 oocysts), To some degree, these difficulties can be overcome using one of several modified methods recently published [19, 25, 30], i.e. higher throughput SMFA using transgenic parasites that allow (semi-) automated detection of infection and may eliminate the need for mosquito dissection. However, the modified assays are not necessarily applied to all laboratories at present. Therefore, the comparison study was conducted using the SMFA methods usually performed in many laboratories; i.e. by dissection of 20–30 mosquitoes per feed. With these study conditions, SMFA showed high precision only at higher levels of inhibition. The modifications of the assay discussed above (e.g., dissecting 200 mosquitoes per COM, only utilizing SMFA data when mean control oocyst intensity is >35) should expand the range of % inhibition where the precision of the assay is acceptable. However, application of such modifications for every single assay is impractical at present. The aim of this study was to evaluate the inter-assay and inter-laboratory variations using methods which can be applied in many laboratories routinely. The minor differences between the two laboratories in methods did not cause any measurable “laboratory” effect on percent inhibitions. The present work stems from observations from a preliminary study where results with 4B7 mAb from two additional laboratories were compared. While there was an unfortunate loss in stability of the antibody during preparation and shipment, the results showed similar trends in percent inhibition between the labs despite several differences in methodology (Yimin Wu, Rhoel Dinglasan, personal communication). The approach used here will be useful to determine whether any difference in SMFA procedure affects SMFA readouts.

Because %TRA plateau at low and high concentrations of the test antibody (which gives 0 %TRA and 100 %TRA, respectively), the SMFA cannot discriminate between doses that fall within these upper or lower plateaus. The results of this study indicate that precision of the SMFA is very high at (near-) saturated antibody concentrations, where the sensitivity to changes in antibody concentrations is low. To reveal any differences in assay sensitivity between the two laboratories, IC50 values observed for the rat 85RF45.1 and human polyclonal samples were compared. For all antibodies tested here, the dose–effect relationship fitted well to a Hill equation, concordant with the notion that the interaction between an antibody and a ligand can be described by equilibrium kinetics [31]. The differences in IC50 values between laboratories were not larger than the inter-assay differences observed within a single laboratory. The SMFA data from this study also fitted well with a linear regression when doses and %TRA were transformed appropriately [11]. While the dose effect was significant, the test site did not significantly influence the outcome. Compared to analyses that evaluate single doses, the full dose response analyses takes into account all data points. This increase in number of observations is likely to increase precision of the measure of inhibition. Moreover, an understanding of the dose–effect relationship will guide predictions of the minimally required titer to achieve the desired reduction in transmission in human SSM-VIMT studies and campaigns. However, to demine the dose–effect relationship, each test sample needs to be tested at multiple concentrations. Therefore, the best balance between throughput of assay and precision/sensitivity needs to be optimized on a per study basis.

This study showed large variation in baseline oocyst intensities between different feeds (ranging from 4 to 62 mean oocysts in this study). Standardization of the average number of oocysts in assay controls would benefit from a gametocyte fertility marker, which is lacking at the moment. Nevertheless, the data indicate that the relative reduction in oocyst intensity (%TRA) can be measured reproducibly at higher levels of inhibition when oocyst intensity in the controls differed between 4 and 62. A recent study conducted at LMVR (repeat assays with 4B7 mAb and simulations using a zero-inflated negative binomial model) also supports that %TRA is independent from control oocyst intensity in SMFA (mean control oocyst intensity in the study ranged from 0.1 to 73.7) [26]. This study reconfirms that an inter-assay variability in the SMFA decreases with increasing percentage inhibition and that variability was demonstrated to be similar between different laboratories. While a weaker inhibitory activity reported from any laboratory needs to be interpreted carefully (unless tested many times or with many mosquitoes), results are very reproducible in higher percentage inhibition. The SMFAs that are performed at the two test sites described here give similar results in terms of precision and sensitivity. This is important for the screening and development of transmission reducing antibodies or compounds, as it alleviates the necessity of performing head-to-head analyses in the same assay(s) at the same laboratory, and thereby increases the flexibility in multi-center vaccine development programs. While additional tests are likely to be required when other laboratories participate in multi-centre SMFA evaluations, the study design and findings presented in this manuscript will provide guidance for the additional testing, thus supporting future SSM-VIMT development.