Introduction

Developing countries have faced important challenges to respond to the COVID-19 pandemic. Access to low-cost options to assess the extent and occurrence of infection has been a key limitation for epidemic monitoring. Serology testing is used to measure previous population exposure, an approximation to population immunity and provides adequate denominators for the estimation of attack and lethality rates1.

Validating serologic tests is key to adequately inform seroprevalence. Serologic cross-reactivity between endemic HuCoV’s and SARS-CoV is well documented2,3,4; thus, the risk of false-positives as a result of previous HuCoV exposure has to be considered to validate serologic assays during the current COVID-19 pandemic5. Beyond cross-reactivity, other factors such as viral antigen and antibody kinetics, antibody class, and reagent quality could influence the performance of a serology-based test6, so rigorous validation and contextualization is required.

In general, infection by Coronaviruses initiate when the spike (S) protein interacts with its cellular receptor. The spike protein is a type I viral trimeric glycoprotein composed by an N-terminal S1 subunit and a C-terminal S2 subunit. S1 binds to the cellular receptor through the Receptor Binding Domain (RBD). The RBD is a key determinant of host cell tropism and the major target for antibody-mediated neutralization7. There is structural conservation and cross-reactivity between SARS-CoV and SARS-CoV-2 RBD’s8,9, but less conservation with the RBD of other HuCoV’s, which makes the RBD a good antigen target for developing serologic assays.

Another widely used serologic marker for SARS-CoV-2 previous infection are antibodies against the nucleoprotein (N), a predominant non-structural protein. Antibodies against N are non-neutralizing. A number of studies have described different kinetics for anti-S and anti-N IgG10,11,12, but the impact of such differences in the interpretation of serological surveys remains to be determined.

Most COVID-19 vaccines elicit an immune response against the spike protein, whereas natural infection by SARS-CoV-2 induces a general immune response against several viral proteins, indicating that the use of different serological markers at a population level can provide relevant information of the vaccination efforts. We report an adapted version of the SARS-CoV-2 anti-S IgG ELISA described by Amanat et al.13 as a single-step method for anti-RBD IgG antibody detection. We evaluated and compared its diagnostic accuracy with a commercial semi-automated method for anti-nucleoprotein IgG antibody detection in a set of 438 serum samples from pre-pandemic and confirmed COVID-19 cases. Furthermore, to define both markers baseline seroprevalence prior to the initiation of nation-wide vaccination, we used and compared both methods in parallel in 9068 serum samples from a nationwide COVID-19 serological survey implemented during August–November 2020.

Results

Serology tests validation

We collected and analyzed 438 serum samples from 391 RT-PCR confirmed COVID-19 cases, of which 43 had paired samples, one taken during acute disease and the other during convalescence, and two cases had two serial convalescent samples. The sociodemographic characteristics of control and COVID-19 donors are shown in Supplementary Figure S1.

As an initial approach to measure diagnostic accuracy of the IgG anti-RBD ELISA, we performed Receiver Operating Characteristic (ROC) curve analysis in cases and controls, according to weeks post-symptom onset (PSO) and estimated the area under the ROC curve (AUC) and 95% confidence intervals as indicators of diagnostic performance (Supplementary Table S1 and Supplementary Fig. S2). AUC did not significantly increased from > 1-week PSO (AUC = 0.94, 95% CI 0.92–0.97) to > 7-weeks PSO (AUC = 0.97, 95% CI 0.95–1.0). These results indicate that AUC for the anti-RBD IgG ELISA is critically dependent on weeks PSO, but from > 1-week PSO high specificity and sensitivity can be achieved.

We compared the AUC of anti-RBD IgG ELISA versus anti-Nucleoprotein Elecsys®. Although AUC values were slightly higher for Elecsys®, no statistically significant difference was found at any of the time PSO categories (Supplementary Table S2 and Supplementary Fig. S3). Agreement between both methods was high. In all samples, regardless of time PSO, there was a 91.1% agreement with a Cohen’s Kappa = 0.81 (P =  < 0.001) (Supplementary Table S3), which corresponds to an almost perfect agreement14.

It is evident that the pattern of both methods ROC curves is different and suggested differences, particularly at higher specificities and earlier PSO (Supplementary Fig. S3). The partial AUC (pAUC) for both methods was calculated at the 1.0–0.9 specificity interval15 (Supplementary Table S4). As suggested by the differences in ROC curve pattern, pAUC was larger for Elecsys® in earlier times PSO, indicating that in terms of sensitivity, the Elecsys® system performs slightly better than the RBD ELISA at earlier times PSO.

Finally, we determined the absorbance cut-off for the anti-RBD IgG ELISA. By the time the current work was performed, we estimated a national seroprevalence of 25%, indicating that a highly specific method was required to obtain optimal positive predictive value16. Thus, we defined the absorbance cut-off as the lowest value to achieve ≥ 0.99 specificity. At 0.4 absorbance, we obtained the best sensitivity without lowering specificity below 0.99. Consistent with the ROC analysis, sensitivity, negative predictive value (NPV) and Negative Diagnostic Likelihood ratio (NDLR) were deeply affected by time PSO (Supplementary Fig. S4A-C) being reliable until > 3-weeks PSO (Table 1).

Table 1 Comparative diagnostic performance of Elecsys® and IgG RBD ELISA at ≥ 3 weeks PSO.
Full size table

As for specificity, both methods had the same specificity (0.995), as only one out of 199 pre-pandemic samples were false-positive (different sample). As a result, Positive Predictive Value (PPV) and Positive Diagnostic Likelihood ratio (PDLR) were also optimal and largely unaffected by time PSO (Table 1 and Supplementary Fig. S4D–F).

IgG anti-N and anti-RBD comparative results in the National Health and Nutrition Survey

Anti-RBD National seroprevalence was 23.6% (CI 95% 21.5, 25.6), which is slightly lower (1.3%) than the one observed for anti-N IgG 16. National seroprevalence for both markers was 19.7% (CI 95% 18.3, 21.3) (Table 2). Anti-N single-positive seroprevalence was 3.72% (CI 95% 3.1, 4.34) and anti-RBD single-positive seroprevalence was 1.98% (CI 95% 1.58, 2.38). Among seropositives to either markers, 77.7% were double positives, 14.1% were single positive to anti-N IgG and 8.1% were single positive to anti-RBD IgG.

Table 2 Seroprevalence (%) of anti SARS-CoV-2 IgG in México according to sociodemographic characteristics.
Full size table

We analyzed the seroprevalence based on positivity to both or single markers according to socio-demographic characteristics (Table 3). The prevalence of double positives was similar across age and sex groups. By region, only Mexico City seemed to have a higher prevalence of double positives (90.4%, 95%CI 85.9, 93.6) compared to the rest of the regions that ranged from 69.7% (95% CI 60.8, 77.4%) in the Central-North to 82.4% (95% CI 77.3, 86.5%) in the Yucatan Peninsula. The prevalence of single-positive anti-N ranged from 9.8% (95% CI 6.9, 13.8%) in the age group 40–49 to 18.5% (95% CI 13.7, 24.5%) in the age group 20–29 years; 17.0% (95% CI 14.2, 30.1%) in males and 12.3% (95% CI 9.8, 15.2%) in females; across regions, Mexico City had the lowest prevalence (4.2%, 95% CI 2.3, 7.4%) and Central-North, the highest (19.6%, 95% CI 14.5, 26.0%). The prevalence of only positive to anti-RBD ranged from 6.2% (95% CI 3.8, 10.0%) in the age group 60 and older to 10.5% (95% CI 6.5, 16.6%) in 10–19 years; males and females had the same prevalence (7.8%); and across regions, Central-Pacific had the highest prevalence (12.3%, 95% CI 6.9, 20.9%) and Peninsula, the lowest (3.4%, 95% CI 2.1, 5.5%).

Table 3 Distribution of double-positive (anti-N and anti-RBD), only positive to anti-N, and only positive to anti-RBD by age, sex, region, and symptoms. Ensanut-Covid19, 2020, Mexico.
Full size table

To identify the sociodemographic factors associated with single positivity compared to double positivity we performed a multinomial logistic regression (Table 4). Females, compared to males, had 0.66 times the relative probability of being only positive to anti-N over double-positive (p value = 0.01). Compared to Mexico City, all regions had a higher relative probability of being only positive to anti-N over double-positive and the North-Border, Central-Pacific, and Central-North had a higher relative probability of being only positive to anti-RBD over double-positive. No significant differences were observed in seropositivity to either single marker according to self-reported time post-symptom onset.

Table 4 Multinomial logistic model for only positive to anti-N and only positive to anti-S compared to double-positive (anti-N and anti-S). Ensanut-Covid19, 2020, Mexico.
Full size table

Discussion

Serosurveillance is a highly valuable approach to study epidemic dynamics and to inform public health interventions. As such, the election of the serologic method employed determines the quality of data retrieved. Aiming to validate a useful method for nation-wide serosurveillance, we have analyzed the diagnostic accuracy of an in-house, single-step adaptation of the SARS-CoV-2 RBD IgG ELISA shared by the Krammer laboratory13,17.

Once validated with a panel of pre-pandemic controls and RT-PCR-confirmed COVID-19 cases, we further compared the performance of this method with a commercial system for determining anti-N IgG in a nation-wide serosurvey. Our results indicate no significant differences in the performance of either method for serosurveillance, although slight differences in sensitivity were found, mainly related to lower sensitivity of our in-house method at early times post-symptom onset. A plausible explanation for such difference is that Elecsys® is a much more analytically sensitive electro-chemiluminescence immunoassay (ECLIA) that may detect lower quantities of antigen-bound IgG than a standard ELISA. Thus, anti-RBD ELISA test could be missing individuals recently infected, or infected several months ago, when anti-RBD response has waned and the amount of antibody is below limits of detection. In agreement with our results, ECLIA is more sensitive and has a wider linear range for detecting seroconversion against malarial antigens18.

The in-house SARS-CoV-2 RBD IgG ELISA method has several advantages for a middle-income country, such as Mexico. First, it can be implemented using standard equipment, which is usually available at serology labs. Second, reagents are low cost and can be more easily produced and /or purchased than proprietary kits, reducing the economic burden of surveillance. For the whole serosurvey, we used ~ 1 mg of recombinant RBD. Finally, adaptations can be made to increase the efficiency and reduce waste. In our case, we further simplified the original method by obviating the second confirmatory ELISA that uses the recombinant stabilized S protein, which represented a reduction in cost and time.With these changes, a validation study was performed for both tests to evaluate their diagnostic performance and to assess it in the specific context of Mexico, prior the introduction of COVID-19 vaccination. Endemic CoV’s have been present in Mexico and are frequently detected during flu season; also, Mexico is endemic for vector-borne diseases such as dengue, Zika and malaria. CoV’s and malaria have shown some degree of cross-reactivity with SARS-CoV-2 and could affect the specificity of the in-house method19,20,21; thus, a context-specific validation was needed. Given our concern with specificity, we assembled a large panel of controls from all regions of the country, using historical blood samples, allowing for a wide representation of prior viral exposures. The diagnostic accuracy of our adapted RBD IgG ELISA is comparable to what has been previously described by Indenbaum et al., in Israel who performed the adaptation of the same method shared by the Krammer lab, and showed no additional gain in sensitivity by the inclusion of IgA or IgM class detection22.

As reported by manufacturer, Elecsys® system has an overall specificity of 0.998 (95%CI 0.996–0.999)23. In an in-house validation with samples of Mexican origin, both methods displayed excellent specificity (0.995, 95% CI 0.985–1.0), ensuring high PPV in low prevalence settings (i.e. early pandemic phase, rural areas, etc.). The high specificity observed suggests that cross-reactivity in Mexico is low, increasing our confidence in the assessment, however SARS-CoV-2 serosurveillance in malaria-endemic foci along the Mexican Pacific coast, Central and South America should be interpreted carefully.

Apart from the diagnostic accuracy of each method, there is evidence that both tests provide qualitatively different information. Anti-RBD IgG titers correlate better with SARS-CoV-2 neutralization than anti-N titers10,11,24, and antibody decay seems slower for anti-S than for anti-N IgG12,25,26. Interestingly, children are often asymptomatic and in general less affected than adults27, and make more robust anti-S IgG antibody response than for the anti-N IgG SARS-CoV-228, suggesting that the sensitivity of N-based IgG immunoassays could be lower in children. Moreover, asymptomatic and mild COVID-19 adults make less robust antibody responses against SARS-CoV-210,29,30,31. A limitation of the validation approach used in this work is that all COVID-19 samples included in the panel were only from symptomatic adults, potentially underestimating false negatives derived from asymptomatic adult and children infections in large-scale serosurveillance.

Despite differences of both methods in the antibody kinetics, age differences, and analytical sensitivity differences, the results obtained at a nation-wide survey are remarkably similar for seroprevalence estimation. Nevertheless, the impact of single-marker seropositivity at an epidemiological scale is not negligible. We can think of two broad explanations of single positivity as a result of natural infection in non-vaccinated individuals:

(1) True single positivity, implying that for some biological reason, some individuals seroconvert to only one marker. As discussed, predominant seroconversion to S over N has been described in children, presumably because less viral replication favors immune response against structural antigens (i.e. Spike) over non-structural proteins such as (N)28. Although tested, we did not find any association between single S positivity and age. A matter of sample size (14 cases with single S positivity of a total of 1405 children under 9 years old) could explain lack of association.

(2) False single positivity: Individuals that seroconverted to both markers, but at the time of sampling, only one marker came out positive. In this case, different kinetics, test sensitivity or both could explain such scenario. Different kinetics of anti-N and anti-S assessed during the first pandemic waves in longitudinal cohorts for up to 8 months is well documented, indicating as a more rapid waning of anti-N antibodies than anti-S12, which could explain at least some cases of single anti-S positivity. However, our regression model failed to reveal any association with single positivity and self-reported number of months after infection.

As for test sensitivity, in our test validation we described that anti-S is slightly less sensitive than anti-N in early times PSO (not statistically significant). However, the calculation of partial AUC at high specificity values yielded significant differences in partial AUC’s (Supplementary Table S2). Thus is possible that the in-house anti-S test may be missing true positives in early phase of infection, contributing to anti-N single positivity. Not surprisingly, our regression model also failed to reveal any association between anti-N single positivity and months after infection, because seroconversion is measured in days/weeks, not months. Women have a distinctive qualitative and quantitative antibody response to influenza vaccination32. The observed differences in single-positivity to anti-RBD IgG in women are intriguing and could be the result immune response sex dimorphism. In addition, it should be noted that differences obtained in single-positivity to serological markers could also be the result from differences in the time of survey sampling and time since infection, which was different according to the phase of the epidemic wave in which each region was at the time of the survey.

Beyond the validation of the diagnostic accuracy of anti-N and anti-S IgG, this work also provides a useful baseline of seroreactivity based on natural infection. As COVID-19 vaccination coverage increases and novel SARS-CoV-2 variants are selected, seroprevalence estimation will have to take into account that most vaccines elicit anti-spike antibodies only. Thus, the combination of anti-N and anti-S serological markers may be the only approach to discriminate between natural infection and vaccination-induced seroconversion.

In conclusion, the in-house single-step anti-RBD IgG ELISA is a simple, economic, and robust assay with optimal sensitivity and specificity, with a comparable diagnostic performance in a population seroprevalence survey than the anti-N Elecsys® commercial system. As global vaccination coverage increases, combining both serological markers could be more informative in public health decisions than a single test alone.

Methods

COVID-19 cases and control serum panel for anti-RBD and N IgG test validation

Between April and August 2020, we prospectively recruited participants 18 years of age and older, with clinical presentation compatible with COVID-19 and hospitalized in preselected Mexican Institute for Social Security (IMSS) clinics of the Mexico City metropolitan area. Participants provided written informed consent. All methods were approved and performed according to the Research Ethics board of the Instituto Mexicano del Seguro Social (IMSS) guidelines (R-2020-785-065). For those who agreed to participate in the study, a serum sample was taken as soon as the RT-PCR confirmation was available. A second serum sample was taken > 3 weeks later, when possible. For validation, only RT-PCR + samples were analyzed, irrespectively of viral load. Asymptomatic individuals, or those that did not recalled their date of symptom onset were excluded. After blood draw, samples were centrifuged, and the serum was stored at − 80 °C. As non-COVID-19 samples (controls), we used a collection of 199 serum samples stored at − 80 °C, obtained during the National Health survey (ENSANUT 2018) in 2018 when no circulation of SARS-CoV-2 existed. According to the geographical origin of samples, control samples are countrywide representative, whereas COVID-19 samples derive from the Mexico City metropolitan area (MCMA) (Supplementary Fig. S1A). Age and gender distribution between cases and controls was comparable (p-value = 0.08) (Supplementary Fig. S1B–C), despite a deliberate enrichment of children samples in pre-pandemic controls to address potential antibody cross-reactivity with seasonal human coronavirus infection, which are common in the younger population33.

Anti-RBD IgG ELISA

The first step (anti-RBD screening) of the SARS-CoV-2 protocol was used as reference for a single-step in-house protocol13, and only minor modifications due to reagent and material availability were implemented. The plasmid encoding SARS-CoV-2 RBD was kindly provided by Dr. Florian Krammer at Mount Sinai Medical Center and stably transfected in CHO cells and cryopreserved. For large scale production (50 L), CHO cells were grown serially for four days at 37 °C, 5% CO2 and 130 rpm agitation in 1000-mL shake flasks (200 mL culture volume), then seeded in 5 L New Brunswick bioreactor for 3 additional days at similar conditions and with dissolved oxygen tension controlled > 30% of saturation. Finally, 3.7 L were further transferred to a 200 L Biostat CultiBag bioreactor and cultured for one more week. Fifty L of supernatant were depth-clarified and concentrated to 4 L by filtration with a Cogent M1 ultrafiltration system and a membrane cutoff of 10 kDa. Concentrated supernatant was subjected to affinity purification with Ni column in an AKTA prime chromatograph and further dialysis and sterile filtration with Millipak 20 filters. Final RBD purity was above 80% as quantified by SDS-PAGE.

RBD was immunoadsorbed to High bind 96-well ELISA flat bottomed plates (Corning CLS9018) at 2 μg/mL in PBS at 4 °C overnight and washed manually in PBS-Tween 20, followed by blocking at 4 °C in PBS-5% milk (Difco) for two hours at 4 °C. After washing, plates with 1:50 diluted serum samples were incubated at RT for two hours. After washing, plates were incubated with a 1:5,000 dilution of HRP-conjugated goat anti-human IgG (Sigma) for 1 h at RT, washed and developed with OPD in Phosphate/Citrate buffer. Absorbance was read in an ELISA plate reader (Biotek ELx808) with a 450 nm filter. Background absorbance was the average absorbance of 16 blank wells, which was subtracted to each sample well.

Anti-N IgG electro-chemiluminescence immunoassay (ECLIA)

Detection of anti-Nucleoprotein IgG was performed with the Elecsys® Anti-SARS-CoV-2 Electro-chemiluminescence immunoassay (ECLIA) (Roche Diagnostics), using the Cobas e 411 analyzer, according to manufacturer’s instructions.

Statistical analysis of diagnostic performance

Antigen-specific class-switched antibody secretion and seroconversion are highly dependent of the time since antigen exposure up to the development of functional germinal centers25. To account for the time of exposure, we categorized COVID-19 cases according to weeks PSO (Supplementary Fig. S1 and Supplementary Table S2) and evaluated diagnostic accuracy by estimating different parameters of diagnostic performance such as AUC of the ROC curve, and partial AUC using the R package pROC34. For statistical comparison of AUC and partial AUC we used the DeLong35 and Bootstrap methods36, respectively. ROC analysis allowed us to define the optical density (O.D.) cutoff for anti-RBD ELISA based on an expected specificity of ≥ 0.99.

To estimate sensitivity, specificity, predictive values and diagnostic likelihood ratios at different weeks PSO and to compare the performance of anti-RBD ELISA with the Elecsys® Anti-Nucleoprotein ECLIA, we used the statistical R package DTComPair37. Differences in sensitivity and specificity between methods were analyzed using the McNemar test38. Differences in predictive values were evaluated with the Generalized Score Statistic39. Diagnostic likelihood ratios were evaluated with the DLR regression model40. The concordance between both methods was estimated using Cohen’s Kappa statistic41 and Maxwell’s RE coefficient42 with the R package irr43.

Nationwide serological survey

The National Health and Nutrition survey (ENSANUT 2020-COVID-19) is a nationwide probabilistic household survey based on multistage and stratified sampling of participating households, implemented during August to November 2020. It included 10,216 complete household interviews and 9464 serum samples for biochemical markers and COVID-19 serology44. Participants provided written informed consent as approved by the Research ethics board of the National Institute of Public Health (CI 1679), and all methods were performed according to the Declaration of Helsinki and corresponding Institutional guidelines. All serum samples were independent of the validation sample set and were processed with the Elecsys® Anti-SARS-CoV-2 Electro-(ECLIA), whereas 9,068 were processed with the anti-RBD IgG ELISA.

Statistical analysis of anti-N and anti-RBD seroprevalence

We calculated the prevalence with 95% confidence intervals of only anti-RBD, only anti-N and both anti-RBD and anti-N, overall and by sociodemographic characteristics. Prevalence was adjusted by sensitivity and specificity of each validation test45. We grouped seropositive individuals as double-positive (anti-N and anti-RBD), positive only to anti-N, and positive only to anti-RBD. We calculated the distribution by age, sex, region, and symptoms with 95% confidence intervals. We then used a multinomial logistic regression including age, sex, and region, using the three groups as the outcome. We report the exponentiated coefficients as relative probabilities of only anti-N and only anti-RBD, using double positive as the reference group. We used the survey commands to account for the survey design. The analysis was made in Stata v14 (StataCorp).