FormalPara Key Summary points

Quantitative serological assays detecting response to SARS-CoV-2 infection are urgently needed to quantify immunity.

This manuscript presents the results of direct comparison of two independent quantitative anti-S1 assays (Euroimmun Anti-SARS-CoV-2 QuantiVac ELISA [IgG] and Roche Elecsys® Anti-SARS-CoV-2 S) in oligo-/asymptomatic individuals from a previously characterized population-based cohort.

Both assays showed similar performance and a high level of agreement with direct virus neutralization and surrogate neutralization tests, arguing for their utility in quantifying immune protection against SARS-CoV-2.

In certain circumstances and following rigorous validation, these quantitative assays may replace direct neutralization assays (the current gold standard) which are unsuitable for large-scale studies and diagnostic routine testing.

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.14697141.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 (COVID-19), emerged in Wuhan, China, in December 2019, and within months caused millions of infections and deaths across the globe [1]. Despite multiple interventions, including social distancing, wearing of protective equipment in public, and introducing enhanced disinfection procedures, the number of infected individuals worldwide continued to rise beyond the end of 2020 [2].

The gold standard for diagnosis of acute COVID-19 is molecular detection of the viral ribonucleic acid (RNA) by reverse transcription-polymerase chain reaction (RT-PCR) [3]. In addition, antibody testing can be used to detect humoral immune responses after the infection. Immunoassays detecting anti-SARS-CoV-2 antibodies can be especially valuable to confirm the extent of population exposure and to quantify vaccine responses [4, 5].

Seroconversion typically starts 5–7 days after SARS-CoV-2 infection [4]. All antibody types (immunoglobulin [Ig] A, IgG, and IgM) can be detected within the same time frame around week 2–4, and the IgG response persists the longest [4, 6]. The most important targets of humoral response are the nucleocapsid protein (N), involved in viral RNA replication and viral assembly, and parts of the trimeric spike complex, in particular the receptor-binding domain (RBD) of S1 which interacts with the angiotensin-converting enzyme 2 (ACE2) receptor on human cells [7, 8]. Antibodies that bind to RBD in a way that prevents its attachment to the host cell have a convincing functional likelihood to neutralize the virus and are viewed as the key indicator of immune protection [9,10,11]. In line with these observations, the spike protein became the leading antigen target in vaccine development [12].

Accumulating data suggest that high titers of IgG in convalescent plasma correlate with the presence of neutralizing antibodies, which may correlate with protection against infection [8, 13,14,15]. However, the long-term persistence of neutralizing antibodies and the degree of protection they confer, as well as the degree and clinical significance of seroconversion in asymptomatic individuals, remain largely unknown.

A number of immunoassays from different manufacturers are currently available and have been compared directly in several head-to-head studies [16,17,18,19]. Since most are qualitative in nature, the emergence of quantitative assays is needed for a precise evaluation of immune response to viral antigens. Importantly, a reliable quantitative assay can be used to quantify protection in different settings (e.g., mild disease or vaccination). Such evaluation will require robust data on quantitative assay performance and its correlation with available neutralization tests.

Here, we present the results of a direct comparison of two novel quantitative anti-S1/RBD antibody tests applied to a subset of samples derived from a prospective population-based cohort study of COVID-19 incidence/prevalence in Munich, Germany. The ongoing KoCo19 study investigates the prevalence of SARS-CoV-2 infections among a randomly selected cohort, analyzes transmission within households and risk factors, and compares the performance of various immunoassays in testing asymptomatic and oligosymptomatic individuals [20]. The primary results were reported elsewhere including the estimated seroprevalance in Munich [21] and a comparison of seven qualitative seroassays to identify effective testing strategies [22]. This manuscript reports the analysis of the performance and correlation of quantitative Euroimmun Anti-SARS-CoV-2 QuantiVac enzyme-linked immunosorbent assay (ELISA) IgG assay that recognizes S1 (hereafter called EI-S1-IgG-quant) and quantitative Elecsys® Anti-SARS-CoV-2 S pan-Ig assay that recognizes S1 RBD (hereafter called Ro-RBD-Ig-quant) in a subset of samples derived from the KoCo19 cohort. Both assays were compared with previously described qualitative primary assays, namely Euroimmun Anti-SARS-CoV-2 ELISA IgG (hereafter called EI-S1-IgG) [23, 24] and Elecsys Anti-SARS-CoV-2 Roche anti-N pan-Ig (hereafter called Ro-N-Ig) [25]. The primary tests were also assessed alongside assays that confirm infection, including direct virus neutralization test (NT) with SARS-CoV-2 wild-type virus (SARS-CoV-2 strain MUC-IMB-01 isolated in January 2020), GenScript®cPass™ (hereafter called GS-cPass) and Mikrogen-recomLine-N/RBD IgG line (hereafter called MG-N and MG-RBD) immunoassays.

Methods

Study Design and Participants

Samples were derived from the population-based prospective COVID-19 cohort KoCo19 from Munich, Germany [20]. Out of the total 6658 samples analyzed previously [22], 362 (due to having NT arrays and all other confirmatory tests) were included in this analysis. These included samples from (i) asymptomatic or oligosymptomatic individuals who had at least one RT-PCR-positive SARS-CoV-2 test on a pharyngeal swab (true positive samples), (ii) those who did not have an RT-PCR-positive test, but experienced seroconversion in at least one of the primary tests used (“other seropositive” samples), and (iii) negative controls—partially blood donors collected before the surge of SARS-CoV-2/COVID-19, and negative samples obtained during the pandemic (true negative samples). All samples were collected during the same time period (between April and June 2020), except for true negative samples from blood donors (October 2019 and March 2020, i.e., before and after the seasonal common cold period). Samples were defined as “other seropositive” if one of the serological tests yielded a positive result, suggesting a likely but unconfirmed SARS-CoV-2 infection. To ensure a dataset with exclusively independent variables, only one serum sample per participant was used for analyses.

The study was approved by the Ethics Commission of the Faculty of Medicine at Ludwig-Maximilians-Universität Munich (reference numbers 20–275-V, 20–371-V, and 20–262-V) and the protocol is available online (www.koco19.de) [20]. This study was conducted in accordance with the principles of the Declaration of Helsinki and Good Clinical Practice Guidelines. Informed written consent was obtained from all study participants.

Laboratory Assays

Blood samples were obtained as previously described [20]. Briefly, blood was collected in ethylenediaminetetraacetic acid-coated tubes, refrigerated and maintained at 4 °C from the moment of extraction until centrifugation to separate the cell pellet from the plasma. Plasma samples were analyzed and stored at − 80 °C in temperature-controlled biobank freezers; freeze–thaw cycles were minimized to avoid sample degradation.

The presence of antibodies was analyzed using appropriate assay kits according to the manufacturers’ instructions.

An overview of all assays, their cutoff values, and readouts are shown in Supplementary Table 1. The World Health Organization (WHO) reference sera (National Institute for Biological Standards and Control [NIBSC] code 20/268) were measured on the individual assays in replicates (n = 3) to standardize the results [26].

Primary Assays

EI-S1-IgG-quant and EI-S1-IgG results were measured on a Euroanalyzer-1 robot (Euroimmun, Lübeck, Germany). For the qualitative assay EI-S1-IgG, presented values show quotients of the optical density measurements given by the manufacturer’s software. For the quantitative assay EI-S1-IgG-quant, values are shown in units per milliliter (U/mL). Two cutoffs were applied as recommended by the manufacturer: 25.6 U/mL to separate negative values from indeterminate values, and 35.2 U/mL to separate indeterminate values from positive values (Supplementary Table 1). Values between 1 and 120 U/mL represent linear range, samples with values below 1 U/mL were assigned a categorical value of 0, whereas samples with values above 120 U/mL were diluted 1:4 with sample buffer (Euroimmun, Lübeck, Germany) and measured again.

Ro-RBD-Ig-quant and Ro-N-Ig results were measured on cobas e411 and/or e801 modules (Roche, Mannheim, Germany). For the qualitative assay Ro-N-Ig, values correspond to the sample cutoff index. For the quantitative assay Ro-RBD-Ig, values are shown in U/mL. Manufacturer cutoff was applied to separate negative and positive values. Values between 0.4 and 250 U/mL represent linear range. Samples with values below 0.4 U/mL were assigned a categorical value of 0, whereas samples with values above 250 U/mL were diluted 1:10 with sample diluent buffer (Roche, Mannheim, Germany).

Assays Confirming Infection

Confirmatory testing was performed using micro-virus NT assays as described previously [27], with the exception that confluent cells were incubated instead of adding cells following neutralization reaction, and the serum dilutions started with 1:5 instead of 1:10. Samples with a titer < 1:5 were classified as NT-negative, and samples with a titer ≥ 1:5 as NT-positive.

Binding inhibition was measured using the SARS-CoV-2 surrogate virus neutralization test (GS-cPass; GenScript®, Piscataway, New Jersey, USA) according to the manufacturer’s instructions. Photometric measurements were performed using the Tecan Sunrise (Tecan, Männedorf, Switzerland). Binding inhibition was calculated in percentages (range from − 30% to 100%; cutoff was 20% as recommended by the manufacturer).

The recomLine SARS-CoV-2 IgG line immunoassay (MG-N and MG-RBD; Mikrogen, Neuried, Germany) based on nitrocellulose strips with recombinant SARS-CoV-2 antigens N and RBD was measured using the fully automated recomLine strip processor CarL (Mikrogen, Neuried, Germany) according to the manufacturer’s instructions. Raw values were presented in arbitrary units, and manufacturer’s cutoffs were applied.

Statistical Analysis

Only one sample per participant was included in the statistical analyses; in case of individuals with multiple blood samples, the sample with the most comprehensive dataset was included. For multiple measurements with complete datasets, only the first measurement was considered; for operational replicates the latest measurement was included. Assay comparison was performed as described previously [22]. Statistical analysis and visualization was performed using software R, version 4.0.2 (https://cloud.r-project.org/).

Square roots R of coefficients of determination (R2) were calculated for continuous variables. Paired sample comparisons were performed using Wilcoxon-sign-rank tests; multiple group comparisons were performed using Kruskal–Wallis tests, followed by post hoc Dunn tests and the Benjamini–Yekutieli adjustment for pairwise comparisons [28].

Results

A total of 362 samples from the KoCo19 cohort were included in the analysis: 108 samples from individuals who had viral RNA detected in pharyngeal swabs (true positives), 143 “other seropositive” samples, and 111 negative controls [20].

Performance of Anti-S1 Tests

The diagnostic accuracy measurements of all primary and confirmatory tests are presented in Table 1.

Table 1 Performance of primary tests and confirmatory tests
Full size table

The raw value distributions of Ro-RBD-Ig-quant on n = 357 evaluable samples and EI-S1-IgG-quant on n = 354 evaluable samples are presented in Fig. 1. Both assays showed a bimodal distribution. Ro-RBD-Ig-quant assay showed a better signal spread with a clear separation of true negative and true positive samples, and did not produce discordant results. Ro-RBD-Ig-quant detected 100% of the positive and 100% of the negative samples, whereas EI-S1-IgG-quant detected 96% of the positive and 97% of the negative samples. Thirteen samples produced discordant results in EI-S1-IgG-quant and were categorized as indeterminate as they did not meet criteria for the positive or negative categories (Table 1).

Fig. 1
figure 1

Performance of primary tests Ro-RBD-Ig-quant (a) and EI-S1-IgG-quant (b). Black dashed lines represent manufacturers’ cutoff values and red dotted lines represent WHO standards (from the left to the right: 20/142, 20/144, and 20/140 for a (20/140 and 20/144 for b) with almost identical values, 20/148, 20/150). Histograms show counts of individual samples, whereas solid blue and orange lines show cumulative distribution of true positive and true negative samples. Orange and blue numbers give the percentage of true positive and true negative samples, which were correctly detected by the tests

Full size image

Titer values of true positive samples with available data on time between RT-PCR and blood sampling for Ro-RBD-Ig-quant (n = 232) and EI-S1-IgG-quant (n = 228) are shown in Fig. 2. Values were widespread in the cohort with < 30 days between RT-PCR and antibody test for both assays. The mean titer values tended to decrease over time, with statistically significant differences between value distribution in the cohort with < 30 days vs cohort with > 240 days for both assays (p < 0.0001). After 240 days, the majority (80%) of Ro-RBD-Ig-quant values remained in the positive range whereas almost half of EI-S1-IgG-quant values no longer met the positivity threshold. Pairwise comparison between time-dependent groups after adjustment for multiple comparison are shown in Table 2.

Fig. 2
figure 2

Time-dependent values of positive samples in primary tests Ro-RBD-Ig-quant (a) and EI-S1-IgG-quant (b). Black dashed lines represent manufacturers’ cutoff values and red dotted lines represent the WHO standards (from the bottom to the top: 20/142, 20/144, and 20/140 for a (20/140 and 20/144 for b) with almost identical values, 20/148, 20/150). Assay results were categorized according to the time after the positive RT-PCR test (< 30 days, 30–90 days, 90–150 days, 150–240 days, and > 240 days). Plots show the individual read-out (orange dots), a density estimate (orange area), the 25th, 50th, and 75th percentiles (black box), and the mean (black dot); mean and median numbers are included for each group

Full size image
Table 2 Pairwise comparison between the time-dependent groups after adjusting for multiple comparison
Full size table

Concordance Between Quantitative and Semi-quantitative Anti-S1 Tests

To allow for comparison of scale, results of individual assays with the WHO reference panel (NIBSC 20/168) are presented in Table 3.

Table 3 Results from individual assays with the WHO reference panel for anti-SARS-CoV-2 immunoglobulin (NIBSC code 20/268)
Full size table

Pairwise comparison of primary tests is shown in Supplementary Fig. 1; agreement of 95–98% was observed for all comparisons. When categorical values were excluded, Ro-RBD-Ig-quant showed a high numerical correlation with EI-S1-IgG (R = 0.72, p < 0.0001; Supplementary Fig. 1A), while the numerical correlation with Ro-N-Ig was lower (R = 0.34; p < 0.0001; Supplementary Fig. 1B). EI-S1-IgG-quant showed a high numerical correlation with EI-S1-IgG (R = 0.55, p < 0.0001; Supplementary Fig. 1C) while with Ro-N-Ig the correlation was lower (R = 0.20, p < 0.001; Supplementary Fig. 1D). A high level of categorical agreement was observed between Ro-RBD-Ig-quant and EI-S1-IgG-quant with a high correlation (96% of positive samples and 97% of negative samples; R = 0.50, p < 0.0001; Supplementary Fig. 1E). Notably, Ro-RBD-Ig-quant gave a clearer separation of positive and negative values than EI-S1-IgG-quant; EI-S1-IgG-quant showed many values at the intermediate range and some non-specific reactivity among the negative samples (3%).

Concordance with Tests Confirming Infection

Ro-RBD-Ig-quant values showed significant increases between NT dilution categories (p < 0.001), with mean values increasing from 39.64 in the NT dilution category < 1:5 to 486.24 in the NT dilution category > 1:80 (Fig. 3a). Notably, NT at dilution 1:5 still contained approximately 20% of true positive samples. Ro-RBD-Ig-quant also showed a high categorical agreement and correlation with GS-cPass (96%/99%, R = 0.54, p < 0.0001; Fig. 3b).

Fig. 3
figure 3

Pairwise comparison of primary tests with confirmatory tests. Bivariate comparisons shown as violin and scatter plots for quantitative Ro-RBD-Ig-quant vs NT at indicated dilutions (a) and vs GS-cPass (b) and for quantitative EI-S1-IgG-quant vs NT at indicated dilutions (c) and vs GS-cPass (d). Black dashed lines represent manufacturers’ cutoff values and red dotted lines represent the WHO standards (from the bottom to the top: 20/142, 20/144, and 20/140 for a (20/140 and 20/144 for c) with almost identical values, 20/148, 20/150). Orange and blue numbers give the percentage of true positive and true negative samples, which were correctly detected by the tests. Bold dashed lines are linear fit and gray areas surrounding them represent 95% CI; for the interested region, the polynomial fit was within the 95% CI of the linear fit. Square root R of coefficients of determination is given for association among continuous variables. Pairwise comparison between NT dilution categories (for a and c) after adjustment for multiple comparison are shown in Table S2

Full size image

EI-S1-IgG-quant values also showed significant increases between NT dilution categories, with mean values of EI-S1-IgG-quant increasing from 44.33 (NT dilution < 1:5) to 956.6 (NT dilution > 1:80; Fig. 3c). NT at dilution 1:5 still contained approximately 16% of true positive samples. EI-S1-IgG-quant showed a high level of correlation with GS-cPass (93%/97%, R = 0.41, p < 0.0001; Fig. 3d), although some unspecific reactivity in the negative samples was detected for EI-S1-IgG (3%).

The predictive value (95% accordance of the positive predictive value) of the two quantitative tests at different thresholds was investigated through the alignment of their results with NT dilution categories ≥ 1:5 and ≥ 1:10, and GS-cPass categories ≥ 20% and ≥ 30%. The lowest Ro-RBD-Ig-quant and EI-S1-IgG-quant values [with 95% CI] for which GS-cPass is ≥ 20% (6.99 and 27.49, respectively) or ≥ 30% (11.60 and 40.62) and NT is ≥ 1:5 (28.67 and 49.78) or ≥ 1:10 (51.41 and 104.06) are presented in Table 4 and Supplementary Fig. 2. These values refer to the intersection of the linear fit with the selected values for GS-cPass and NT of Fig. 3.

Table 4 Predictive value of quantitative tests
Full size table

Ro-S1-Ig-quant showed a high level of correlation with line blot assay MG-RBD and a lower numerical correlation with MG-N (R = 0.44, p < 0.0001 and R = 0.32, p < 0.001, respectively; Supplementary Fig. 3A, B). EI-S1-IgG-quant showed a high level of correlation with MG-RBD (R = 0.46, p < 0.0001; Supplementary Fig. 3C), but the agreement with MG-N was not statistically significant (R = 0.15, p = 0.089; Supplementary Fig. 3D).

Discussion

Since the surge of the COVID-19 pandemic, the number of serological assays continues to increase, and the comparative assessment of their analytical performance is essential to inform strategies in diagnostic, epidemiological, and vaccination studies. In this study, we performed a head-to-head direct comparison of two independent quantitative assays directed against S1 and compared their performance with two qualitative primary assays and several assays that confirm infection, including direct virus neutralization. Samples from oligo-/asymptomatic patients from a population-based cohort were utilized to help understand the true impact of COVID-19, given mild/asymptomatic cases are predominant in the population and data regarding immunity protection that develops in such patients are limited but of critical importance for policy makers worldwide.

The quantitative tests Ro-RBD-Ig-quant and EI-S1-IgG-quant showed a high level of correlation when used in a population cohort containing samples from mostly oligo- or asymptomatic individuals; both assays showed categorical agreement with Ro-N-Ig, micro-virus neutralization assay, GS-cPass and recomLine. This suggests both tests can detect correlates of neutralization, which is understood to mediate humoral protection following SARS-CoV-2 infection. While the mean titers for both assays tended to decrease after their peak (ca. 1 month or ca. 3 months after infection for EI-S1-IgG-quant and Ro-RBD-Ig-quant, respectively) to > 240 days after positive RT-PCR, a higher proportion of Ro-RBD-Ig-quant values remained positive after 240 days.

Finally, both quantitative assays showed a good level of concordance with each other, with Ro-RBD-Ig-quant performing slightly better in terms of clearer separation of positive and negative samples and less non-specific reactivity.

Currently, the most reliable method of detecting antibody response indicative of protection is direct virus neutralization; however, this test requires intact virus and has to be performed under biosafety level 3 conditions, making it infeasible for large-scale studies and diagnostic routine testing [17]. There are also numerous different protocols for direct viral neutralization with poor overall comparability. Commercial high-throughput tests use parts of viral proteins instead and combine these with other components of chemiluminescence detection or ELISA [3, 6]. Although all viral proteins are likely to elicit some degree of immune response, most efforts concentrated on measuring antibodies directed against N and S1/RBD so far [5]. Some studies suggest that S1 may be the optimal antigen for SARS-CoV-2 serological tests, as it is more sensitive than RBD and more specific than S trimer [29]; however, this assumption could not be confirmed in this study.

Quantitative anti-S1 assays will be a valuable tool for measuring antibody responses to SARS-CoV-2. Importantly, quantitative assays will allow us to precisely enumerate and compare antibody titers in individuals who had severe disease, mild disease, asymptomatic individuals, and those who achieved immunity after vaccination. The assays may also be applied to screen for plasma samples that contain specific high-affinity neutralizing antibodies and help identify potential donors of plasma for convalescent plasma therapy [30]. Once established and rigorously validated, these assays may replace the current gold standard of direct neutralization, which requires handling at biosafety level 3 and has severe limitations in signal resolution at the lower end of the range.

In this study, both EI-S1-IgG-quant and Ro-RBD-Ig-quant showed a high level of correlation with direct virus micro-neutralization and surrogate neutralization test, GS-cPass. For example, raw values above 28.67 U/mL for Ro-RBD-Ig-quant and above 49.78 U/mL for EI-S1-IgG-quant, respectively, predicted virus neutralization > 1:5 in 95% of cases. We may hypothesize that when the value of the quantitative tests is above the predictive value (e.g., 95%), there is little benefit in performing NT and that this could act as a surrogate marker for neutralizing titers, e.g., after mass vaccinations or post-infection.

Conclusions

To the best of our knowledge, we present for the first time a comparison with quantitative tests which can be reproduced by others to assess neutralization as a bridge to serologically mediated immunoprotection. Our results suggest that both quantitative assays may be useful in future studies aimed to assess immunization efficiency, determine the degree of herd immunity, and estimate how long the response persists over time.