A versatile portfolio of diagnostic tests is essential for the containment of the severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) pandemic. Besides nucleic acid-based test systems and point-of-care (POCT) antigen (Ag) tests, quantitative, laboratory-based nucleocapsid Ag tests for SARS-CoV-2 have recently been launched. Here, we evaluated four commercial Ag tests on automated platforms and one POCT to detect SARS-CoV-2. We evaluated PCR-positive (n = 107) and PCR-negative (n = 303) respiratory swabs from asymptomatic and symptomatic patients at the end of the second pandemic wave in Germany (February–March 2021) as well as clinical isolates EU1 (B.1.117), variant of concern (VOC) Alpha (B.1.1.7) or Beta (B.1.351), which had been expanded in a biosafety level 3 laboratory. The specificities of automated SARS-CoV-2 Ag tests ranged between 97.0 and 99.7% (Lumipulse G SARS-CoV-2 Ag (Fujirebio): 97.03%, Elecsys SARS-CoV-2 Ag (Roche Diagnostics): 97.69%; LIAISON® SARS-CoV-2 Ag (Diasorin) and SARS-CoV-2 Ag ELISA (Euroimmun): 99.67%). In this study cohort of hospitalized patients, the clinical sensitivities of tests were low, ranging from 17.76 to 52.34%, and analytical sensitivities ranged from 420,000 to 25,000,000 Geq/ml. In comparison, the detection limit of the Roche Rapid Ag Test (RAT) was 9,300,000 Geq/ml, detecting 23.58% of respiratory samples. Receiver-operating-characteristics (ROCs) and Youden’s index analyses were performed to further characterize the assays’ overall performance and determine optimal assay cutoffs for sensitivity and specificity. VOCs carrying up to four amino acid mutations in nucleocapsid were detected by all five assays with characteristics comparable to non-VOCs. In summary, automated, quantitative SARS-CoV-2 Ag tests show variable performance and are not necessarily superior to a standard POCT. The efficacy of any alternative testing strategies to complement nucleic acid-based assays must be carefully evaluated by independent laboratories prior to widespread implementation.
Non-PCR-based point-of-care testing (POCT) has been widely introduced into national test strategies and independently evaluated using different settings and approaches [1,2,3,4]. While the detection and quantification of SARS-CoV-2 genomes by nucleic acid amplification testing represent the gold standard in diagnostic laboratories, the reagent supply chain can be limiting and turnaround times for PCR testing prolonged, making an effective clinical and outbreak management difficult at times, especially when incidences are high.
To add to the repertoire of quality-controlled, laboratory-based SARS-CoV-2 testing from respiratory material, several companies have recently introduced automated, quantitative SARS-CoV-2 Ag assays. First field studies indicate that these medium- to high-throughput assays’ sensitivities ranged from 40 to 93% and specificities between 91 and 100% [5,6,7,8,9,10,11,12,13,14,15,16]. While assay specificities were frequently found to be relatively high [5,6,7,8,9,10,11,12,13,14,15,16], assay sensitivities according to international and national guidelines for rapid Ag tests in general, requiring positive rate percentages ≥ 80, have frequently not been met [17, 18]. Further, the studies published so far did not compare different automated SARS-CoV-2 Ag assays among each other, which will be critical for diagnostic laboratories seeking to implement such assays.
Since the beginning of 2021, VOCs have started to dominate the pandemic in different parts of the world. For Germany, the Robert-Koch Institute estimated that in the middle of April 2021 ca. 90% of newly diagnosed SARS-CoV-2 infections were caused by VOC Alpha (pangolin lineage B.1.1.7), in line with surveillance reports from many other countries [19, 20]. In South Africa, the VOC Beta (pangolin lineage B1.351) has been driving the second wave in late 2020/early 2021  and has been detected around the globe, including many European countries [19, 22]. It is largely unclear whether SARS-CoV-2 Ag assays are able to sensitively detect VOCs, which besides their defining mutations in spike also carry up to four amino acid mutations in the respective nucleocapsid proteins, i.e., Alpha: D3L, R203K, G204R, S235F; Beta: T205I; Gamma: P80R, R203K, G204R; Delta: D63G, R203M, G215C, D377Y .
The aim of our study was to compare four different automated SARS-CoV-2 Ag assays, which all detect the nucleoprotein of SARS-CoV-2 applying different technologies, for their analytical performance. In parallel, a widely used POCT rapid Ag test was used with an identical set of samples. We evaluated respiratory samples collected from patients at the University Hospital of Munich (LMU Klinikum) at the end of the second pandemic wave in Germany (February–March 2021) as well as patient-derived isolates, including EU1 (pangolin lineage B.1.177), VOC Alpha (B.1.1.7) and VOC Beta (B.1.351), which had been expanded in cell culture in a biosafety level 3 laboratory.
Materials and methods
In the period February 1 to March 1, 2021, respiratory swabs (nasopharyngeal, oropharyngeal or unrecorded sampling site) were collected by health-care professionals from individuals, who were seen in the emergency room or on clinical units of the LMU Klinikum, the second-largest University Hospital in Germany, and three teaching hospitals of the LMU Munich (Helios Amper Hospital Dachau, Helios Hospital München West and Helios Hospital München Perlach). For this study, flocked swabs were collected in IMPROVIRAL™ with 3 ml Viral Preservative Medium (VPM) (Improve Medical, Guangzhou, Republic of China) or CITOSWAB® with 3 ml Viral Transport Medium (VTM) (Citotest Scientific Co.,Ltd, Jiangsu, Republic of China) and analyzed by RT-PCR for SARS-CoV-2 RNA. All samples with a measurable Cp/Ct value by RT-qPCR under accredited conditions were scored “SARS-CoV-2-positive”. Analysis for the lower limit of detection was not performed in this study, but analyzed elsewhere . Original respiratory swabs and transport media were stored at 4 °C for up to 24 h, until samples were inactivated and SARS-CoV-2 Ag testing was performed. A total of 107 SARS-CoV-2-PCR-positive and 303 PCR-negative respiratory samples were analyzed.
SARS-CoV-2 antigen tests
All tests were performed according to the manufacturer's instructions. The swab set/transport media and specimen storage conditions described above do not deviate from the manufacturer's recommendations unless further specified below.
The SARS-CoV-2 Rapid Antigen Test (“RAT”) from Roche Diagnostics (Rotkreuz, Switzerland) is a rapid chromatographic immunoassay intended for the qualitative, visual detection of SARS-CoV-2 present in the human nasopharynx . This assay is based on mouse monoclonal antibodies against SARS-CoV-2 nucleocapsid. Besides the extraction buffer provided, the manufacturer recommends the use of three specific VTMs. The presence of a test line (T), no matter how faint, together with a control line (C) means a positive test result. The manufacturer proposes it as a screening test in POCT settings for both symptomatic and asymptomatic individuals and states in the product sheet a test sensitivity of 96.52% and a test specificity of 99.68% based on results from studies conducted in Israel and Brazil referred to in the product sheet .
The Lumipulse G SARS-CoV-2 Ag (“CLEIA”) from Fujirebio Inc. (Tokyo, Japan) is an assay system containing a range of immunoassay reagents for the quantitative measurement of SARS-CoV-2 Ag in samples based on CLEIA technology , a specific two-step immunoassay on the LUMIPULSE G system . This assay uses several monoclonal antibodies against SARS-CoV-2 nucleocapsid coated on ferrite particles. When preparing the sample from virus preservative solutions for nucleic acid testing, the manufacturer recommends centrifuging the sample at a minimum of 2000×g for at least 5 min and using the supernatant for measurement. Using a cutoff of 1.34 pg/ml, the sensitivity was 91.7% and the specificity 97.3% in a study of 325 specimens from Japanese hospitals (nasopharyngeal swabs with virus preservative solution) . The manufacturer recommends the determination of an individual cutoff value according to the requirements of the respective laboratory.
The LIAISON® SARS-CoV-2 Ag (“CLIA”) assay from DiaSorin S.p.A. (Saluggia, Italy) utilizes a direct, two-step sandwich chemiluminescence immunoassay (CLIA) for the quantitative determination of SARS-CoV-2 in nasal swabs or nasopharyngeal swabs . The test uses rabbit polyclonal antibodies against the SARS-CoV-2 nucleocapsid. Furthermore, the manufacturer restricts use to individuals with suspected COVID-19 within 10 days of symptom onset. The test may only be performed on the LIAISON® XL Analyzer.
For nasopharyngeal swabs, the clinical performance of the LIAISON® SARS-CoV-2 Ag test was determined using a total of 408 samples from symptomatic patients. The sensitivity is reported to be 99.1% and the specificity 98.7%. The manufacturer specifies in the package insert that nasopharyngeal swabs in VTM/UTM should only be stored at 2–8 °C for up to 12 h before transferring the sample to the inactivation buffer. In the above study, nasopharyngeal swabs were collected in UTM/VTM, stored frozen, thawed, eluted in sample inactivation buffer and tested with LIAISON® SARS-CoV-2 Ag according to the test procedure. In a further statement, the manufacturer recommends that each laboratory should investigate the used pre-analytical methods (storage stability) to determine the validity.
The Elecsys SARS-CoV-2 Antigen (“ECLIA”) assay from Roche Diagnostics GmbH (Mannheim, Germany) uses the antibody sandwich principle (monoclonal anti‑SARS‑CoV‑2 antibodies (mouse and rabbit)) in an electrochemiluminescence immunoassay (ECLIA) to detect SARS-CoV-2 nucleocapsid protein in nasopharyngeal and oropharyngeal swab samples from patients with signs and symptoms suggestive of COVID‑19, or known or suspected exposure to SARS‑CoV‑2 . The manufacturer has described the sample stability when using three different liquid transport media in the package insert. The assay is intended for use on the cobas e411, e601 and e602 analyzers. A result of COI ≥ 1.0 can be interpreted as reactive for SARS‑CoV‑2 Ag. The manufacturer describes the clinical performance data of the test based on three cohorts and gives different sensitivities and specificities for symptomatic/asymptomatic patients with different disease duration and viral load. Values between 65.5 and 100% are given for sensitivity, and between 99.8 and 100% for specificity .
The SARS-CoV-2 Antigen ELISA ("ELISA") from Euroimmun Medizinische Labordiagnostika AG (Lübeck, Germany) is a semi-quantitative enzyme-linked immunosorbent assay (ELISA) for the in vitro detection of SARS-CoV-2 nucleocapsid from nasopharyngeal swabs . The reaction tubes are coated with a monoclonal anti-SARS-CoV-2 nucleocapsid antibody. For interpretation of results, ratios ≥ 0.60 results are considered positive. Clinical performance was determined by the manufacturer using 98 nasopharyngeal swabs, with PCR-positive samples all from symptomatic patients < 10 days after symptom onset. A sensitivity of 93.6% and a specificity of 100% are reported .
Quantitative viral load determination
The following PCR assays were used for quantification in the accredited routine diagnostics laboratory of the Max von Pettenkofer Institute : the nucleocapsid (N1) reaction (Center for Disease Control (CDC) protocol , the nucleocapsid amplification (Seegene Allplex 2019-nCoV Assay), the Roche Cobas SARS-CoV-2 nucleocapsid reaction or the Xpert Xpress SARS-CoV-2/Flu/RSV run on the GeneXpert System. For nucleic acid extraction, the Maxwell RSC Viral Total Nucleic Acid Purification Kit was used with the Maxwell RSC-48 device (Promega GmbH, Fitchburg, USA). Quantification was based on two reference samples from INSTAND e.V.  with 106 and 107 RNA copies per ml and reference patient sample-based dilution series. The reference and dilution samples were tested in duplicate with the respective instruments and by different methods.
Standard curves, specific to each instrument and method, were generated based on the results of the analyses and calculated using factors for slope and y-intercept, natural logarithmic function equations and by calibrating the second standard dilution on the INSTAND reference material . For the CDC protocol, Seegene Allplex Assay, Roche Cobas and GeneXpert System, the following formulas were used, respectively: y = − 146ln(x) + 46,721; y = − 1,48ln(x) + 45,118; y = − 1,401ln(x) + 44,576; y = − 1,5ln(x) + 45,904. In general, the calculations for quantification do not take into account variability between separate PCR runs. However, since this variability applies to all study groups, they do not affect the interpretation of the results in this study.
Analysis of SARS-CoV-2 whole-genome sequencing
Amplicon pools covering the SARS-CoV-2 genome were prepared according to the ARTIC network nCoV-2019 sequencing protocol v2 and analyzed utilizing the Artic bioinformatics protocol, in principle as reported . The consensus sequences and associated sample metadata were uploaded to the GISAID repository.
Propagation of SARS-CoV-2 from primary patient material
Caco-2 cells (American Type Culture Collection, ATCC, Virginia, USA) in virus isolation medium (Dulbecco's modified Eagle's medium (Gibco, ThermoFisher) containing 2% fetal bovine serum (Sigma-Aldrich), 100 U/ml penicillin–streptomycin (Sigma-Aldrich), 1 × non-essential amino acids (Gibco, ThermoFisher), 0.5 µg/ml gentamicin (Ratiopharm), and 0.25 µg/ml amphotericin B (Gibco, ThermoFisher)) were challenged for 2 h with a clinical isolate (GISAID EPI ISL: 466888) previously obtained from a nasopharyngeal swab of a COVID-19 patient. Subsequently, virus isolation medium was replaced with culture medium, and 3 days post-infection the supernatant was collected and passaged onto Vero-E6 cells (American Type Culture Collection, ATCC, Virginia, USA). After three additional days, cell culture supernatants were harvested and stored at − 80 °C. Further propagation of virus was performed in the expansion medium (Dulbecco's modified Eagle's medium containing 5% fetal bovine serum (Sigma-Aldrich), 100 U/ml penicillin–streptomycin (Sigma-Aldrich) and 1 × non-essential amino acids (Gibco, ThermoFisher)). Alpha VOC (B.1.1.7; GISAID EPI ISL: 2094739) and Beta (B.1.351; GISAID EPI ISL: 1752394) were initially grown on CaCo-2 or Vero-E6 cells and then expanded on Vero-E6 cells. Virus stocks were characterized by real-time RT-PCR as reported previously .
Statistical analysis was performed in R version 4.1.0, using the pROC package to perform receiver operator characteristic curve (ROC curve) analysis . Binomial confidence intervals for sensitivities and specificities were computed using the Wilson score interval. To further analyze analytical sensitivities, we used logistic regression, with viral loads as independent and test outcomes as the dependent variable, yielding detection probabilities for each viral load level.
Specificities of quantitative, automated SARS-CoV-2 antigen tests range between 97.0 and 99.7%
The specificity of four quantitative, automated SARS-CoV-2 Ag tests was compared to the qualitative POCT from Roche Diagnostics. Samples taken from nasopharynx (n = 118), oropharynx (n = 174) or unrecorded sampling site in the upper respiratory tract (n = 11) from 303 SARS-CoV-2 PCR-negative hospitalized adults were analyzed (Table 1). The specificity of the quantitative SARS-CoV-2 Ag tests ranged from 97.03% for CLEIA, over 97.69% for ECLIA to 99.67% for CLIA and ELISA, while RAT showed a specificity of 100% in this sample collection, higher than previously reported .
Analytical performance of quantitative, automated SARS-CoV-2 antigen tests
To evaluate the analytical performance of the quantitative SARS-CoV-2 Ag tests, we included swabs taken by health-care professionals from the nasopharynx (n = 83), oropharynx (n = 19) or unrecorded sampling sites in the upper respiratory tract (n = 5) from 107 SARS-CoV-2 PCR-positive hospitalized adults with viral loads ranging between 83 and 1,548,572,803 Geq/ml (median: 6,045 Geq/ml; Fig. 1).
The analytical performance of the four quantitative SARS-CoV-2 Ag tests is depicted in Fig. 2. Here, the cutoffs for positive and negative scoring were set according to the manufacturers’ recommendations. Accordingly, 56 PCR-positive SARS-CoV-2 patients were tested true positive with CLEIA, corresponding to a sensitivity of 52.34% (Fig. 2a, Table 2).
Only 21 PCR-positive swabs were tested true positive with CLIA, reflecting a sensitivity of 19.63% (Fig. 2b, Table 2). The ELISA showed a comparable, low sensitivity of 17.76% (Fig. 2c, Table 2). In the ECLIA, 33 PCR-positive COVID-19 patient samples were true positive corresponding to a sensitivity of 31.42% (Fig. 2d, Table 2). In comparison, the RAT scored 25 PCR-positive SARS-CoV-2 patients true positive, reflecting a sensitivity of 23.58% (Table 2).
Next, we analyzed the receiver-operating characteristics (ROCs) to evaluate the overall performance of the quantitative SARS-CoV-2 Ag tests (Fig. 3). The calculated area under the curves (AUCs) indicated that CLEIA performed best with an AUC of 0.873. Second among the four automated and quantitative SARS-CoV-2 Ag tests was ECLIA with an AUC of 0.670. In comparison, ELISA and CLIA had an AUC of 0.650 and 0.516, respectively.
Based on these ROC analyses, we then determined the Youden’s index, which is a measure of a diagnostic test's ability to balance sensitivity and specificity. The maximal Youden’s index of 0.6525 reached an optimal cutoff of 0.095 for CLEIA, yielding a sensitivity of 79% and a corresponding specificity of 86% (Table 3). Of note, the cutoff with the calculated Youden’s index was significantly lower than that recommended by the manufacturer . Aiming for a minimal sensitivity of 80%, recommended by the WHO for SARS-CoV-2 rapid Ag tests, would lower the specificity of CLEIA to 85% only. Conversely, a specificity of 97% would decrease the corresponding sensitivity to 52%. For CLIA, the sensitivity and specificity were 27% and 99%, respectively, with a maximal Youden’s index of 0.2611 and a cutoff of 115.92. As a consequence, aiming at a sensitivity of 80% would lower the specificity to only 11%, while a specificity of 97% would yield a sensitivity of 27%. The ELISA showed an optimal cutoff of 0.325 with a Youden’s index of 0.2721 and a sensitivity and specificity of 31% and 96%, respectively. Here, a sensitivity of 80% would drastically reduce the specificity to 29%, yet a target specificity of 97% would only marginally decrease the sensitivity to 30%. Only the cutoff based on the calculated Youden’s index (0.3007) for ECLIA was very close to the manufacturer’s recommendation and reached a sensitivity and specificity of 33% and 98%, respectively. The specificity of this assay would have to be lowered to 30% to reach a sensitivity of 80%. Conversely, a target specificity of 97% would only marginally decrease the sensitivity to 32%.
We further assessed the analytical sensitivity of the quantitative and qualitative SARS-CoV-2 Ag tests (Fig. 4) by calculating the 50% and 95% limits of detection (LoD) based on a logistic regression model as recently reported . The virus concentrations at which 50% and 95% detection rates were achieved with CLEIA were 6,181 and 422,689 Geq/ml, respectively. In comparison, LoD50 and LoD95 values for CLIA were 473,279 and 11,452,782 Geq/ml, i.e. 77 and 27 times higher, respectively. The performance of the ELISA was even worse with 121 and 61 times higher LoD50 and LoD95 values, respectively, corresponding to 749,792 and 25,711,669 Geq/ml. The LoD50 and LoD95 for ECLIA were only 11- and 6-times lower than those for the CLEIA, corresponding to 69,002 and 2,654,696 Geq/ml, respectively. In comparison to the quantitative SARS-CoV-2 Ag tests, the LoD50 and LoD95 values of RAT yielded 255,537 and 9,324,079 Geq/ml, respectively, i.e. 41 and 22 times higher relative to CLEIA.
Quantitative, automated SARS-CoV-2 antigen tests are able to detect VOCs Alpha and Beta
Given the increasing genetic diversification of SARS-CoV-2 and the emergence of VOCs, which carry mutations not only in spike but also in nucleocapsid, we assessed whether these quantitative SARS-CoV-2 Ag tests are also able to detect VOCs Alpha and Beta. First, we evaluated the detection of clinical SARS-CoV-2 isolates that had been expanded in tissue culture and confirmed as a VOC by whole-genome sequencing . As shown in Fig. 5, both VOCs Alpha and Beta were detected by all quantitative Ag tests. While CLEIA was able to detect viral loads of an isolate from 2020 (EU1, B.1.177) as well as VOCs Alpha and Beta down to 1 × 105 Geq/ml, CLIA and ELISA were only able to reliably detect viral loads up to 8 × 106 Geq/ml. This was in a range similar to RAT, which was tested in parallel (data not shown). The ECLIA was slightly more sensitive and able to detect 5 × 105 Geq/ml for Alpha and 2 × 106 Geq/ml for Beta.
Second, the genotype of SARS-CoV-2 in respiratory swabs taken from 79 COVID-19 patients was determined by whole-genome sequencing. Since February of 2021 fell into the late phase of the second pandemic wave in Germany, 14% of our respiratory samples were already Alpha positive. Of note, the Alpha-positive patients’ median viral load was 1,929,014 Geq/ml, i.e. 320 times higher than the median of non-VOC SARS-CoV-2-positive respiratory samples from COVID-19 patients included in the study. Among the 19 COVID-19 patients with primary diagnosis, 4 were infected with VOC Alpha (21%). Non-VOC SARS-CoV-2 was detected in seven patients (36.8%) and for eight patients no genotypic information was available. Although the subgroup of patients with primary diagnosis was rather small, Alpha-infected patients had also here a median viral load of 12,424,703 Geq/ml compared to 9,528 Geq/ml in non-VOC-positive patients.
In the current study, we evaluated the performance of four commercial, automated SARS-CoV-2 Ag tests that were launched on the European market in early 2021 for the quantitative detection of SARS-CoV-2 in respiratory samples from COVID-19 patients hospitalized at the LMU Klinikum and three LMU München teaching hospitals as well as on cell culture-expanded clinical isolates, including VOCs. We assessed the sensitivity and specificity as well as method-based test characteristics of these four laboratory-based Ag assays and also compared them to a widely used POCT.
The minimum requirements of performance recommended by international organizations for rapid SARS-CoV-2 Ag tests (sensitivity ≥ 80%, specificity > 97%) are certainly not simply transferable to automated Ag assays. Since this study was conducted at a time during the pandemic when the number of newly infected individuals in Germany was declining at the end of the second pandemic wave, it was difficult to include primarily fresh respiratory samples from patients at the time of initial diagnosis and presumably high viral load. Therefore, the low clinical sensitivities, which only have relevance in the specific cohort investigated, are not discussed further. The analytical sensitivity (95% detection limit) of the four automated SARS-CoV-2 Ag tests examined ranged from 420,000 to 25,000,000 Geq/ml. In comparison, the detection limit of the Roche Rapid Ag Test was 9,300,000 Geq/ml, which is also in good agreement with a recent report .
This indicates that automated Ag testing does not necessarily go along with an increased sensitivity. However, it is evident that test indication and overall usefulness of automated Ag tests cannot be generalized. Rather, cutoffs must be individually adapted to and evaluated for the specific diagnostic focus. The ROC analyses in this study demonstrate impressively that lowering the cutoff for either specificity or sensitivity is not suitable for every test. In the case of CLIA, ELISA and ECLIA, it is evident that cutoffs have already been set by the manufacturers just above the lower asymptote of negative results. Only CLEIA shows a wide dynamic range that allows marked variations. For this assay, the manufacturer also explicitly recommends adjusting the cutoff to the requirements of the specific laboratory performing the assay. Recently, Häuser et al. performed similar analyses on the automated CLIA . In their study cohort of more than 130 COVID-19 patients hospitalized in the period December 9, 2020 to January 29, 2021, the overall analytical sensitivity was 40.2% at a specificity of 100%, while lowering the assay cutoff from 200 to 100 arbitrary units (AU) per mL increased the sensitivity to 49.7% while decreasing the specificity to 98.3%.Possible areas of application of automated Ag tests are the screening of asymptomatic patient groups with regard to possible infectivity at the time of testing during a phase of increased incidence or testing for de-isolation of known PCR-positive patients in the hospital setting in addition to local symptoms-dependent hygiene concepts.
The specificity of an Ag test determines the number of false-positive results depending on the pre-test probability. Independent studies and meta-analyses  now show that high specificity can be achieved with SARS-CoV-2 rapid Ag tests [2, 3]. In our current analysis, the RAT used confirms this observation. Independent testing and summaries of automated Ag test specificities are also important, as substantial variability can occur due to individually definable cutoff values for tests that have a quantitative or semi-quantitative output measure. In our current study, all Ag tests investigated were able to stay above the limit required by the WHO for the specificity of SARS-CoV-2 rapid Ag tests of more than 97%. To our knowledge, there is no corresponding recommendation for automated laboratory Ag tests from any official institution. Laboratory operators should responsibly and proactively plan the use of automated Ag tests depending on local incidence and circulating variants, the latter potentially also affecting assay performance.
Undoubtedly, any positive Ag test, POCT or laboratory based, has to be confirmed by PCR. Specificities of around 97%, as found in our current study for the CLEIA and ECLIA, can lead to significant rates of false-positive results and corresponding burden on laboratory workflows due to a need for PCR retesting with associated delays in the time to result.
Recent analyses showed that individuals infected with VOC Alpha had a viral load at primary diagnosis of one order of magnitude higher than individuals infected with a non-VOC [36,37,38,39,40,41]. We were able to observe this trend between these two patient groups, despite the small number of COVID-19 cases. As a consequence, Alpha VOC-infected patients may be more likely to be recognized by an Ag test early during the course of infection. Unfortunately, we were unable to demonstrate such an effect, likely due to the small number of primary diagnoses. In this context, however, it is more important to investigate whether different virus variants—especially VOCs—can be detected by automated Ag tests with comparable sensitivity. Changes in the virus genome can lead to non-synonymous amino acid exchanges. In the spike protein, for example, the polymorphisms K417N and E484K in VOC Beta can reduce the efficacy of neutralizing antibodies. The nucleocapsid gene harbors a mutational hotspot in the region around amino acid positions 202–205, which is probably an important phosphorylation site for packaging. While these SNPs are not VOC specific, almost all current SARS-CoV-2 lineages carry mutations in this area of the genome compared to the Wuhan reference sequence. In addition to these four amino acid positions, there are five other amino acids in nucleocapsid that can differ between VOCs and the early pandemic isolates (D3, D63, P80, G215, S235 and D377). Mutations in the nucleocapsid of the major VOCs and the wild-type virus used in this study are shown in Table 4 .
Four of the five assays reported in this study are based on monoclonal antibodies that bind to the viral nucleocapsid protein in the immunoassay. Only CLIA uses polyclonal antibodies for Ag detection and Fujirebio states for the CLEIA that they use a mixture of different monoclonal antibodies. Although the stage of infection and pre-analytical conditions may have the greatest impact on a false-negative Ag test result, it is essential to also investigate failures in the binding ability of the antibodies used in these assays due to VOC-specific mutations. An immunoassay that includes polyclonal antibodies certainly has a conceptual advantage of reliably detecting multiple epitopes on the nucleocapsid protein even if mutations occur at a single site, whereas the use of a monoclonal antibody in the assay design is more vulnerable to drastic loss of binding and assay failure . Since different manufacturers are likely to have the diagnostic antibodies bind in different domains of the nucleocapsid protein, this survey must be repeated for each test on the market. It can be assumed that Ag test antibodies react to more conserved domains of the nucleocapsid, but so far this information has not been disclosed by the manufacturers. In our experiments with supernatants from cell culture of primary patient clinical isolates of Alpha and Beta, we did not observe a marked drop in test sensitivity. This result is in line with the findings of a recent publication in which no difference was found between cell culture supernatants of Alpha and Beta VOC isolates and non-VOC isolates for detection rates in RATs .
In summary, automated assays for the detection of SARS-CoV-2 nucleocapsid protein from respiratory samples significantly differ in their assay dynamics, with marked differences in analytical sensitivity. However, sensitivity is not consistently higher in automated, laboratory-based assays compared to a widely used RAT and no marked difference in detecting a SARS-CoV-2 isolate from early 2020 relative to VOC Alpha and Beta could be established. Nucleic acid-based testing remains the gold standard for high-performance SARS-CoV-2 detection and quantification in a laboratory setting.
Corman VM, Haage VC, Bleicker T, Schmidt ML, Mühlemann B, Zuchowski M, Jo WK, Tscheak P, Möncke-Buchner E, Müller MA, Krumbholz A, Drexler JF, Drosten C (2021) Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests: a single-centre laboratory evaluation study. Lancet Microbe 2(7):e311–e319. https://doi.org/10.1016/s2666-5247(21)00056-2
Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, Dittrich S, Emperador D, Takwoingi Y, Cunningham J, Beese S, Domen J, Dretzke J, Ferrante di Ruffano L, Harris IM, Price MJ, Taylor-Phillips S, Hooft L, Leeflang MM, McInnes MD, Spijker R, Van den Bruel A (2021) Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 3(3):Cd013705. https://doi.org/10.1002/14651858.CD013705.pub2
Osterman A, Baldauf HM, Eletreby M, Wettengel JM, Afridi SQ, Fuchs T, Holzmann E, Maier A, Döring J, Grzimek-Koschewa N, Muenchhoff M, Protzer U, Kaderali L, Keppler OT (2021) Evaluation of two rapid antigen tests to detect SARS-CoV-2 in a hospital setting. Med Microbiol Immunol 210(1):65–72. https://doi.org/10.1007/s00430-020-00698-8
Toptan T, Eckermann L, Pfeiffer AE, Hoehl S, Ciesek S, Drosten C, Corman VM (2021) Evaluation of a SARS-CoV-2 rapid antigen test: potential to help reduce community spread? J Clin Virol 135:104713. https://doi.org/10.1016/j.jcv.2020.104713
Hirotsu Y, Maejima M, Shibusawa M, Nagakubo Y, Hosaka K, Amemiya K, Sueki H, Hayakawa M, Mochizuki H, Tsutsui T, Kakizaki Y, Miyashita Y, Yagi S, Kojima S, Omata M (2020) Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients. Int J Infect Dis 99:397–402. https://doi.org/10.1016/j.ijid.2020.08.029
Amendola A, Sberna G, Lalle E, Colavita F, Castilletti C, Menchinelli G, Posteraro B, Sanguinetti M, Ippolito G, Bordi L, Capobianchi MR, On Behalf Of Inmi Covid-Study G (2021) Saliva is a valid alternative to nasopharyngeal swab in chemiluminescence-based assay for detection of SARS-CoV-2 antigen. J Clin Med 10:7. https://doi.org/10.3390/jcm10071471
Aoki K, Nagasawa T, Ishii Y, Yagi S, Okuma S, Kashiwagi K, Maeda T, Miyazaki T, Yoshizawa S, Tateda K (2021) Clinical validation of quantitative SARS-CoV-2 antigen assays to estimate SARS-CoV-2 viral loads in nasopharyngeal swabs. J Infect Chemother 27(4):613–616. https://doi.org/10.1016/j.jiac.2020.11.021
Asai N, Sakanashi D, Ohashi W, Nakamura A, Kawamoto Y, Miyazaki N, Ohno T, Yamada A, Chida S, Shibata Y, Kato H, Shiota A, Hagihara M, Koita I, Yamagishi Y, Suematsu H, Ohta H, Mikamo H (2021) Efficacy and validity of automated quantitative chemiluminescent enzyme immunoassay for SARS-CoV-2 antigen test from saliva specimen in the diagnosis of COVID-19. J Infect Chemother 27(7):1039–1042. https://doi.org/10.1016/j.jiac.2021.03.021
Favresse J, Gillot C, Oliveira M, Cadrobbi J, Elsen M, Eucher C, Laffineur K, Rosseels C, Van Eeckhoudt S, Nicolas JB, Morimont L, Dogné JM, Douxfils J (2021) Head-to-head comparison of rapid and automated antigen detection tests for the diagnosis of SARS-CoV-2 infection. J Clin Med 10:2. https://doi.org/10.3390/jcm10020265
Gili A, Paggi R, Russo C, Cenci E, Pietrella D, Graziani A, Stracci F, Mencacci A (2021) Evaluation of Lumipulse® G SARS-CoV-2 antigen assay automated test for detecting SARS-CoV-2 nucleocapsid protein (NP) in nasopharyngeal swabs for community and population screening. Int J Infect Dis 105:391–396. https://doi.org/10.1016/j.ijid.2021.02.098
Hirotsu Y, Maejima M, Shibusawa M, Amemiya K, Nagakubo Y, Hosaka K, Sueki H, Hayakawa M, Mochizuki H, Tsutsui T, Kakizaki Y, Miyashita Y, Omata M (2021) Prospective study of 1308 nasopharyngeal swabs from 1033 patients using the LUMIPULSE SARS-CoV-2 antigen test: comparison with RT-qPCR. Int J Infect Dis 105:7–14. https://doi.org/10.1016/j.ijid.2021.02.005
Häuser F, Sprinzl MF, Dreis KJ, Renzaho A, Youhanen S, Kremer WM, Podlech J, Galle PR, Lackner KJ, Rossmann H, Lemmermann NA (2021) Evaluation of a laboratory-based high-throughput SARS-CoV-2 antigen assay for non-COVID-19 patient screening at hospital admission. Med Microbiol Immunol 210(2–3):165–171. https://doi.org/10.1007/s00430-021-00706-5
Lefever S, Indevuyst C, Cuypers L, Dewaele K, Yin N, Cotton F, Padalko E, Oyaert M, Descy J, Cavalier E, Van Ranst M, André E, Lagrou K, Vermeersch P (2021) Comparison of the quantitative diasorin liaison antigen test to reverse transcription-PCR for the diagnosis of COVID-19 in symptomatic and asymptomatic outpatients. J Clin Microbiol 59(7):e0037421. https://doi.org/10.1128/jcm.00374-21
Menchinelli G, Bordi L, Liotti FM, Palucci I, Capobianchi MR, Sberna G, Lalle E, Romano L, De Angelis G, Marchetti S, Sanguinetti M, Cattani P, Posteraro B (2021) Lumipulse G SARS-CoV-2 Ag assay evaluation using clinical samples from different testing groups. Clin Chem Lab Med 59(8):1468–1476. https://doi.org/10.1515/cclm-2021-0182
Yin N, Debuysschere C, Decroly M, Bouazza FZ, Collot V, Martin C, Ponthieux F, Dahma H, Gilbert M, Wautier M, Duterme C, De Vos N, Delforge ML, Malinverni S, Cotton F, Bartiaux M, Hallin M (2021) SARS-CoV-2 diagnostic tests: algorithm and field evaluation from the near patient testing to the automated diagnostic platform. Front Med (Lausanne) 8:650581. https://doi.org/10.3389/fmed.2021.650581
Yokota I, Shane PY, Okada K, Unoki Y, Yang Y, Iwasaki S, Fujisawa S, Nishida M, Teshima T (2021) A novel strategy for SARS-CoV-2 mass screening with quantitative antigen testing of saliva: a diagnostic accuracy study. Lancet Microbe. https://doi.org/10.1016/S2666-5247(21)00092-6
Paul-Ehrlich Institute (2021) Mindestkriterien für SARS-CoV-2 Antigentests im Sinne von § 1 Abs. 1 Satz 1 TestVO: antigenschnelltests (in German). https://www.pei.de/SharedDocs/Downloads/DE/newsroom/dossiers/mindestkriterien-sars-cov-2-antigentests-01-12-2020.pdf?__blob=publicationFile. Accessed 19 Jul 2021
World Health Organization (WHO) (2020) Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays. Accessed 19 Jul 2021
European Centre for Disease Prevention and Control (2021) COVID-19 report. https://covid19-surveillance-report.ecdc.europa.eu/archive-COVID19-reports/index.html. Accessed 19 Jul 2021
World Health Organization (WHO) (2021) Weekly epidemiological update on COVID-19. https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---13-april-2021. Accessed 19 Jul 2021
Tegally H, Wilkinson E, Giovanetti M, Iranzadeh A, Fonseca V, Giandhari J, Doolabh D, Pillay S, San EJ, Msomi N, Mlisana K, von Gottberg A, Walaza S, Allam M, Ismail A, Mohale T, Glass AJ, Engelbrecht S, Van Zyl G, Preiser W, Petruccione F, Sigal A, Hardie D, Marais G, Hsiao NY, Korsman S, Davies MA, Tyers L, Mudau I, York D, Maslo C, Goedhals D, Abrahams S, Laguda-Akingba O, Alisoltani-Dehkordi A, Godzik A, Wibmer CK, Sewell BT, Lourenço J, Alcantara LCJ, Kosakovsky Pond SL, Weaver S, Martin D, Lessells RJ, Bhiman JN, Williamson C, de Oliveira T (2021) Detection of a SARS-CoV-2 variant of concern in South Africa. Nature 592(7854):438–443. https://doi.org/10.1038/s41586-021-03402-9
SARS-CoV2 lineages (2021) Latest epidemiological lineages of SARS-CoV-2. https://cov-lineages.org/. Accessed 19 Jul 2021
Alaa Abdel Latif JLM, Manar A, Ginger T, Marco C, Emily H, Jerry Z, Mark Z, Emory H, Nate M, Chunlei W, Kristian GA, Andrew IS, Karthik G, Laura DH, and the Center for Viral Systems Biology. http://outbreak.info. Accessed 21 Jul 2021
Muenchhoff M, Mairhofer H, Nitschko H, Grzimek-Koschewa N, Hoffmann D, Berger A, Rabenau H, Widera M, Ackermann N, Konrad R, Zange S, Graf A, Krebs S, Blum H, Sing A, Liebl B, Wölfel R, Ciesek S, Drosten C, Protzer U, Boehm S, Keppler OT (2020) Multicentre comparison of quantitative PCR-based assays to detect SARS-CoV-2, Germany, March 2020. Euro Surveill 25:24. https://doi.org/10.2807/1560-7917.Es.2020.25.24.2001057
Roche Diagnostics GmbH SARS-CoV-2 Rapid Antigen Test, Package Insert. https://assets.cwp.roche.com/f/94122/x/2bfe52794a/2-packungsbeilage_sars-cov-2_rapid_antigen_test-c-roche.pdf. Accessed 14 Apr 2021
Nishizono I, Iida S, Suzuki N, Kawada H, Murakami H, Ashihara Y, Okada M (1991) Rapid and sensitive chemiluminescent enzyme immunoassay for measuring tumor markers. Clin Chem 37(9):1639–1644
Fujirebio Inc. Lumipulse G SARS-CoV-2 Ag, Package Insert. https://www.e-labeling.eu/FRI. Accessed 21 Apr 2021
DiaSorin S.p.A. LIAISON®SARS-CoV-2 Ag instructions for use. https://www.marigolab.it/wp-content/uploads/2020/11/IFUk_en_311490_01.pdf. Accessed 21 Apr 2021
Roche Diagnostics GmbH Immunoassay Elecsys® Anti-SARS-CoV-2 Instructions for use. https://diagnostics.roche.com/ch/de/products/params/elecsys-anti-sars-cov-2.html. Accessed 21 Apr 2021
Euroimmun Medizinische Labordiagnostika AG (2020) Anti-SARS-CoV-2 ELISA (IgG) Instructions for use. https://www.fda.gov/media/137609/download. Accessed 21 Apr 2021
Centers for Disease Control and Prevention (CDC) Information for laboratories about coronavirus (COVID-19). https://www.cdc.gov/coronavirus/2019-ncov/lab/index.html. Accessed 16 Apr 2021
INSTAND e.V. Quantitative Bezugsproben zur Verbesserung der Vergleichbarkeit und Bewertungvon Laborergebnissenzum Virusgenom-Nachweis von SARS-CoV-2 (in German). https://www.instand-ev.de/fileadmin/uploads/user_upload/Dokumente/Virologie/20210118g_Begleitheft_-_quantitative_Bezugsproben_1_und_2_-_SARS-CoV-2.pdf. Accessed 21 Apr 2021
Autenrieth H (2021) Etablierung der RNA-Quantifizierung für den Nachweis von SARS-CoV-2-RNA mit verschiedenen PCR-Testsystemen (in German). LMU Munich, Germany, Bachelor-Thesis
Weinberger T, Steffen J, Osterman A, Mueller TT, Muenchhoff M, Wratil PR, Graf A, Krebs S, Quartucci C, Spaeth PM, Grabein B, Adorjan K, Blum H, Keppler OT, Klein M (2021) Prospective longitudinal serosurvey of health care workers in the first wave of the SARS-CoV-2 pandemic in a quaternary care hospital in Munich Germany. Clin Infect Dis. https://doi.org/10.1093/cid/ciaa1935
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform 12(1):77. https://doi.org/10.1186/1471-2105-12-77
Calistri P, Amato L, Puglia I, Cito F, Di Giuseppe A, Danzetta ML, Morelli D, Di Domenico M, Caporale M, Scialabba S, Portanti O, Curini V, Perletta F, Cammà C, Ancora M, Savini G, Migliorati G, D’Alterio N, Lorusso A (2021) Infection sustained by lineage B.1.1.7 of SARS-CoV-2 is characterised by longer persistence and higher viral RNA loads in nasopharyngeal swabs. Int J Infect Dis 105:753–755. https://doi.org/10.1016/j.ijid.2021.03.005
Challen R, Brooks-Pollock E, Read JM, Dyson L, Tsaneva-Atanasova K, Danon L (2021) Risk of mortality in patients infected with SARS-CoV-2 variant of concern 202012/1: matched cohort study. BMJ 372:n579. https://doi.org/10.1136/bmj.n579
Frampton D, Rampling T, Cross A, Bailey H, Heaney J, Byott M, Scott R, Sconza R, Price J, Margaritis M, Bergstrom M, Spyer MJ, Miralhes PB, Grant P, Kirk S, Valerio C, Mangera Z, Prabhahar T, Moreno-Cuesta J, Arulkumaran N, Singer M, Shin GY, Sanchez E, Paraskevopoulou SM, Pillay D, McKendry RA, Mirfenderesky M, Houlihan CF, Nastouli E (2021) Genomic characteristics and clinical effect of the emergent SARS-CoV-2 B.1.1.7 lineage in London, UK: a whole-genome sequencing and hospital-based cohort study. Lancet Infect Dis. https://doi.org/10.1016/S1473-3099(21)00170-5
Jones TC, Biele G, Mühlemann B, Veith T, Schneider J, Beheim-Schwarzbach J, Bleicker T, Tesch J, Schmidt ML, Sander LE, Kurth F, Menzel P, Schwarzer R, Zuchowski M, Hofmann J, Krumbholz A, Stein A, Edelmann A, Corman VM, Drosten C (2021) Estimating infectiousness throughout SARS-CoV-2 infection course. Science 373:6551. https://doi.org/10.1126/science.abi5273
Kidd M, Richter A, Best A, Cumley N, Mirza J, Percival B, Mayhew M, Megram O, Ashford F, White T, Moles-Garcia E, Crawford L, Bosworth A, Atabani SF, Plant T, McNally A (2021) S-variant SARS-CoV-2 lineage B117 is associated with significantly higher viral load in samples tested by TaqPath polymerase chain reaction. J Infect Dis 223(10):1666–1670. https://doi.org/10.1093/infdis/jiab082
Ratcliff J, Nguyen D, Fish M, Rynne J, Jennings A, Williams S, Al-Beidh F, Bonsall D, Evans A, Golubchik T, Gordon AC, Lamikanra A, Tsang P, Ciccone NA, Leuscher U, Slack W, Laing E, Mouncey PR, Ziyenge S, Oliveira M, Ploeg R, Rowan KM, Shankar-Hari M, Roberts DJ, Menon DK, Estcourt L, Simmonds P, Harvala H (2021) Virological and serological characterization of critically ill patients with COVID-19 in the UK: interactions of viral load, antibody status and B.1.1.7 variant infection. J Infect Dis. https://doi.org/10.1093/infdis/jiab283
Ascoli CA (2021) Could mutations of SARS-CoV-2 suppress diagnostic detection? Nat Biotechnol 39(3):274–275. https://doi.org/10.1038/s41587-021-00845-3
Jungnick S, Hobmaier B, Mautner L, Hoyos M, Haase M, Baiker A, Lahne H, Eberle U, Wimmer C, Hepner S, Sprenger A, Berger C, Dangel A, Wildner M, Liebl B, Ackermann N, Sing A, Fingerle V (2021) Detection of the new SARS-CoV-2 variants of concern B.1.1.7 and B.1.351 in five SARS-CoV-2 rapid antigen tests (RATs), Germany, March 2021. Euro Surveill 26:16. https://doi.org/10.2807/1560-7917.Es.2021.26.16.2100413
We are grateful to members of the Diagnostic Laboratory of Virology at the Max von Pettenkofer Institute for support. We like to thank Volker Fingerle, Nikolaus Ackermann, Alexandra Dangel, Ute Eberle, Mona Hoyos, Lena Mautner, Andreas Sing and the Bavarian SARS-CoV-Public Health Laboratory Epidemiology Team at the LGL for support. We thank Dr. Xaver Sewald for critical discussions.
Open Access funding enabled and organized by Projekt DEAL. This work was supported in part by the German BMBF initiative “NaFoUniMedCovid19” (01KX2021), subproject B-FAST (to U.P. and O.T.K.), and by the Free State of Bavaria (research initiatives Bay-VOC (M.M., A.G., A.B., S.K., H.B., U.P., O.T.K.) and FORCOVID (M.M., U.P., O.T.K.)).
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval was not required because all samples used were residual samples. Samples were identified in the laboratory and assigned a number without reference to patient or clinical details. Results from testing using the study assays did not impact on clinical care.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Edited by Matthias J. Reddehase.
About this article
Cite this article
Osterman, A., Iglhaut, M., Lehner, A. et al. Comparison of four commercial, automated antigen tests to detect SARS-CoV-2 variants of concern. Med Microbiol Immunol (2021). https://doi.org/10.1007/s00430-021-00719-0
- Automated SARS-CoV-2 antigen test
- Nucleocapsid protein
- Diagnostic test