INTRODUCTION

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection rates continue to rise globally. The rapid identification and isolation of infected individuals are key components to prevent viral transmission.1 The current gold standard in SARS-CoV-2 diagnostics is the nucleic acid-based reverse transcription-polymerase chain reaction (RT-PCR) of nasopharyngeal (NP) swab specimens. However, PCR-based methods are relatively expensive and often time-consuming. Antigen tests (AgTs) for the detection of SARS-CoV-2 offer an inexpensive and simple alternative for rapid virus detection. Since the approval of the first SARS-CoV-2 AgT by the US Food and Drug Administration (FDA) in May 2020, more than 150 tests have been approved.2

Data on the use of AgTs in the clinical setting are crucial; however, only very few studies are available. In a meta-analysis including 18 publications and a total of 3198 samples, four AgTs were evaluated.3 The average sensitivity was 56% and varied widely between different studies (0–94%). The differences in test sensitivity and the studies may be attributable to the heterogeneity between study populations, with various disease severity levels and sampling at different time points. Furthermore, a contributing role may be the use of different RT-PCR assays, sample types, and extraction methods.

The present study evaluated the test characteristics of five different SARS-CoV-2 AgTs, including four rapid antigen tests (RATs) and one semi-automated chemiluminescence immunoassay (CLIA) in NP swabs, and compared the test sensitivities with the RT-PCR. Furthermore, the effect of age, sex, time from symptom onset, and virus load were also analysed.

MATERIAL AND METHODS

Study Population

The study was performed at the Department of Infectious Diseases and Tropical Medicine, Clinic Favoriten, Vienna, Austria. Routine testing for SARS-CoV-2 from NP swabs via RT-PCR was performed twice weekly during hospitalization. In the case of a positive result, patients were invited to participate in the study. After obtaining patient consent, additional NP swabs were collected for Ag (antigen) testing on the day of the next routine test, e.g. after 3–4 days. In the case of a negative RT-PCR upon subsequent testing due to viral clearance, samples were excluded from analysis. Time from symptom onset, sex, and age were recorded. The study was conducted over 2.5 months.

SARS-CoV-2 Antigen Tests

Four RATs for the qualitative detection of the SARS-CoV-2 nucleocapsid protein in NPs were evaluated: AMP Rapid Test SARS-CoV-2 Ag (AMP) from AMP Diagnostics (Graz, Austria); NADAL® COVID-19 Antigen Rapid Test (NADAL) from nal von minden GmbH (Moers, Germany); CLINITEST® Rapid COVID-19 Antigen Test (CLINITEST) from Siemens Healthcare (Erlangen, Germany); and SARS-CoV-2 Rapid Antigen Test (Roche) from Roche Diagnostics GmbH (Mannheim, Germany). These RATs use gold-conjugated monoclonal antibodies against the SARS-CoV-2 nucleocapsid protein and control antibodies that react with immobilized counterparts in the testing regions to provide a test line and a control line. Testing was performed using a comparably similar procedure. Briefly, following NP sample collection, the swab was inserted into a sample tube, and extraction buffer was added to extract viral cells from the sample. After 1–2 min, the swab was removed and 3–5 droplets added to a test strip. The test strip is conjugated with monoclonal antibodies against the nucleocapsid protein of SARS-CoV-2. If the sample contains SARS-CoV-2 nucleocapsid antigen, a complex forms with the SARS-2 antibodies and a coloured line appears in the test line within 15 min. A second line, the control line (C), indicates whether sufficient sample or buffer volume has been added, demonstrating the validity of the test. If the sample does not contain SARS-CoV-2 nucleocapsid antigen, no colour appears in the test line, only the control line. The main difference between the examined RATs is the limits of detection (LoD), which are provided in TCID50/ml\L (median tissue culture infection dose): 1.15 × 102 TCID50/mL for AMP, 2 × 102.4 TCID50/mL for NADAL, 1.15 × 102 TCID50/mL for CLINITEST, and 3.12 × 102.2 TCID50/mL for Roche. Further, it is speculated that different extraction buffers may be used; however, this information is unavailable.

The semi-automated AgT included a direct, two-step sandwich CLIA-based technology for the quantitative determination of the nucleocapsid protein (LIAISON® SARS-CoV-2 assay (LIAISON); DiaSorin, Saluggia, Italy). Relative light units (RLUs) obtained from samples or controls and directly proportional to calibrators were used to calculate the antigen concentration, expressed as TCID50/mL.

Material Collection and Processing

Fifty samples were collected for analysis via the RATs, whereas 100 samples were collected for analysis using the semi-automated CLIA.

Patients could be swabbed twice weekly as part of routine PCR testing during hospitalization and as a study participant. Each patient and sample was analysed in only one antigen test. Therefore, sample collection was performed until the required number of samples was achieved for each AgT, i.e. samples were collected from different patients for the next AgT.

Initially (collection time between 2.6. and 12.6.2020), Ag testing using the RAT NADAL was performed in conjunction with routine RT-PCR testing. Accordingly, a total of 50 samples from 27 patients were analysed using the RAT from NADAL. From mid-to-end June 2020, 50 samples from 28 patients were analysed using the RAT Roche. In early-to-mid July, 50 samples from 28 patients were analysed using the RAT from CLINITEST, whereas 50 samples from 26 patients were tested using the AMP RAT from mid-to-end July 2020. Thereafter, 100 consecutive samples collected from 23 individuals in August 2020 were analysed via the LIAISON assay. Processing of the 100 samples was performed based on two protocols (50 samples each) according to the manufacturer’s instructions. Patients were requested not to eat, drink, or brush their teeth for at least 30 min prior to specimen collection.

Rapid Antigen Tests (RATs)

Two NPs were obtained sequentially. The first swab was stored in a virus transport medium (VTM) and sent promptly to the laboratory for RT-PCR analysis. RAT was performed on the second swab immediately after sample collection at the patient’s bedside according to the manufacturer’s instructions. Qualitative results were recorded after 15 min.

Semi-automated Chemiluminescence Immunoassay (CLIA)

In accordance with the manufacturer’s recommendations, the initial 50 NPs were stored in VTM and sent to the laboratory at room temperature (RT). The VTM tube was rotated 10 times, and 1 mL of sample was transferred into a tube with an inactivation buffer. For the remaining 50 samples, the NP swab was stored in a tube with inactivation buffer immediately after sample collection.

For both collection methods, tubes with inactivation buffer were incubated for 3 h at RT. After centrifugation for 10 min at 3000g, the CLIA was performed using the Liason® XL (DiaSorin) analyser. Quantitative results were obtained by measuring the extinction of antigen/antibody reactions, considering values ≥ 200 TCID50/mL as positive.

SARS-CoV-2 RT-PCR

RT-PCR amplification was performed using the SARS-CoV-2 E-gene assay and NP material obtained from the first swab, according to Corman et al.4 Briefly, RNA was extracted from 200 μL of NP supernatants using the MagNa Pure 24 platform and the MagNa Pure 24 Total RNA Isolation Kit (Roche Diagnostics GmbH, Mannheim, Germany) according to the standard protocol and was eluted in a volume of 50 μL. Amplification and detection of SARS-CoV-2 were performed using a commercial primer/probe mix (LightMix® ModularDx Kit, SARS and Wuhan CoV E-gene, TIB Molbiol, Germany) and LC Multiplex RNA Virus Master (Roche Diagnostics GmbH, Mannheim, Germany) on a Cobas® z480 real-time RT-PCR instrument (Roche Diagnostics GmbH, Mannheim, Germany). Nuclease-free water and a synthetic RNA control provided with the primer/probe mix were included as positive and no-template control. Ten microlitres of RNA isolate and master mix were pooled and mixed, and RT-PCR was performed according to the manufacturer’s instructions. In the event of virus detection, the cycle threshold (Ct value) was documented. The Ct value represents the number of amplification cycles required for viral detection. Therefore, the lower the viral load in the sample, the higher the Ct value. Individuals with a Ct value higher than 30 are usually considered non-infectious in many countries.5 Further, studies have shown that infectivity is rather unlikely or low in individuals with a Ct value higher than 25.6 In our laboratory, an RT-PCR cutoff value of 38 cycles is considered positive. The RT-PCR comes to an end after 38 cycles, with results considered negative.

Statistical Analysis

Analysis of variance (ANOVA) was used to determine differences between patient samples analysed using the different AgTs for age, days from symptom onset, and the Ct value. Brown and Forsythe’s F-test was used to correct for heterogeneity of variance. A significant F-test was followed by the Games-Howell test for post hoc analysis to correct the familywise error rate. The chi-squared test was applied to assess differences between the different samples in terms of sex. An independent sample t-test was conducted to determine differences between the CLIA and RATs in terms of age, days from symptom onset, and Ct value. The chi-square test was used to test for differences between the CLIA and RATs with regard to sex. Means (M) and standard deviations (SD) are reported for normally distributed data and medians (MD) and interquartile ranges (IQR) for skewed distribution.

The sensitivity of the AgTs was calculated as the percentage of RT-PCR-positive patients that tested positive by the AgT. The chi-squared test was used to compare the sensitivity of the different AgTs. Differences between the various groups, including the different AgTs, two collection methods for CLIA, and RAT and CLIA samples, were analysed using the independent t-test for age (years), days from symptom onset, and Ct value, and the chi-squared test for sex.

Multiple logistic regressions were computed to model CLIA and RAT results using age (years), sex, days from symptom onset, and Ct values as predictors. Multicollinearity among the predictors was assessed using the variance inflation factor. No multicollinearity was present in the model. A significance level of 5% was used for statistical inference. Calculations were conducted using IBM® SPSS® Version 26 software.

Ethical Considerations

The study was approved by the local research ethics committee (approval number EK-20-208-0920). All participants provided written consent. This study received no specific grant from any funding agency.

RESULTS

From June to August 2020, 132 patients were enrolled in the study, providing a total of 300 samples. Baseline characteristics of the study population and patient subcohorts whose samples were collected for RAT and CLIA are summarized in Table 1. There was no significant difference in sex, age, days from symptom onset, and Ct value between the two groups (Table 1). Eleven patients were excluded due to a negative RT-PCR upon enrolment, indicating viral clearance during hospitalization. Following exclusion, there were 191 of 200 NP swab samples collected for RAT and 98 of 100 samples collected for CLIA available for analysis.

Table 1 Patient and Sample Characteristics of the Study Population
Full size table

Table 2 depicts the sensitivities of CLIA and RAT for the detection of SARS-COV-2 compared to RT-PCR, which is stratified by various CT values. Retrospectively, specificity analysis was performed on the 11 excluded patients with a negative RT-PCR result. The overall specificity was 90% (1 positive result from 10 negative results; 95% CI 55.5%; 99.8%). There was a false positive result for one RAT (CLINITEST), with a negative RT-PCR result, leading to a specificity of 87.5% (95% CI 47.4%; 99.7%). The specificity of CLIA was 100.0% (95% CI 15.8–100.0%).

Table 2 Sensitivity of the Antigen Tests Compared to 289 Positive RT-PCR Tests
Full size table

Comparison of Four RATs

There was no significant difference in age (p=0.548), sex distribution (p=0.087), or time from symptom onset (p=0.083) between the patient cohorts tested with the four different RATs. Patient samples analysed with the NADAL test had a higher median Ct value than samples analysed with the AMP RAT (p=0.017). Ct values were distributed equally across all other patient cohorts.

Compared to RT-PCR, the overall sensitivity of the RATs was 80% for the ROCHE, 72% for the AMP, 60% for the NADAL, and 58.5% for the CLINITEST assays. Table 3 shows the sensitivities of the evaluated tests grouped according to the Ct values. In samples with a Ct value <30, there was a trend towards a lower sensitivity of the NADAL test compared to ROCHE (p=0.066), as well as to CLINITEST (p=0.05).

Table 3 Comparison of the Sensitivity of the Individual Antigen Tests
Full size table

Comparison of Different Collection Methods for CLIA

Two collection methods were compared to assess the overall sensitivity of detecting SARS-CoV2. The first method involved inserting the NP swab into VTM after collection, followed by subsequent transfer into inactivation buffer in the laboratory. The overall sensitivity was 58% (29/50; 95% CI: 43.2%; 71.8%). The second method involved inserting the NP swab in inactivation buffer immediately after sample collection. The overall sensitivity was 39.6% (19/48 true positive samples; 95% CI: 25.8%; 54.8%). The sensitivity in samples with Ct values lower than 30 was 63.3% (95% CI: 43.8%; 80.0%) and 65.9% (95% CI: 50.1%; 79.5%) using the first and second methods, respectively. There was no significant difference between the sensitivity of the two collection methods (p=0.070). Table 3 depicts the sensitivity of the CLIA in all samples grouped according to the Ct value.

Effect of Collection Timepoint and Ct Value on Test Sensitivity

All RATs demonstrated a sensitivity of 100% until day 5 after symptom onset (Fig. 1), whereas the CLIA had a sensitivity of 100% until day 6 (Fig. 2). Subsequently, the sensitivity declined continuously. The mean number of days from symptom onset in samples with a positive AgT result was 6.7 (SD 2.1) and 7.4 (SD 2.6) days for RAT and CLIA, respectively. In patients with a negative result, the mean number of days from symptom onset was 11.5 (SD 3.6) and 12.3 (SD 3.9) (p<0.001).

Figure 1
figure 1

Overall sensitivity and confidence intervals of the AMP, NADAL, CLINITEST, and ROCHE rapid antigen tests in relation to time from symptom onset.

Full size image
Figure 2
figure 2

Overall sensitivity and confidence intervals of the CLIA in relation to time from symptom onset.

Full size image

In samples with a CT value <20, SARS-CoV-2 was detected in all cases using the different RATs. A decrease in positivity via RAT was observed with increasing C values (Table 3). The mean Ct value in samples with a true positive AgT result was 21.3 (SD 3.5) and 24.2 (SD 3.9) for RAT and CLIA, respectively, and for a false negative AgT result 30.2 (SD 3.1) and 30 (SD 4.5) (p<0.001), respectively.

Effect of Patient Characteristics on the Sensitivity of AgTs

The RATs had a lower sensitivity in females than in males (72.9 %, 95% CI: 62.9%; 81.4% vs. 85.3%, 95% CI: 77.0%; 91.5% respectively; p=0.05). There was no significant difference in the sensitivity of the CLIA in regard to sex (40%, 95% CI: 26.0%; 55.3% for females vs. 56.6%, 95% CI: 42.3%; 70.2% for males; p=0.339). Patient age did not significantly affect AgT results.

Multivariate Logistic Regression

A multivariate logistic regression analysis was performed for both test types (RAT and CLIA) to evaluate the impact of the Ct value, symptom onset, age, and sex on the probability of a true positive AgT (Table 4). These parameters could explain the 58.2% variability of AgT results (Nagelkerke R2 = 0.582) in the RAT model and 89.4% (Nagelkerke R2 = 0.894) for the CLIA. Both models were statistically significant (χ2(4) = 102.341, p<0.001; and χ2(4) = 110.892, p<0.001).

Table 4 Multivariable Logistic Regression Analysis to Evaluate the Effect of Sex, Age, Critical Threshold Value, and the Number of Days from Symptom Onset on the Probability of a True Positive Antigen Test
Full size table

A significantly inverse proportional effect of the Ct value or time from symptom onset on the probability of a true positive AgT was obtained in both models. Each unit increase in Ct value decreased the true positive rate by 15.7% and 59.3% for RAT and CLIA, respectively. Each increasing day from symptom onset led to a decrease in the AgT positive rate by 38.6% and 63.1% for RAT and CLIA, respectively. The male sex represented an independent factor for true positivity of AgT using RAT.

DISCUSSION

The COVID-19 AgTs evaluated in the present study demonstrated a good sensitivity for the detection of SARS-CoV-2 in samples with low Ct values, despite a lower overall sensitivity compared to RT-PCR. A sensitivity greater than 90% was obtained in samples with a Ct value lower than 25 in all AgTs analysed. An increasing Ct value correlated with a decrease in the sensitivity of the AgTs. Although infectivity is unlikely or low in individuals with a Ct value greater than 25,6 this does not present a clinically relevant disadvantage. The CLINITEST and ROCHE RATs met the recommended minimum performance requirements of the World Health Organization (WHO), with a sensitivity >85% compared to RT-PCR in samples with Ct values lower than 30.5

The collection time point was observed to have a strong effect on the sensitivity of the AgTs. SARS-CoV-2 was detected in all patients until day 5 after symptom onset. Subsequently, each increasing day led to a decrease in the positive rate of the AgTs by 38.6% and 63.1% for RAT and CLIA, respectively. These findings are consistent with those of Corman et al., who demonstrated AgTs to have the highest sensitivity during the first week of symptom onset.7

Our study demonstrates that the RATs analysed have a significantly higher sensitivity than the CLIA in samples with a Ct value greater than 25. Regression analysis also demonstrated a stronger inverse correlation between the Ct value and sensitivity of the CLIA compared to RAT. Hence, negative results were more common in the CLIA than in RAT in samples with increasing Ct values.

Rapid antigen tests have several advantages, including the short assay time (15 min), simplicity of use, and test performance which does not require trained personnel or laboratory space and equipment.8 The NP swab was most sensitive in the CLINITEST and ROCHE assays, whereas the NADAL test had the lowest sensitivity. However, samples evaluated using the NADAL test generally had higher Ct values, which may contribute to the lower sensitivity observed.

Despite a lower sensitivity observed for the CLIA compared to the RAT analysed, the CLIA method has some advantages. The tests are performed on a semi-automated platform, facilitating up to 126 tests per hour, results can be obtained within 42 min, and the test can be performed at a low cost.9 To exclude preanalytical bias, we compared two different collection methods. There was no significant difference in the sensitivity of the two methods compared to RT-PCR. The quantitative test results obtained from the CLIA offer the potential to monitor viral loads. However, further studies are required in which a comparison is made with PCR using larger sample sizes.

The specificity of the AgTs examined was 100% with the CLIA and 87.5% with RAT. However, the confidence intervals were too wide, with results requiring confirmation in larger studies.

Interestingly, the female sex was independent factor associated with lower sensitivity in RAT. A recent study from Nagura-Ikeda et al.10 also showed that viral RNA was detected significantly more often in male patients than in females. The authors postulated that women and men might have different salivary flow rates, potentially affecting the viral load. Another explanation might be that women have a generally lower severe and symptomatic infection rate than men with COVID-19.11 A correlation between symptom severity and the positive rate of AgTs has been recently reported.12

The present study has several limitations. Results of AgTs obtained from samples of different patients were compared. Although patients were balanced equally in terms of age, sex, and time from symptom onset, the potential for selection bias remains. Because of the prospective design of the study, it was not possible to collect samples in patients with identical Ct values. However, a significant difference in the Ct value was only noted in patients tested with the NADAL and the AMP RAT. Analysis was restricted to hospitalized patients with moderate to severe COVID-19. Further studies are required to evaluate whether results can be transferred to individuals with mild disease in an outpatient setting.

In conclusion, COVID-19 AgTs present a reliable alternative to PCR-based methods for the detection of SARS-CoV-2 in symptomatic individuals with a high viral load and a short time after symptom onset. However, all potentially infectious individuals may not be detected, with a risk of false positive results.