Analytical performance of the point-of-care BIOSYNEX COVID-19 Ag BSS for the detection of SARS‐CoV‐2 nucleocapsid protein in nasopharyngeal swabs: a prospective field evaluation during the COVID-19 third wave in France

Background The accuracy and reliability of rapid diagnostic tests are critical for monitoring and diagnosing SARS-CoV-2 infection in the general population. This study aimed to evaluate the analytical performance of the BIOSYNEX COVID-19 Ag BSS (Biosynex Swiss SA, Fribourg, Switzerland) antigen rapid diagnostic test (BIOSYNEX Ag-RDT), which targets the SARS-CoV-2 N-nucleocapsid protein for the diagnosis of COVID-19. The Ag-RDT was compared with a real-time RT-PCR (rtRT-PCR) as gold standard for performance measurement. Methods Two nasopharyngeal flocked swabs were prospectively collected simultaneously in March and April 2021 from 967 individuals aged ≥ 18 years tested for SARS-CoV-2 in two private laboratories, Paris, France. Results Overall, the Ag-RDT demonstrated high sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 81.8%, 99.6%, 96.6%, and 97.5%, respectively. The agreement (97.0%), reliability assessed using Cohen’s κ-coefficient (0.87), and accuracy evaluated using Youden index (J) (81.6%) in detecting SARS-CoV-2 were high. The analytical performance of the Ag-RDT remained high when there was significant viral shedding (i.e., N gene Ct values ≤ 33 on reference RT-PCR). The sensitivity was only 55.2% in case of low or very low viral excretion (Ct > 33). Conclusions The BIOSYNEX Ag-RDT is a promising, potentially simple diagnostic tool, especially in symptomatic COVID-19 patients with substantial viral excretion in the nasopharynx. Supplementary Information The online version contains supplementary material available at 10.1007/s15010-021-01723-5.


Introduction
The 2019 coronavirus pandemic  continues to spread worldwide. The effective isolation and early treatment of patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) require rapid, accurate, and straightforward diagnostic tools.
While currently recommended nucleic acid amplification tests (NAAT), such as real-time reverse transcriptionpolymerase chain reaction (rtRT-PCR) assays, remain the gold standard cornerstone for the diagnosis of SARS-CoV-2 infection [1,2], viral antigens can be detected using immunological methods [2][3][4]. Indeed, conducting rtRT-PCR is expensive, time-consuming, and requires special equipment and qualified operators. Point-of-care antigen-detecting rapid diagnostic tests (Ag-RDT) constitute simple and less expensive alternative tests [3]. Ag-RDT relies on direct detection of SARS-CoV-2 viral proteins in nasal swabs and other respiratory secretions. The N-nucleocapsid protein is frequently targeted because of its relative abundance and conserved structure, or other viral proteins such as the spike protein [4]. Most Ag-RDTs rely on sandwich catching using anti-SARS-CoV-2 monoclonal antibodies to detect viral antigens in the simpleto-use lateral flow immunoassay format allowing results in < 30 min. However, significant variability has been reported about their diagnostic performance and a lack of external validation for many available tests, which still require clinical validation [5][6][7][8][9].
Our study aimed to evaluate the qualitative membranebased immunochromatographic BIOSYNEX COVID- 19 Ag BSS Ag-RDT (Biosynex Swiss SA, Freiburg, Switzerland; reference SW40006; abbreviated by BIOSYNEX Ag-RDT) using monoclonal antibodies detecting SARS-CoV-2 N-nucleocapsid protein to diagnose COVID-19 from prospectively collected nasopharyngeal secretion samples in adults living in the Paris region throughout the third wave of the COVID-19 epidemic in France.

Rapid antigen test
The BIOSYNEX Ag-RDT consists of a reaction membrane and three buffers (sample, reagent, and absorbent). The reagent buffer contains colloidal gold particles conjugated with monoclonal antibodies directed against the N protein of SARS-CoV-2. Secondary antibodies against the N protein are fixed on the reaction membrane. The manufacturer's instructions were followed to conduct the test by mixing nasopharyngeal secretions with 300 µl of dilution buffer in a tube. After 1 min, four drops were added to the well on the cassette.
If SARS-CoV-2 antigens are present in the sample, the complexes between the anti-SARS-CoV-2 conjugate and the virus are captured by anti-SARS-CoV-2 monoclonal antibodies specific to the test line area (T). The lack of the T line indicates that the result is negative. A red line appears in the control line area (C) to serve as a procedural control, indicating that the correct sample volume has been added and the membrane has played its role. Reading is carried out after 15 min.

Study population and procedures
During the third wave of the COVID-19 epidemic (March and April 2021), two sites had been used to consecutively collect paired nasopharyngeal swabs. Site A was the Centre Cardiologique du Nord, Saint-Denis, France. Site B was the Laboratoire Paris XV, Paris, France. Participants aged ≥ 18 years and those consenting to undergo two nasopharyngeal swabs for rtRT-PCR and Ag-RDT were included. All participants were given a questionnaire that recorded demographic information (sex and age), reasons for testing, and current and past 14-day symptoms in symptomatic patients. Suggestive symptoms of COVID-19 were headache, fatigue, fever, or upper or lower respiratory symptoms. Asymptomatic individuals were defined as those not reporting any of these symptoms. At both sites, a health care professional first collected nasopharyngeal secretions in one nostril, using the swab provided in the BIOSYNEX Ag-RDT. A second nasopharyngeal swab in the other nostril served as specimen for the rtRT-PCR. The COVID-19 antigen rapid test was performed immediately on-site using the Ag-RDT following the manufacturer's instructions. The other nasopharyngeal swabs were stored in physiological saline (NaCl 0.9%) (1000 μL) at + 4 °C and analyzed within 24-48 h by the reference rtRT-PCR.

Molecular detection of SARS-CoV-2
The multiplex real-time PCR Novel Coronavirus (2019-nCoV) Real-Time Multiplex RT-PCR Kit (Detection for 3 Genes) (Liferiver & Shanghai ZJ Bio-Tech Co., Ltd, Shanghai, China) was the reference multiplex molecular detection of SARS-CoV-2 RNA. Individual cycle threshold (C t ) values for each target gene (E, N and RdRP). According to manufacturer's recommendations, samples with C t values ≤ 41 for three or two gene targets were considered as positive; those with C t values ≤ 41 for only one gene target were possibly positive; samples with C t value > 41 for the three gene targets were negative.
The C t values of the N gene in the RT-PCR reference were chosen for stratification of viral load in clinical samples because the Ag-RDT detects the SARS-CoV-2 N-nucleocapsid protein.

Statistical analyses
Collected data were analyzed using IBM® SPSS® Statistics 20 software (IBM, SPSS Inc, Armonk, New York, USA). Results of quantitative variables were expressed as medians; however, the proportion with their 95% confidence interval (CI) assessed according to the Wilson score bounds were estimated for categorical variables [10]. Comparisons were carried out using Pearson's Chi square test or Fisher's exact test based on validity conditions. The PPV and NPV were calculated according to Bayes' formulas, taking into account the officially reported prevalence of SARS-CoV-2 RNA positivity in symptomatic patients in the Paris region on 12th April 2021, e.g., around the peak of the third wave epidemic in France (Santé publique France 2021; https:// www. sante publi quefr ance. fr/).

Ethics statement
The purpose of the study was to clinically evaluate the continuous quality improvement program and performance evaluation of COVID-19 management measures following the National Medical-Biological Laboratory Accreditation [11]. The data set was anonymous and contained no identifiable personal health information.
Among the 148 positive samples using the gold standard rtRT-PCR, 146 were positive for the three gene targets, and two were positive for only E and N genes. The mean ± SD of the C t values were 26.1 ± 4.4 arbitrary units (a.u.) for the E gene, 26.5 ± 5.0 a.u. for the RdRP gene, and 26.9 ± 5.1 a.u. for the N gene.
The vast majority (114/123, 92.7%) of positive results were visible in the window of the cassette of the Ag-RDT within the first 5 min. Table 2 shows the test results and primary performance characteristics of the BIOSYNEX Ag-RDT compared with the reference rtRT-PCR in the study population according to COVID-19-compatible symptoms. Using rtRT-PCR as the standard, three false-positive BIO-SYNEX Ag-RDT results occurred among specimens from asymptomatic individuals (n = 2) or symptomatic patients (n = 1). Of the 148 rtRT-PCR positive results, 27 (18.2%) were false-negative BIOSYNEX Ag-RDT (23 specimens from asymptomatic persons and 4 specimens from symptomatic patients). Overall, the BIOSYNEX Ag-RDT showed high sensitivity (81.8%), specificity (99.6%), PPV (96.6%), and NPV (97.5%). Among symptomatic patients, sensitivity was 95.0%, specificity was 99.4%, PPV was 95.6%, and NPV was 96.3% (Table 2). Within 7 days from symptom beginning, the BIOSYNEX Ag-RDT showed a sensitivity of 96.6%, a specificity of 99.4%, whereas the PPV and NPV were 95.7% and 99.4%, respectively.
The Table 3 shows the analytical results based on the level of viral excretion assessed by the N gene C t values using the reference rtRT-PCR. Overall, the BIOSYNEX Ag-RDT showed high agreement (97.0%), reliability using Cohen's κ coefficient (0.87), and accuracy using Youden's J index (81.6%) to detect SARS-CoV-2.
In case of high or very high viral loads (C t ≤ 33), the BIOSYNEX Ag-RDT had a good analytical performance (sensitivities between 83.3% and 100.0%, specificities of 99.8%, PPV between 98.3% and 98.6%, and NPV between 97.7% and 100.0%). In case of low or very low viral loads (C t > 33), the sensitivity of the BIOSYNEX Ag-RDT had reduced analytical performance (sensitivity of only 55.2%), while its specificity remained high (98.8%). Similar observations were made when the C t values of the E or ORF1ab gene targets were chosen for stratification of viral load in clinical samples (data not shown).

Discussion
We evaluated the analytical performance of the novel point-of-care BIOSYNEX Ag-RDT compared to multiplex rtRT-PCR as gold standard for detecting SARS-CoV-2 RNA in a real-life setting. In this study, the sensitivity of the BIOSYNEX Ag-RDT was lower among specimens from asymptomatic persons (79.4%) than among specimens from symptomatic patients (95.0%). It was high in patients with suspected COVID-19 (96.8%). Specificity (> 99.0%) was high in specimens from both asymptomatic individuals and symptomatic patients. The prevalence of SARS-CoV-2 RNA-positive rt-RT-PCR results in this population was 15.3% overall, 9.4% for asymptomatic individuals, and 32.6% for symptomatic patients. The estimated PPVs and NPVs of the BIOSYNEX Ag-RDT were elevated in all groups of participants. However, administering the Ag-RDT in low prevalence settings will likely result in lower predictive values. In the event of significant viral excretion (i.e., N gene C t values below 33 based on reference rtRT-PCR), the BIOSYNEX Ag-RDT showed high sensitivity (from 83.3% to 100.0%) and specificity (> 99.0%) for SARS-CoV-2 RNA detection. Concordance, reliability, as well as accuracy were great with the reference assay and PPVs and NPVs above 97.0%. However, the sensitivity of the study Ag-RDT dropped to 55.2% with low or very low viral shedding (C t > 33). Together, these observations demonstrated the high analytical performance of the BIOSYNEX Ag-RDT. This performance made it suitable for use as point-of-care Ag-RDT in various hospital and non-hospital settings where a rapid diagnosis of SARS-CoV-2 is necessary. Although less sensitive than RT-PCR, the BIOSYNEX Ag-RDT could be beneficial by obtaining quick results, ease of use, and independence from existing laboratory structures. Testing criteria focusing on patients during the early onset of symptoms could further increase its diagnostic value. The sensitivity of the BIOSYNEX Ag-RDT was 81.8% overall, and the positive detection rate was comparable to the rtRT-PCR in the majority (88.2%) of subjects with C t ≤ 33. False-negative test results of 12/14 (85.7%) subjects with significant viral excretion (C t ≤ 33) were asymptomatic, although conflicting evidence exists about the relationship between symptom severity and viral shedding [12]. False-positive test results were rarely observed, providing 99.6%-specificity, exceeding the performance recommended by the World Health Organization (WHO) [13]. False-positive results have been reported as well in other antigen tests [14][15][16]. False positivity could be associated with high viscosity of tested specimen samples as well as interference with mucosal antibodies [17].
We analyzed our results based on the estimated viral load in SARS-CoV-2 in the samples. There is an ongoing debate about the C t value corresponding to the threshold of infectivity (i.e., patient considered as contagious) [7,36,37]. La Scola et al. found that patients with C t values > 33 are not infectious because of the low number of positive cultures [38]. The Centers for Disease Control and Prevention (CDC), Atlanta, USA, propose a C t cut-off value of 33 as a marker for contagiousness [39], and stress that C t values ≤ 20 correspond to very high viral excretion [7,36,40,41]. Our results confirm that the analytical performances of the BIOSYNEX Ag-RDT were much better in specimens with a high viral load. These observations demonstrate the capability of the BIOSYNEX Ag-RDT as a rapid rule-in test for COVID-19 with samples at high viral load in symptomatic patients, for example, and raise caution about its use as a singular rule-out test, especially in samples with lower viral loads.
Our study has several strengths. All samples were collected from one nasopharynx with flocked swabs, optimal for evaluating Ag-RDT clinical performances in our study. The Ag-RDT and reference rtRT-PCR were carried out in parallel. The study population included various situations outside the hospital setting, with mostly young adults without comorbidities who had typical and mild COVID-19 symptoms when being symptomatic.
The study presents also some limitations. Participants may have inadvertently reported general, non-specific symptoms as COVID-19 compatible symptoms. This investigation evaluated the BIOSYNEX Ag-RDT; the results presented here cannot be generalized to other agencies-authorized SARS-CoV-2 antigen tests. Otherwise, the CDC clarified that C t values using the rtRT-PCR platform is not a quantitative measure of viral burden in clinical samples and cannot be used to assess whether a person is infectious [42]. Consequently, our stratification of samples according to C t values of the N gene does not necessarily reflect the actual infectivity of the participants. Finally, higher rate of asymptomatic persons in the study (with lower virus level) could have resulted in decreased sensitivity of the BIOSYNEX Ag-RDT.

Conclusion
The BIOSYNEX Ag-RDT demonstrated high specificity and sufficient sensitivity for detecting SARS-CoV-2. Given the simple procedures and short turnaround time for this test, it is a promising option as an alternative diagnostic modality, especially in symptomatic COVID-19 patients. The test may also be used to test asymptomatic individuals suspicious of exposure to SARS-CoV-2 and as part of a population-level mass screening.