The diagnostic test under evaluation in this study was the microfluidic immunofluorescence assay, SARS-CoV-2 Ag test developed by LumiraDx™ Limited, London, United Kingdom (henceforth called LumiraDx™). The assay runs on a portable platform using a dry, single-use, disposable, microfluidic test strip. The strip contains antibodies specific to the nucleocapsid protein of SARS-CoV-2 that form a sandwich-like immuno-complex and emit a fluorescent signal if the nucleocapsid protein is detected .
The sample collection was performed with Dryswab™ Standard Tip Rayon (Medical Wire & Equipment, Corsham, England), not included in the test kits, but recommend for use by the manufacturer. Ag-RDT testing was performed on-site in a dedicated workspace that was divided into separate areas where infectious and non-infectious materials were handled. Laboratory personal for Ag-RDT testing was blinded to the results of RT-PCR testing and vice versa. Following manufacturer instructions for use, the collected sample was processed using the LumiraDx™ proprietary extraction buffer vial for ten seconds. A single-use test strip was placed in the designated slot on the testing device, and one drop (~ 20 μL) of the prepared extraction buffer solution was applied. The designated device slot was closed to initiate automatic processing and testing. The device showed a qualitative result (“positive”, “negative” or “error”) on the digital touch screen within 12 minutes. In case of an erroneous test result, the testing was immediately repeated using the solution from the same extraction vial and a new test strip.
For every test, a cut-off-index (COI) value was automatically generated and documented in the testing device, reporting the test immunofluorescent signal on a continuous scale with a pre-set cut-off categorizing the results as positive or negative. COI values were retrieved from the test devices by the manufacturer after completion of the study. The device can operate in a cloud-based mode enabling the manufacturer to receive all impersonal data through an online connection. During this study, the manufacturer could not access the device as the study was performed offline.
After every test, the surface of the device was disinfected with proprietary disinfection wipes provided by the company. A 5-minute drying time was recommended before the next sample was inserted, following the manufacturer information for use .
Reverse transcriptase-polymerase chain reaction (RT-PCR) was used as the reference test. The RT-PCR samples were collected by health-care workers using the IMPROSWAB® (Guangzhou Improve Medical Instruments Co., Ltd., Guangzhou, China) at the Heidelberg study site and the eSwab™ (Copan Diagnostics Inc., Murrieta, CA, USA) at the study site in Berlin. Collected samples were transferred to the referral laboratories for RT-PCR testing in Heidelberg and Berlin. The RT-PCR assays, referral laboratory standard assays, used for clinical diagnosis as comparators were Allplex™ SARS-CoV-2 assay (Seegene, Seoul, South Korea) in Heidelberg and cobas™ SARS CoV-2 assay on the cobas® 6800 or 8800 system (Roche, Pleasanton, CA, United States) or the SARS CoV-2 assay from TIB Molbiol (Berlin, Germany) in Berlin. Interpretation of the RT-PCR assays followed the manufacturers’ instructions. The RT-PCR assays targeted the E gene of SARS-CoV-2, which was used for the CT-value determination and the viral load calculations. Based on testing of standardized material, CT-values of the three tests are expected to differ by around 2–3 with the same amount of virus present . The conversion of CT-values into viral-load was based on calibrated RT-PCR testing with quantified SARS-CoV-2 in vitro transcripts .
Clinical diagnostic accuracy
The standards for reporting diagnostic accuracy studies (STARD) were followed for reporting of this study .
Study design and inclusion criteria
This prospective multi-centre study enrolled participants at two sites, Heidelberg and Berlin, Germany. In Heidelberg, participants were enrolled at a drive-in testing site whereas in Berlin participants were enrolled at a clinical ambulatory testing facility. All participants were identified for testing according to the criteria of the national health authority as being at risk of SARS-CoV-2 infection based on reported symptoms or recent contact with a confirmed case . Inclusion criteria were a minimum age of 18 years and no prior positive RT-PCR result for SARS-CoV-2. Written informed consent was obtained from each participant prior to testing. Individuals unable to give written informed consent due to limited command of German or English were excluded from the study. The study protocol is available upon request.
The sample for the Ag-RDT was a NMT, following the definition from the Centers for Disease Control and Prevention . This swab was collected by the participants themselves, with a health-care worker providing instructions, supervision and corrections according to the instruction for use by the manufacturer. The participants were instructed to tilt their head back 70°, insert the swab approximately 2 cm into one nostril, and to rotate it several times against the full interior nasal wall surface for at least 10 s. The identical procedure was repeated with the same swab in the contralateral nostril.
Subsequently, the professional-collected routine swab for RT-PCR testing was taken, following institutional procedures with a nasopharyngeal (NP) swab (Heidelberg) or a combined NP/oropharyngeal (OP) swab (Berlin). Participants underwent OP swabbing only if there were clinical contraindications for NP sampling.
All participants were asked for information on comorbidities, symptom presence and duration, and severity of disease (questionnaire available in the supplement material, Section (B)). For data collection, we used the Research Electronic Data Capture (REDCap) tools hosted at Heidelberg University .
RT-PCR from Ag-RDT extraction buffer
All false-positive and false-negative Ag-RDT results were retested with RT-PCR from the Ag-RDT proprietary buffer solution that was stored at a temperature of − 20 °C, if sufficient volume was available. This testing is not validated by the manufacturer but was performed to resolve discrepant results that could have occurred due to variability between the NP and NMT sampling for RT-PCR and Ag-RDT. To avoid the introduction of a discrepant analysis bias, a subset of samples (first 5 per day in the first 2 weeks) was also re-tested with RT-PCR from the antigen buffer.
Clinical samples from common respiratory tract pathogens or commensals were assessed for cross-reactivity. The respiratory swab samples contained four seasonal coronaviruses, adenovirus, bocavirus, influenza virus, metapneumovirus, parainfluenza virus, respiratory syncytial virus, rhinovirus or Mycoplasma pneumoniae as identified by RT-PCR in the laboratory. For Staphylococcus aureus and Streptococcus sp. swab samples from bacterial culture were utilized. As recommended by the manufacturer these samples were added to the extraction buffer in a 1:10 dilution (30 µL sample and 270 µL extraction buffer) and processed according to the standard test protocol.
Limit-of-detection assay using cell culture-grown SARS-CoV-2
The limit-of detection of the LumiraDx™ was assessed in comparison with Abbott PanBio (Chicago, Illinois, USA) and SD Biosensor Standard Q (Gyeonggi-do, Korea), Two strains of SARS-CoV-2 were used. The BavPat1/2020 strain was kindly provided by Christian Drosten through the European Virus Archive and the HD strain was isolated from a patient at the Heidelberg clinic. Working virus stocks were generated by passaging the virus two times in VeroE6 cells or Calu3 cells (a kind gift from Dr. Manfred Frey, Mannheim). Virus stocks were tittered using a plaque forming unit (PFU) assay . The stock was pre-diluted in DMEM supplemented with 2% FCS to 10,000 PFU/mL. Then twofold dilutions were generated and added 1:1 to the isolation buffer provided for each test kit. Following this, the manufacturer’s protocol for each kit was used. Note, that the amount of volume used for each kit was slightly different according to the protocols (50 µL for Abbott PanBio, 100 µL for Biosensor Standard Q and 35 µL for LumiraDx™). The limit of detection was calculated based on the number of PFUs in this volume. Three replicates were done for each dilution.
System usability scale and ease-of-use assessment
To determine and quantify the usability of the test, a standardized System Usability Scale (SUS) questionnaire was used and a dedicated ease-of-use assessment (EoU) was developed. The detailed surveys are provided in the supplement material (Section (C) and (D)). Staff performing the testing at both study sites were invited to complete the questionnaires. A SUS score above 68 is interpreted as above average . For the visualization of the EoU assessment a colour-coded rating (heat-map) was generated. Each aspect of the assessment was ranked as satisfactory, average, or unsatisfactory. The supplement material (Section (E)) shows the matrix used for this analysis.
To determine sensitivity and specificity of the Ag-RDT (with 95% CIs), results were compared to RT-PCR results from the same participant, as per Altman . Predefined sub-analyses were conducted for presence of symptoms, duration of symptoms (≤ 7 days, > 7 days and > 12 days), CT-values (using two categorizations: ≤ 25, > 25 and ≤ 30, > 30), viral load (log10 SARS-CoV2 RNA copies/ml), and study site. A two-sided alpha value of 0.05 was defined as a significance cut-off. We used “R” version 4.0.3. (R Foundation for Statistical Computing, Vienna, Austria) to generate all analyses and plots.