Diagnostic test accuracy of an automated device for the MALDI target preparation for microbial identification

The objective of this study was to evaluate the performance of the Copan Colibrí™ against the manual preparation of the MALDI targets. We analyzed 416 (31 different species) non-duplicate strains covering the most important species identified in clinical routine. We also assessed the intra-strain repeatability between the comparable methods. We then analyzed the performance of this new method after implementation in routine on 12,253 aerobic bacterial isolates and yeasts, encompassing a total of 42 different species. Among the 416 strains analyzed, 6.3% (26/416) and 10.8% (45/416) had a score value < 2 when processed by the Colibri™ and manual method, respectively. Only 5.9% (9/152) of the Gram positive rods and cocci had a score values < 2 by the Colibri™ versus 20.4% (31/152) by the manual method. We confirmed that this relative superiority observed for the Colibri™ was due primarily in the use of the formic acid protocol. For the Gram-negative bacteria, the results of both methods were comparable; 6.6% (17/256) and 4.7% (12/256) had a score value < 2 by the Colibri™ and the manual method, respectively. After implementation in routine, the results according to the Biotyper score cut-off values were distributed as follows: < 1.70: 2.5% (304/12,253), 1.70–1.79: 1.9% (227/12,253), 1.80–1.89: 3.1% (377/12,253), 1.90–1.99: 6.7% (825/12,253), and ≥ 2: 85.9% (10,520/12,253). The Colibrí™ coupled to MALDI-TOF/MS revealed good performances and higher intra-strain repeatability as compared to the manual preparation of the MALDI targets. Supplementary Information The online version contains supplementary material available at 10.1007/s10096-022-04531-3.


Introduction
Over the last decade, a large number of studies have assessed the performance of Matrix-Assisted Laser Desorption Timeof-Flight Mass Spectrometry (MALDI-TOF/MS) for the identification of bacteria, mycobacteria, yeasts, and molds in comparison with the well-established conventional diagnostic methods [1][2][3][4][5][6]. Accurate and rapid identification of microorganisms by MALDI-TOF/MS is contributing to swiftly support treatment decisions, especially when the identification of the pathogen is unexpected [7][8][9]. Hence, the real improvement of the turn-around times by MALDI-TOF/MS in conjunction with antibiotic stewardship could enable stopping unnecessary antibiotics and likely improving patient outcomes [10]. The implementation of total laboratory automation (TLA) in clinical microbiology has leveraged the advancement brought by MALDI-TOF/MS. Specifically by facilitating additional structural and organizational changes [11][12][13][14]. Copan's TLA has recently integrated an automated device (Colibrí™) that can reproducibly prepare the MALDI target for microbial identification as well as a standardized bacterial inoculum for AST. The Coli-brí™ integrates a pipetting system that permits the picking of the specific colonies defined by the technologists on the WASPLab screen (reading step), the transfer of the bacterial cells on the MALDI target, and the deposition of the matrix. These automated steps can improve the workflow and laboratory organization while simultaneously reducing the handling errors and ensuring proper documentation.
The main objective of the present study was to assess the accuracy and the diagnostic performances of the Colibrí™ coupled to MALDI-TOF/MS for the routine identification of a large number of clinical aerobic bacteria and yeasts. Figure 1 gave an overview of the present study design. We first evaluated the performance of the Colibrí™ coupled to MALDI-TOF/MS (MBT Compass 4.1, Bruker Daltonics, Bremen, Germany) against the manual preparation of MALDI target on 416 (31 different species) non-duplicate strains covering the most important species identified in clinical routine. Primarily the strains were subcultured on Columbia blood agar and chocolate blood agar using the WASPLab. The MALDI targets for the two compared methods were prepared using the same culture media plates by selecting colonies after overnight growth. Each isolate was analyzed in only one position on the MALDI target plate.

Material and methods
To assess the intra-strain repeatability between the Coli-brí™ and the manual preparation of MALDI target, 14 strains from 14 different species were analyzed in quadruplicates on the MALDI target plates using overnight growth cultures on the WASPLab. For the manual method, the MALDI targets were independently prepared for each of the 14 strains by four experienced technologists, thereby assessing the routine reproducibility. The same culture media plates (Columbia blood agar and chocolate blood agar for Haemophilus influenzae) were used for the two compared methods.
After implementation of the Colibrí™ in routine, we analyzed the performance of this new method on 12,253 aerobic bacterial isolates and yeasts encompassing a total of 42 different species. This corresponds to all routine specimens sequentially referred to the bacteriology laboratory of Geneva University Hospitals between May and October 2021. In compliance with the routine procedures for the identification by MALDI-TOF/MS and to maintain an optimal turnaround time, the preparation of the MALDI target was performed by the Colibrí™ from different culture media used in our laboratory, according to the specimen's type (e.g., chromogenic media, chocolate agar, CNA Columbia agar, MacConkey agar, and Columbia agar) without any subculture on a non-selective media. Each isolate was analyzed in only one position of the MALDI target plate.
The Colibrí™ proposes two protocols with or without an extraction step using formic acid. Hence, different protocols can be applied on different spots. The technologist defines the appropriate protocol by spot on the WASPLab during the reading step.
In this study, we used the Colibri™ protocol with formic acid for the Gram-positive bacteria and yeast, and without formic acid for the Gram-negative bacteria. In contrast, the manual method did not apply formic acid on any strain analyzed in compliance with our routine procedures. To address this bias, we performed additional experiences using the formic acid method by the two compared methods on a further set of Gram-positive bacteria and yeast. We used the Biotyper scoring and the processing protocols according to the manufacturers' instructions. Table 1 depicts the performances of the Colibri™ and manual preparation of MALDI target according to the Biotyper score values. Among the 416 (31 different species) strains analyzed, 6.3% (26/416) and 10.8% (45/416) had a score value < 2 by the Colibri™ and manual method, respectively. For the Gram-negative bacteria, the results of both methods were comparable; 6.6% (17/256) and 4.7% (12/256) had a score value < 2 by the Colibri™ and manual method, respectively ( Table 2 and Table S1). For Gram-positive rods and cocci, 5.9% (9/152) had a score values < 2 by the Colibri™ versus 20.4% (31/152) by the manual method (Table 2 and Table S1). Considering that we did not use formic acid for the manual method, we hypothesized that the differences observed in the score values for the Gram-positive rods and cocci between the Colibri™ and the manual method could be related to the formic acid protocol used with the Colibri. To test this hypothesis, we performed additional experiments using the formic acid in the two compared methods (i.e., the Colibri™ and the manual method) on an additional set of Gram-positive bacteria and yeasts (Table S2). The conclusion is that the difference observed for Gram-positive rods and cocci in Table 2 and Table S1 was related to the formic acid because no significant difference was observed when both methods used the formic acid.

Results
The eight Candida albicans strains analyzed had a score values > 2 by the Colibri™ (Table 2 and Table S1).
The analysis of the intra-strain repeatability between the Colibri™ and the manual preparation of MALDI target showed that the score values of all the strains processed by the Colibri™ were > 2. However, the inter-technologist reproducibility showed a large variation in the score values for identical isolates ( Fig. 2a and b). The quadruplicates of all strains analyzed presented a score value > 2 using the Colibri™. Among the 56 spots (4 by strain) prepared manually, 4 (7.1%) gave a score value < 2.

Discussion
The manual preparation of the MALDI target for microbial identification (ID) remains a tedious and error-prone task especially with the ever-increasing number of ID processed daily in diagnostic laboratories. In addition, the quality of the spot (i.e., the delivery of the adequate amount of microorganisms) on the MALDI target is highly operator-dependent and can affect the performance of the ID score as well as the laboratory workflow, by requiring re-test. While the use of formic acid, when deemed necessary, can improve the yield of identification, its execution further complicates the preparation of the MALDI target and complicates the laboratory workflow. The Colibrí™ enables a smooth integration of formic acid in the process without further complicating the workflow. The intra-strain repeatability and the inter-technologist reproducibility assessment revealed that the Colibri™ made good performances and higher intrastrain repeatability. A comparative analysis on the 416 clinical strains revealed that all the Enterobacteriales strains were identified with a score value > 2 using the Colibri™. That was not the case for non-fermenting Gram-negative bacilli. Among Pseudomonas monteilii, Pseudomonas putida, and Achromobacter xylosoxidans, strains analyzed 66.7 (16/24) displayed a score value < 2 using the Colibri™ and required a re-test. We did not apply the formic acid protocol on these strains. When observing a score value < 2, we routinely re-test the sample, using the same media plate, because the Colibri™ uses precisely the pickpoint previously defined by the technologist during the reading step (WASPLab). Thus, if the position of the pickpoint is Fig. 2 Box-plots show the intrastrain repeatability between the Colibrí™ and the manual preparation of MALDI targets. a Gram-negative bacteria. b Gram-positive bacteria. Each strain was analyzed in quadruplicate on the MALDI target plates using overnight growth culture. For the manual method, the MALDI targets were independently performed for each of these 14 strains by four experienced technologists, thereby evaluating the reproducibility slightly offset from the microorganism's colony, the score value can be affected. Our second step is to re-test the sample, after subculture on Columbia blood agar if the score value remains < 2.
Since the implementation of MALDI-TOF/MS in microbiology laboratories, a large number of studies reported that the approach is efficient, rapid, inexpensive, and highly accurate for the identification of the overwhelming majority of bacteria and yeasts [15][16][17]. While it allows the correct identification to the species level for a high percentage of microorganisms, the use of the conservative scoring system defined by the manufacturer decreases its diagnostic performance by rejecting some correctly identified microorganisms. As highlighted in various studies, adjusting this scoring system, relying on carefully validated studies, could significantly improve the number of correct identifications to the species level. The results of the study conducted by Scott et al. [18] suggest that it is unusual for a MALDI-TOF/MS to identify microorganisms incorrectly with a high score.
Overall, MALDI-TOF/MS has become an invaluable diagnostic tool in microbiology laboratories. The positive impact of the MALDI-TOF/MS to reduce the turn-around times (TAT) was also thoroughly evaluated. Tan et al. [19] reported that MALDI-TOF/MS enabled the reduction of TATs by an average of 1.45 days in comparison with the traditional phenotypic methods used for the identification.
The TLA has leveraged the improvements brought by MALDI-TOF/MS. In our previous study, we defined the shortest incubation times on the WASPLab for reliable MALDI-TOF/MS-based species for positive blood cultures. The percentage of bacterial strains identified with a score value > 2 were 73% (380/520) and 85% (440/520) after 4 h and 6 h of incubation of the culture media plates on the WASPLab, respectively [20]. The percentage of accurate identification (score value > 2) reached 100% (460/460) after 8 h of incubation when excluding Corynebacterium and Candida spp. from the analysis. This study established that the WASPLab coupled to MALDI-TOF/MS reduces significantly the TAT for positive blood cultures. Therefore, the implementation of the Colibri™ has substantially improved the workflow by reducing hands-on time required for manual preparation of the MALDI targets.

Conclusions
The manual preparation of the MALDI target brings some degree of errors through, for example, the staggered positioning of the spots on the target. Moreover, the inter-technologist reproducibility revealed large variations in score values for a series of identical isolates. The Colibri™ contributes to a more reliable quality of the spot, improved traceability, and smooth integration of formic acid in the process, wherever needed. The Colibrí™ coupled to MALDI-TOF/MS revealed good performances and higher intra-strain repeatability as compared to the manual preparation of the MALDI targets.