External quality assessment for molecular typing of Salmonella 2013–2015: performance of the European national public health reference laboratories

We report the results of three consecutive External Quality Assessments (EQAs) for molecular subtyping of Salmonella to assess the performance of the European national public health reference laboratories (NPHRLs). The EQA included the molecular typing methods used for European enhanced surveillance of human Salmonella infections: pulsed field gel electrophoresis (PFGE), including gel analysis by the use of the software BioNumerics, and 5-locus multiple locus variable number of tandem repeat analysis (MLVA) for serovar Typhimurium. The participation in the PFGE laboratory part was higher (27/35) than in the gel analysis (19/35) and MLVA (15/35), suggestive of the need for capacity building in methods requiring specialized equipment (MLVA) or software (gel analysis). The majority (25/27) of the participating NPHRLs produced inter-laboratory comparable PFGE gel(s). Two laboratories continued to produce low-quality gels and should have additional technical assistance in the future. In particular, two gel quality evaluation parameters, measuring “image acquisition and running conditions” and “bands”, were identified to cause gel quality problems throughout the EQAs. Despite the high number of laboratories participating in the PFGE laboratory part, the participation in gel analysis was low, although increasing. In the MLVA part, the NPHRLs correctly assigned 96% (405/420) allelic profiles according to the nomenclature. In conclusion, the EQAs identified critical parameters for unsuccessful performance and helped to offer assistance to those laboratories that needed it most. The assessments supported the development of quality in molecular typing and promoted the harmonization of subtyping methods used for EU/EEA-wide surveillance of human Salmonella infections.


Introduction
Salmonellosis is one of the most frequent zoonotic diseases worldwide. In Europe, Salmonella enterica ssp. enterica continues to be the second most reported zoonosis and a common cause of foodborne outbreaks responsible for about 20% of all outbreaks in the EU in 2014 [1]. Symptoms range from selflimiting diarrhoea to life-threatening sepsis. Molecular typing of Salmonella strains aids in improving the identification, investigation, and control of an outbreak source so that prevention of further spread is effective. In the EU, the European Centre for Disease Prevention and Control's (ECDC's) disease programme for Food-and Waterborne Diseases and Zoonoses (FWD DP) is responsible for the surveillance of salmonellosis. In order to improve the surveillance of foodborne infections, molecular typing data have been reported to the European Surveillance System (TESSy) by the Member States since 2012 [2]. For Salmonella, this included pulsed field gel electrophoresis (PFGE) data for all serovars and multiple locus variable number of tandem repeat analysis (MLVA) data for Salmonella Typhimurium. The molecular typing for surveillance is based on the capacity of the laboratories in the European FWD network (FWD-Net) to produce typing data of good quality to allow for reliable cross-border surveillance and outbreak detection. The current molecular typing methods used for EU-wide surveillance and outbreak investigation of Salmonella, are PFGE suitable for typing all serovars [3,4], and MLVA for S. Typhimurium [5,6] and S. Enteritidis [7], two of the most prevalent serovars in the EU/ EEA [1]. MLVA provides a higher discriminatory power compared to PFGE within S. Typhimurium, and has been used for surveillance and outbreak investigations in the past decade [8][9][10]. Likewise, there was a need for a more discriminatory method than PFGE for detecting outbreaks of serovar Enteritidis. Therefore, a multi-laboratory validation study of a 5-locus MLVA scheme for S. Enteritidis was recently carried out [7] and since June 2016 MLVA data for S. Enteritidis have also been included in TESSy. Thus, MLVA for S. Enteritidis will be included in future EQAs. Protocols for PFGE and both MLVA methods have been standardized to produce comparable results across laboratories [7,[11][12][13][14][15].
Since 2012, the National Public Health Reference Laboratories (NPHRLs) in the FWD-Net have been given the opportunity to participate in External Quality Assessments (EQAs) for the molecular typing methods used in the European surveillance of foodborne pathogens. The evaluation of the PFGE results of the first round of EQAs (2012-EQA) for the three pathogens Salmonella, Listeria, and verotoxin-producing E. coli is reported by Schjørring et al. 2016 [16]. In the present study, the results of the following three EQA rounds related to molecular typing of Salmonella strains by PFGE for all serovars and MLVA for S. Typhimurium will be presented. The aims of each EQA round were to assess the quality of PFGE typing and the comparability of the test results among the NPHRLs in the FWD-Net, as well as to determine and ensure the quality and integrity of the S. Typhimurium MLVA results among the participating laboratories. Here, we combined the results of the three rounds which took place in the years 2013-15 and assessed the development in the capability of the NPHRLs in producing molecular typing data that can be used for supporting the surveillance of Salmonella infections in Europe. We will focus on development in the capacity to perform the standard methods, the quality and comparability of the typing data, and the identification of common quality issues.

Participants
The EQAs were open to one NPHRL in each of the 30 FWD-Net countries in the EU/EEA and five EU candidate countries. The laboratories could choose to perform all or selected method(s) of the EQA. Over the 3-year period, the number of participating countries increased from 24 to 27. Twenty-one (60%) NPHRL participated in all three EQA rounds. These studies were coordinated by the Unit for Foodborne  [18][19][20].

Test strains
Clinical isolates from the SSI strain collection were selected for inclusion. For each EQA round, candidate strains were selected to represent serovars relevant for the epidemiological situation in Europe, including recent outbreak strains ( Table 1). For MLVA, S. Typhimurium strains were selected to cover common allelic profiles, and include three strains (ac) repeated in all EQAs (Table 2). Stability testing and preparation of strains for distribution were performed according to the International Standard ISO/IEC 17043:2010 [17]. The final selection of ten test strains for each PFGE and MLVA testing was made from the candidate strains that remained stable during ten consecutive passages. Repeat strain c had generated a stable one-repeat change in the STTR10-locus during storage at SSI (changed from 23 to 24 repeats, Table 2). Replicates of each isolate were prepared on agar stabs (SSI Diagnostica, Hillerød, Denmark), blind coded and dispatched by courier. All packages were delivered within 2 weeks (mean 2 days). No damage to the shipments was reported. The coordinating centre provided, both within the packages and individually by email, standard protocols and individual letters stating the unique blind-coded strain IDs, and detailed instructions on setting up the database, Image Acquisition recommendations, obtaining correct MLVA results, importing MLVA into BioNumerics (Applied Math, Sint-Marten-Laten, Belgium) and exporting data from BioNumerics. In addition, pre-configured BioNumerics databases (for all versions of the software), Excel templates for allele-calling and calibration of raw fragment sizes to verified sizes of the reference strains [21] were made available for the participants.

Typing analysis
Participants were instructed to use the Standard PulseNet PFGE protocol for Salmonella [15], and analyse their PFGE gel using the pre-configured BioNumerics database generating comparable profiles by normalization to the size standard and band assignment. For allelic profiling, the 5-locus S. Typhimurium MLVA standard protocol was suggested including the agreed nomenclature [12,14]. Normalization of measured fragment size to actual size could be performed according to the verified fragment size of the 33 reference strains using the distributed calibration file [21]. The allele-calling Excel file returned the true allele number when a submitted fragment size matched to a predesignated allele number. This nomenclature provides a string of five digits representing the true number of repeat units in each of the five loci. An absent allele is reported as NA [12].
Reporting PFGE gel images were submitted to the coordinating centre in a tagged image file format (TIFF file) and by returning an online questionnaire indicating the position on the gel of each coded test strain. The participants who also performed the subsequent gel analysis in BioNumerics submitted an export file including all results and additional information. MLVA results were submitted as allelic profiles, either submitted via the online questionnaire for each coded test strain, or submitted as part of the export file from BioNumerics. The coordinators imported the results to a dedicated BioNumerics database, and reported to the participants if errors in the submission process were identified.

Scoring of participant results
Analysis of submitted results was undertaken by the coordinating centre, and individual evaluation and feedback reports were provided to each participant summarizing, scoring, and commenting on the results. Areas for improvement were identified and optimization suggested. In addition, certificates of participation were provided to the laboratories. Participants were given an anonymised and confidential laboratory number.

PFGE gel quality
The quality of the raw PFGE gels (laboratory part) was graded according to the modified ECDC FWD MolSurv Pilot -SOPs 1.0, PulseNet US protocol PFGE Image Quality Assessment (TIFF Quality Grading Guidelines) [16,[18][19][20]. The Repeat strain (a-c) included in all EQA rounds. Repeat strain c had gained a repeat in the STTR10 locus before distribution in the 2015-EQA NA not applicable (locus not present) [12] *Single locus variants (SLV) of the repeat strain c in STTR10 evaluation examines seven gel parameters: (1) image acquisition and running conditions (TIFF file and band spacing of global standard), (2) cell suspension (evenness of DNA concentrations), (3) bands (appearance of bands, distint/distorted/ fuzzy), (4) lanes (straightness), (5) restriction (completeness), (6) gel background (debris or clear), and (7) DNA degradation (smearing), using four score values 1-4. An acceptable gel quality (score of 2 or better) should be achieved in each parameter. A non-accepted low quality score of 1 in just one parameter impacts the ability to further analyse the image and produce comparable profiles across laboratories. Results are presented as accepted (1)/non-accepted (0), together with the total percentage score of the maximal of 28 points.

PFGE gel analysis
Participants with access to the specialized software BioNumerics were asked to analyse the PFGE gel produced in their own laboratory. The performance of this gel analysis was graded according to the BioNumerics Gel Quality Grading Guidelines, developed by the coordinating centre [16,[18][19][20].
The grading examined five parameters: (1) position of gel, (2) strips, (3) curves, (4) normalization, and (5) band assignment. Each parameter with scores ranging from 1 to 3. As for the gel scoring, acceptable scores of 2 or better should be achieved in each parameter. The parameter Bband assignment^was graded according to the quality of the gel, i.e., a laboratory producing a non-accepted gel could still achieve an accepted score in the following gel analysis. Results are presented as the total percentage score of the maximal of 15 points.

MLVA
The allelic profile of each test strain was scored as correct (B1^) or incorrect (B0^), and the percentage of correctly assigned allelic profiles generated a total score from zero to 100% correct profiles. One-repeat single locus variants (SLVs) of the highly variable loci, STTR5, STTR6, and STTR10 were accepted when evaluating the results. Changes in the rapidly changing loci can be unavoidable during passage and transport.

Participation
The participation rate was highest among the FWD-Net countries, with 26 of the 30 countries (87%) participating in at least one of the three EQA rounds. The participation rate was 60% (3/5) among the EU candidate countries. The number of participants increased from 24 (69%) in the 2013-EQA to 27 (77%) in the 2015-EQA (Table 3). In each EQA, PFGE received more participations (n = 22-25) than MLVA typing (n = 14, Table 3). Due to major technical problems, the PFGE gel from one of two new participants (2015-EQA) was unanalysable and excluded from the results presented. Additional laboratories signed up for the EQAs, but refrained to submit the results: two in the PFGE and three in the MLVA part in the 2013-EQA, and two in the PFGE part of the 2014-EQA. Only participants submitting analysable results are presented.

PFGE gel quality
Most of the participating NPHRLs (27/29) performed PFGE typing in at least one EQA during the 3-year period (Table 4). Applying acceptable versus non-acceptable performance as means of performance, more than half of the participants (15/27) displayed a high performance level, i.e., generating acceptable gels on each submission (Table 4). Four (4/27) laboratories (no. 130, 144, 129, 138) showed a deteriorating level of performance. These laboratories provided acceptable gel(s) in one or two of the early distributions; however, they produced non-acceptable gels in later EQAs (Table 4). Conversely, four participants (no. 114, 140, 148, 145) improved their performance by demonstrating their ability to generate acceptable gels after initially producing nonacceptable gels in the earlier EQA rounds (Table 4). Two laboratories (no. 132 and 160) were unable to produce an acceptable gel; laboratory no. 132 in all EQAs, and no. 160 in EQA-2014 and EQA-2015 (Table 4). In the two rounds (2014-EQA and 2015-EQA), laboratory no. 160 did not use the recommended PFGE protocol for Salmonella, and subsequently generated very fuzzy bands. Laboratory no. 132 had various problems, the main ones being overexposure of the gel image and not using the correct PFGE protocol/running conditions. A common feature in the two laboratories was the Gel and analysis Submission of PFGE gel image including profile analysis in BioNumerics *One laboratory submitted an un-analysable PFGE gel due to major technical problems, and was excluded from analyses, thus n = 7 evaluated results inability to produce a gel that could be normalized or show distinguishable bands, thereby obtaining the lowest score (non-acceptable) in the two parameters Bimage acquisition and running conditions^and Bbands^. In general, nonacceptable scores were given to at least two laboratories for these two parameters in each of the three EQA rounds, while no non-acceptable scores were given in three parameters (cell suspension, lanes and gel background). In the remaning two parameters (restriction and DNA degradation), nonacceptable scores were only given in 2014 (Table 5). Among the 71 gels evaluated, only five (7%) gels (three laboratories) obtained a 100% total score (Table 4), reflecting the difficulty in producing an exemplary gel with respect to all evaluation parameters.

PFGE gel analysis
Nineteen (19/27, 70%) of the NPHRLs performing PFGE analysed their PFGE profiles by the use of BioNumerics and demonstrated a high-quality performance level (Table 6). Fifteen (79%) laboratories performed acceptable gel analysis in every distribution they took part in. Two participants (no. 128 and 129) improved their performance by producing acceptable gel analysis after producing non-acceptable analysis in earlier EQA rounds. In the 2015-EQA, all participants produced an acceptable gel analysis. Among the 40 gel analyses evaluated in total, 12 (30%) gel analyses obtained a 100% score (Table 6).
In two of the five gel analysis parameters, only acceptable scores were generated; all participants defined lanes and curves correctly in accordance to the guidelines in all three EQA rounds (Table 7). A few (n = 6) non-acceptable scores were obtained by four laboratories (no. 128, 129, 130, and 148) for the three other parameters. Laboratories no. 128 and 130 placed the gel frame incorrectly by including the wells in the first step of the gel analysis procedure, leading to the possibility of incorrect auto-band search and band assignment. Laboratories no. 128, 129, and 148 performed incomplete band assignments by failing to assigning all bands to the profiles of the test strains (non-acceptable score in the parameter 1 acceptable gel, 0 non-acceptable gel,no participation a Total percentage score of the maximum score of 28 points Scores given according to the TIFF Quality Grading Guidelines [18][19][20]. Score of 1: non-acceptable, scores of 2, 3 or 4: acceptable band assignment) and/or to the reference profiles (non-acceptable score in the parameter normalization).

MLVA
Overall, 15 of the participating NPHRLs (52%) performed MLVA typing; most (12/15) participated in all three EQA rounds ( The 2015-EQA distribution included a SLV (test strain 17) of one of the repeat strains (c, Table 2). The new allele was stable during the stability test, and reported by all participants identifying the locus, reporting correct allelic profiles for both strains.
General improvement was observed over the three EQA rounds; three (21%) laboratories generated incorrect profiles in the 2013-EQA, whereas only two (14%) laboratories reported incorrect profiles in the 2015-EQA. In the 2014-and 2015-EQA, all participants used the agreed nomenclature, reporting alleles as true number of repeat units and NA/−2 for absent loci [19,20].
The 15 incorrect allelic profiles represented 25 allele errors. Twelve of the errors reported were switching of either two isolates or two alleles. Additionally, eight were missing the presence of a locus, four errors were related to reporting Scores given according to the BioNumerics Gel Quality Grading Guidelines, [18][19][20]. 1 acceptable gel, 0 non-acceptable gel,no participation a Total percentage score of the maximum score of 15 points Scores given according to the BioNumerics Gel Quality Grading Guidelines, [18][19][20]. Scores of 1: non-acceptable gel, scores of 2 or 3: acceptable incorrect allele number, and one error was reporting the presence of a locus where there was none. No common strain characteristics or allele numbers generated errors, and no laboratory repeated the same error in two rounds.

Discussion
Definitive characterization of foodborne pathogens by molecular typing is essential for surveillance, outbreak detection, and source identification. The globalization of food product trade implies that multi-country food-associated outbreaks may occur more frequently [22][23][24][25]. Cross-border comparison of molecular typing data is dependent on robust and comparable standard typing methods. For Salmonella, standard PFGE and MLVA protocols [11,[13][14][15] and nomenclature have been developed [12,21]. Laboratory procedures for the methods accepted for EU-wide surveillance are available at the ECDC website (http://ecdc.europa.eu/en/healthtopics/ food_and_waterborne_disease/surveillance/Pages/index. aspx). However, the full potential of cross-border surveillance relies on ability of the laboratories to generate high-quality typing data routinely and share data as close to real-time as possible. Therefore, this study addressed the quality of the typing data reported from the European NPHRLs expected to share data in TESSy in order to identify critical parameters for improving surveillance, laboratories in need of technical assistance, and performance development. PFGE has been the international gold standard for molecular subtyping of Salmonella, and almost all (27/29) participating NPHRLs registered to this part of the assessment scheme. The quality of a PFGE gel depends on strict adherence to the standard protocol. Even slight deviations from the procedure may result in reduced quality and comparability. This was reflected in the low number of gels (7%) obtaining a total 100% score. Only three laboratories (10%) managed to produce perfect gels, which highlights the challenges related to this method. However, an excellent gel quality is not a necessity for inter-laboratory comparison of profiles, and only two laboratories were unable to produce profiles of high enough quality to be useful for comparison. To improve the performance of these two laboratories, direct technical assistance or training might be useful, as the detailed feedback on their specific problems through individual evalution reports did not circumvent their low performance.
The comparability of profiles relies primarily on the use of correct running conditions, good quality image acquisition, and distinct bands. Indeed, these gel parameters, Bimage acquisition and running conditions^and Bbands^, generated non-acceptable profiles in all EQA rounds. The additional gel parameters received an acceptable score in all cases and seem to be more robust to deviations.
It is of utmost importance to use the PFGE protocol for the relevant organism; however, some (n = 4) laboratories were suspected of applying incorrect running conditions, e.g., the electrophoresis conditions for E. coli O157 were used for Salmonella. Normalization of bands according to an international standard directly depends on correct standard running conditions and adequate numbers of reference lanes. Thus, deviation from the standard renders the profiles incapable of being compared. Laboratories performing PFGE routinely should be expected to use correct running conditions as detailed in the provided protocols. In addition to the use of correct running conditions, proper image acquisition and distinct bands are essential. Production of clear and distinct bands is highly susceptible to deviations from the protocol, and indeed the parameter Bbands^obtained the lowest average score through the three EQA rounds.
Fuzzy and/or thick bands were common causes for nonacceptable and low Bband' scores. In a few cases (4/14 poor band scores), entire lanes were distorted as well. Several factors cause bands to appear fuzzy; poor image capture by improper focusing, use of too narrow wells, and thick gel slices >2 mm. In the specific feedback, laboratories were encouraged to evaluate their in-house procedure systematically to identify even slight deviations from the standard protocol. Low-quality image acquisition highly affects the band quality. The presence of weak bands most likely led several laboratories to increase the exposure time and/or enhance the contrast  of the gel image, causing the bands to appear too thick to separate double bands. Thus, laboratories presumably producing a gel of acceptable quality failed to document this due to improper image capture. A thorough protocol on correct image acquisition was distributed to the participants in the 2014and 2015-EQA rounds, and the quality increased from 2014 to 2015. The combined gel parameter Bimage acquisition and running conditions^could be separated into two different parameters in future EQA rounds to assess the parameters individually.
It can be speculated that the overall quality of PFGE gels produced in NPHRLs not performing this method routinely would diminish. Thus, the substitution of PFGE with whole genome sequence (WGS) typing methods will reduce gradually the need for PFGE EQA schemes. Future assessment schemes should represent the different methodologies applied in parallel in the EU, including PFGE, MLVA and WGSbased analyses.
Participants could select which parts of the EQA to perform, but although the majority (27/29) of the participating NPHRLs performed PFGE typing, only 19 performed the subsequent profile analysis. Laboratories potentially lack access to the software or lack experience in analysing PFGE profiles. However, PFGE-based analysis requires the capacity to analyse PFGE profiles. The performance level displayed by those laboratories participating in the gel analysis was high and improving during the 3-year period. In the EQA-2015, all laboratories demonstrated the ability to produce an acceptable gel analysis in accordance with the guidelines.
The overall performance level for MLVA typing was high and improving through the EQA rounds; all participants reported 100% correct allelic profiles in at least one EQA round. A low rate of 4% (25/420) of misreported allelic profiles was seen among the 420 profiles reported in the 3 years. These errors could have been avoided by proofreading of the results and by taking into consideration the known characteristics of the individual locus, e.g., allele out of expected range, allele not seen before, or locus always present. Absence of amplification of STTR3 was reported a total of five times by two laboratories, although absence of this locus should be considered highly unlikely. In the MLVA scheme, only absence of the loci STTR6 and STTR10 is likely [5]. Furthermore, two of the four incorrect allele numbers reported for STTR3 (allele 409 and 410) are uncommon alleles. STTR3 is a combined locus of two repeat sizes, consisting of 0-5 27-basepair repeat units and 8-14 33-basepair repeat units. However, not all combinations of these seem to occur [12]. Since the incorrect allele numbers of STTR3 were only reported in two of the ten test strains in one of the three EQA rounds, it was not considered as a general error in the participant's procedure. Critical evaluation of results and the use of checkpoints to control the quality and to assess if values are within the expected range and is of high importance.
In summary, lack of proofreading contributed considerably to the errors identified in the MLVA typing. Although these errors were few and should be easy to avoid, uploading of incorrect allelic profiles to TESSy can have significant impact on outbreak detection, since MLVA data are not curated. Thus, proofreading by experienced personnel should be done before sharing of data. TESSy could improve by curation of MLVA data, i.e., confirmation when rare alleles occurs.
Recently, a standard operating procedure for MLVA typing of S. Enteritidis was published [26], and MLVA data for this serovar are now collected in TESSy. Future EQA rounds will include this method to foster harmonization and ensure data quality in the same way as for S. Typhimurium. In Europe, S. Enteritidis and S.Typhimurium are the two most common serovars reported in humans, and MLVA typing provides high resolution with good epidemiological concordance for both serovars [27]. In comparison to whole genome sequence (WGS) typing methods, MLVA is low-cost, and easier to perform and interpret.

Conclusion
The EQA schemes strengthened the laboratory network of FWDs at the EU/EEA level as the number of laboratories participated increased, and results improved during the 3year period. More harmonized and reliable typing methods, particularly molecular typing techniques, allow: (a) effective analysis to detect unusual events, detect uncommon increase of cases and possible national and cross-border outbreaks, (b) increased capacity for further characterization of human isolates, more accurate delineation of outbreaks, and (c) source identification and verification, comparison with data from the food/animals sector.
The participation in the PFGE laboratory part of the assessment was high, whereas the S. Typhimurium specific MLVA method received a lower participation. This emphasizes the need for continued assessment of the different methodologies applied in the NPHRLs in Europe, including both classical methods and new WGS-based typing methods, as it should be anticipated that several methods will be used in parallel by the different NPHRLs in a transition period. This poses a challenge for European-wide surveillance, but the historical data on PFGE and MLVA, combined with the possibility of producing WGS-based typing data on different resolution levels, improves the possibility, of to some degree, correlating results obtained by different methods. Currently, a harmonized procedure for WGS data analysis in routine surveillance remains to be developed. Future EQA shemes could foster the development and harmonization of WGS-based surveillance at the EU level; however, in the transition period, PFGE and MLVA will be a valuable tools for comparison.
External quality assessment is a valuable tool for identifying areas for improvement, and strong and improved performance levels were demonstrated through the course of the three EQA rounds. For PFGE, the main gel quality problems were related to image acquisition, running conditions, and indistinct bands. The current EQA scheme have especially supported the European NPHRLs to use the standard running conditions for PFGE, uniform PFGE band assignment procedures, use of MLVA reference strains, as well as uniform MLVA allelic nomenclature. The European NPHRLs' interest in improving the molecular typing data quality and the participation in EQA schemes is important for the quality of the European-wide surveillance of foodborne infections.