Advertisement

External quality assessment for molecular diagnostic laboratories in Belgium: Can we improve it?

  • Kelly Dufraing
  • Els Lierman
  • Anne Vankeerberghen
  • Sabine Franke
  • Els DequekerEmail author
Open Access
General Paper
  • 139 Downloads

Abstract

External quality assessment (EQA) is an essential part of performance monitoring for molecular laboratories. At the moment, a national law regulates participation in EQA schemes for clinical biology and pathology in Belgium. This study aimed (1) to get insights on how laboratories organize their EQA participation, (2) to poll satisfaction with the current situation (selection of EQA programs in advance by a governmental body), (3) to provide guidance for choosing the most relevant EQA provider and (4) to propose a new model for national performance monitoring. A survey was sent to Belgian laboratories performing molecular tests in the field of microbiology, hematology and pathology with (1) general questions on how they select an EQA provider and (2) their satisfaction of each provider. In total, 25 molecular laboratories [microbiology (N = 13), hematology (N = 8) and pathology (N = 4)] from 14 different hospitals completed the survey regarding their EQA organization. All three laboratory groups indicated to prefer EQA schemes using real patient materials as well as those with varying targets and concentrations. For molecular microbiology and hematology, schemes with a syndromic approach are sought. Since annual participation in EQA becomes burdensome in most laboratories, this paper also offers a risk-based strategy for determining the participation frequency. Based on the needs of Belgian laboratories, three proposals were made: (1) for the proper selection of an EQA scheme, (2) for determining the minimal participation frequency and (3) for the national organization of EQA schemes.

Keywords

External quality assessment Molecular microbiology Molecular hematology Molecular pathology 

Introduction

Participation in external quality assessment (EQA) schemes to monitor the performance of diagnostic, prognostic or predictive testing in clinical routine is fundamental in a properly functioning quality system. EQA participation enables laboratories to regularly check the performance of their routine tests and to benchmark themselves against updated strategies and other laboratories. Previous international studies have indeed demonstrated that EQA schemes help laboratories to continuously improve their testing process [1, 2, 3, 4]. In addition, laboratories can participate in EQA schemes during procedure validation and verification to prove the accuracy of the generated results.

In Belgium, medical laboratories are categorized in three types each with their own license directive: clinical biology, pathology and genetics. Molecular techniques have been widely implemented in all three laboratory types [5, 6, 7]. The costs for these molecular tests are only reimbursed if certain criteria are fulfilled:
  • Molecular testing performed on genetic material of micro-organisms and mentioned in article 24bis of the nomenclature list1 of the ‘Rijksinstituut voor ziekte- en invaliditeitsverzekering/Institut National d’Assurance Maladie-Invalidité’ (RIZIV/INAMI) (e.g., genotyping of hepatitis C) should be done in a laboratory for clinical biology that is licensed by the Ministry and accredited for ISO 15189:2012 according to BELAC, the Belgian accreditation body [8, 9, 10].

  • Molecular testing performed on human genetic material and mentioned in article 33bis of the RIZIV/INAMI nomenclature list (e.g., detection of immunoglobulin gene or T cell receptor gene rearrangements, detection of an acquired variant in the KRAS gene) should be done in a laboratory for clinical biology or pathology that is licensed by the Ministry or in a genetics laboratory recognized by antropogenetics based on the Royal Decrete of 14.12.1987 [9, 11, 12, 13]. In addition, the laboratory should be accredited for ISO 15189:2012 [10].

Both according to the license criteria and ISO 15189:2012, participation to EQA is mandatory (Supplementary Table 1). For the clinical biology and pathology laboratories, Sciensano [the former ‘Wetenschappelijk Instituut Voor Volksgezondheid/Institut Scientifique de Santé Publique’ (WIV/ISP)] is responsible for the organization and follow-up of EQA. Annually, all parameters for which participation in EQA is mandatory are listed by the commissions for clinical biology and pathology. In addition, Sciensano has also founded a commission for oncology that offers a limited number of schemes in the field of solid tumors and hematological cancers. For some molecular markers, Sciensano is the responsible proficiency testing provider (e.g., for KRAS in solid tumors). For other markers, Sciensano subcontracts external EQA providers (e.g., JAK2 p.V617F). For rare or specialized assays or for parameters not provided by Sciensano, laboratories can participate in international EQA schemes, in unofficial schemes (e.g., MolecularDiagnostics.be) or they can exchange samples with other laboratories.

This current situation raises a number of questions which led to the initiation of this study, including: (1) how can you select the right EQA provider for parameters for which Sciensano does not organize an EQA? (2) is it still feasible for each laboratory to participate annually in EQA for each parameter in view of the large number of parameters to be tested and (3) if not, how can a laboratory then determine the minimum frequency? In this study, Belgian laboratories performing molecular testing in the field of microbiology, hematology and pathology were inquired for their current EQA participation strategies and their satisfaction of different EQA providers. Also, it was surveyed what the expectations of the laboratories of the EQA providers are and whether adaptations are desired. Based on these results, a guideline is proposed how to organize EQA participation in a routine setting, while still fulfilling the requirements of the Belgian law and ISO 15189:2012, and how to select a relevant EQA scheme. In addition, a proposal is made for an optimized organization of national EQA programs.

Methods

In 2017, a working group with molecular biologists affiliated to Belgian non-profit clinical laboratories, and representatives from Sciensano, BELAC and the Biomedical Quality Assurance Research Unit of the KU Leuven and UZ Leuven was founded by MolecularDiagnostics.be (a discussion forum for molecular biologists in Belgium). The working group aimed at reaching consensus guidelines for the participation in EQA programs in the field of molecular microbiology, molecular hematology and molecular pathology. There are three different types of EQA providers: those organizing EQA schemes as their core business (e.g., UK NEQAS), those organizing sporadically a limited number of schemes (e.g., EuroClonality) and those organizing sample exchanges between a limited number of participants (e.g., MolecularDiagnostics.be). The first two provider types could be ISO/IEC 17043 accredited, the last type not.

Surveys were spread to all Belgian laboratories where molecular testing is performed in the field of microbiology (N = 38), hematology (N = 29) and pathology (N = 17). The aim of this survey was to get an overview of (1) how the different laboratories in Belgium organize their EQA participation, (2) their experience with different EQA providers, (3) specific problems and (4) unmet needs of the different laboratories with regard to EQA schemes. Three separate surveys were sent for molecular microbiology, molecular hematology and molecular pathology. Each survey contained three sections: laboratory characteristics, general questions regarding the selection of an EQA provider (N = 11) and detailed questions to measure the satisfaction with specific EQA providers for a certain parameter (N = 11 per parameter and provider). During telephone calls, additional questions were asked when needed. Given the low number of participants (N = 25), data were only analyzed in a descriptive way. Based on laboratory input, the working group compared and discussed existing practices and hurdles in the performance of EQA schemes.

Results

In total, 25 molecular laboratories [microbiology (N = 13), hematology (N = 8) and pathology (N = 4)] from 14 different hospitals completed the survey regarding their EQA organization (Table 1). Participating laboratories were mainly from Flanders and affiliated to university and non-university non-profit hospitals. Different healthcare professionals are responsible for monitoring EQA results, and in most laboratories, more than one person is involved in the EQA process.
Table 1

Characteristics of the participating laboratories

 

Molecular microbiology (N = 13)

Molecular hematology (N = 8)

Molecular pathology (N = 4)

N absolute

%

N absolute

%

N absolute

%

Type of laboratory

 Clinical biology

13

100.0

7

87.5

0

0.0

 Pathology

0

0.0

0

0.0

3

75.0

 Center for human genetics

0

0.0

1

12.5

1

25.0

Type of institute

 University

5

38.5

3

37.5

3

75.0

 Non-university non-profit

8

61.5

5

62.5

1

25.0

Region

 Flanders

10

76.9

8

100.0

4

100.0

 Brussels

2

15.4

0

0.0

0

0.0

 Wallonia

1

7.7

0

0.0

0

0.0

Who is responsible for monitoring the EQA results in your laboratory? (multiple answers were possible)

 Quality manager

4

30.8

1

12.5

4

100.0

 Pathologist

0

0.0

0

0.0

2

50.0

 Molecular biologist

11

84.6

6

75.0

4

100.0

 Clinical biologist

7

53.8

5

62.5

0

0.0

 Laboratory director

0

0.0

2

25.0

0

0.0

Data are shown as the absolute number and the percentage of laboratories between brackets

The majority of these 25 laboratories have a procedure in place for the selection of EQA providers (Table 2). Schemes using real primary patient material as well as those varying their focus throughout the years are preferred by all three laboratory groups. For molecular microbiology and hematology, schemes with a syndromic approach are preferred. All laboratories indicated that they participate in EQA schemes to verify their performance in the daily laboratory setting as much as possible. Therefore, next to scoring of analytical test results, the majority of the laboratories also ask for scoring of the clinical interpretation. In reality, the EQA schemes do not always reflect daily routine, which impacts the drive to undertake action based on EQA results (Table 3).
Table 2

Criteria for EQA provider selection and their average importance

 

Molecular microbiology (N = 13) %

Molecular hematology (N = 8) %

Molecular pathology (N = 4) %

Why do you participate in EQA?

 To verify your performance (already validated method)

13

100.0

8

100.0

4

100.0

 To validate your test methods

11

84.6

7

87.5

3

75.0

 Mandatory according to art. 24bis/33bis

13

100.0

6

75.0

1

25.0

 Mandatory by the Belgian law

12

92.3

7

87.5

3

75.0

Documented procedure for the selection of external services including EQA providers

 Yes

10

76.9

5

62.5

4

100.0

 No

3

23.1

3

37.5

0

0.0

Importance of selection criteria (0 = not important—4 = very important)

 Mandatory by the Belgian law

3.7

4.0

2.8

 Real patient material (e.g., blood)

3.3

3.2

3.8

 DNA/RNA from real patient material

2.2

3.2

3

 Artificial samples (cell lines, plasmids)

1.9

2.2

1.8

 Variation in focus of the EQA scheme (sensitivity vs. false positives, cross detection vs. multiple variant detection vs….)

3.1

3.6

3.3

 Syndromic approach

3.6

3.8

2.3

 ISO recognition of the EQA provider

2.7

3.2

3.0

 Scoring by consensus or expert panel

3.2

3.4

3.3

 Number of samples per year

3.1

3.0

3.3

 Samples are divided in multiple distributions per year (e.g., 3 times 4 samples)

2.3

2.2

3.0

 Cost of the EQA scheme

3.1

2.8

2.8

Table 3

Laboratory view on the use of EQA results in general

 

Molecular microbiology (N = 13)

Molecular hematology (N = 8)

Molecular pathology (N = 4)

To which extent do you agree that EQA schemes reflect daily performance?a

2.7

2.7

2.6

To which extent do you agree that EQA schemes encourage you to take actions to increase quality of daily performance?a

2.9

2.9

3.0

To which extent do you agree that EQA should contact poor performers and follow-up on their action plan?a

2.4

2.4

2.5

To which extent do you agree that a poor performance should lead to temporally loss of reimbursement by the RIZIV/UNAMI?a

1.8

1.6

2.3

Which parameters should be scored by an EQA provider?

N = 13

%

N = 8

%

N = 4

%

Analytical test results

12

92.3

8

100.0

4

100.0

Clinical interpretation

10

76.9

6

75.0

3

75.0

a0 = I do not agree—4 = I fully agree

For the most important (in number of samples or most widespread) parameters for molecular microbiology, hematology and pathology, the laboratories gave their view on commonly used EQA providers regarding sample type, number of samples, degree of difficulty, quality of the report, etc. In Table 4, an overview of available EQA schemes for molecular testing is shown. Most laboratories participate in schemes from more than one provider: for molecular microbiology the average number of providers is 2.8, molecular hematology laboratories participate in 3.4 schemes on average and molecular pathology laboratories in 3.8. Some laboratories also participate in different schemes for the same test.
Table 4

Overview of EQA schemes where survey responders participated in for molecular testing

Domain

Provider

Website

Marker type

Sample matrix

Number of samples/year

Number of rounds/years

Syndromic approach possible

Molecular microbiology

QCMD

https://www.qcmd.org/

Bacteria

Dependent on the target, often relevant matrices spiked with cultured bacteria

10–12

1 or 2

Yes

Viruses

Dependent on the target, often relevant matrices spiked with cultured viruses

10–12

1 or 2

Yes

Parasites

Dependent on the target, often relevant matrices spiked with cultured parasites

10–12

1 or 2

Yes

INSTAND

https://www.instand-ev.de/

Bacteria

Dependent on the target, often relevant matrices spiked with cultured bacteria

4

1 or 2

Yes

Viruses

Dependent on the target, often relevant matrices spiked with cultured viruses

4–8

2

Yes

Parasites

Dependent on the target, often relevant matrices spiked with cultured parasites

3

2

No

Fungi

Lyophilized blood or living cultures

2–4

2

No

Sciensano

https://www.wiv-isp.be/QML/

Bacteria

Outsourced to QCMD

   

Viruses

Parasites

CAP

https://www.cap.org/

Bacteria

Liquid or swab simulated clinical isolate specimens

15

3

Yes

Viruses

Liquid specimens

15

3

Yes

Parasites

Liquid specimens

6

2

No

Molecular hematology

UK NEQAS

http://www.ukneqasli.co.uk

IG/TR

Lyophilized cell lines or patient derived material

6

3

No

Gene alterations

Lyophilized cell lines

4–6

2–3

No

Euroclonality

http://kras.eqascheme.org/info/public/static/euroclonality.xhtml

IG/TR

Extracted DNA sample

5

1

No

SKML

https://www.skml.nl/

Gene alterations

Frozen cell lines

5

1

No

MD.be

https://moleculardiagnostics.be/

Gene alterations

(Extracted) DNA/cDNA/RNA sample

4

1

Yes

Sciensano

https://www.wiv-isp.be/QML/

Gene alterations

JAK2 V617F, BCR-ABL, AML translocations (t(8;21), t(15;17) and inv(16)) outsourced to UKNEQAS

   

Molecular pathology

EMQN

www.emqn.org

Gene alterations

FFPE slides; 2 from patient material and 8 artificial reference materials

10

1

Yes

ESP

lung.eqascheme.org

Gene alterations, gene rearrangements

FFPE slides from patient material, occasionally supplemented by reference materials

10

3

No

GenQA/UK NEQAS for molecular genetics

www.genqa.org

Gene alterations

FFPE samples can be supplied as (1) rolled sections, (2) slide mounted sections, or (3) rolled sections plus one slide mounted section for H&E staining

4–10

1 or 2

Yes

UK NEQAS for molecular genetics

https://www.ukneqasiccish.org

Gene rearrangements

Cell lines and tissue samples

4

4

No

cIQc

http://cpqa.ca/

Gene rearrangements

Tissue microarray slides

> 30

1

No

Sciensano

https://www.wiv-isp.be/QML/

Gene alteration

FFPE slides from artificial reference materials

4

1

No

In total, 13 molecular microbiology laboratories gave their opinion on the following EQA providers: QCMD, Sciensano, INSTAND, CAP, UK NEQAS, SKML, NRC/UCL and MolecularDiagnostics.be. An ideal EQA scheme for molecular microbiology uses a syndromic approach (to reduce the number of EQA samples), a real patient matrix (that changes between schemes) and varies in strains (including also recent strains) and concentration. Next to this, it is important that enough material is provided to allow retesting, that the bacterial concentration is mentioned (where useful) and that the data analysis considers when different techniques are used by the participants. There was a clear preference for less samples per round and more rounds per year compared to many samples in one round.

Eight molecular hematology laboratories returned the questionnaire. They participate in EQA schemes organized by UK NEQAS, EuroClonality, SKML, Sciensano, ERIC and MolecularDiagnostics.be. Compared to microbiology and pathology, the EQA schemes sporadically organized by MolecularDiagnostics.be are commonly used. The molecular hematology laboratories prefer the presence of a clinical context to allow a proper clinical interpretation which is currently often lacking. A clear, simple laboratory-specific report showing the laboratory’s performance status over time is appreciated. There is a need to expand quantitative schemes as well as the use of more challenging samples.

Although only four molecular pathology laboratories completed the survey, they agree on positive and negative points of EQA schemes they participated in (ESP, Sciensano, EMQN, cIQc, CAP, UK NEQAS, NordiQC and GenQA). Schemes providing in time extensive personalized reports including a comparison of the participating laboratories and schemes offering a plan of action after a bad result are preferred. Non-representative samples (e.g., cell lines), too little variation in target and absence of challenging samples are common problems of several EQA schemes in the field of molecular pathology.

Since Sciensano is the responsible party for national EQA follow-up for parameters covered by articles 24, 24bis and 33bis of the RIZIV nomenclature list, they annually provide a list which was agreed by the commissions of clinical biology and pathology with parameters for which participation in EQA programs is mandatory. According to the laboratories in molecular hematology, microbiology and pathology, most but not all relevant markers are present (Table 5). Therefore, 40 % of the laboratories would like an extension of the list. Since some laboratories feel that the current situation is not ideal to test their daily performance, the following proposal was phrased: ‘Let’s assume that laboratories would have a choice of a limited number of approved EQA providers to choose for each reimbursed parameter, and Sciensano would still be responsible for the registration, payment of the registration fee and national follow-up. Would you prefer this situation over the current situation?’ Of all questioned laboratories, 12 laboratories answered ‘yes,’ 7 ‘no’ and 1 claimed that national follow-up is irrelevant as they are the only laboratory testing for certain rare parameters. The laboratories which answered ‘no’ all gave this answer because Sciensano already selected the provider of their choice. Elements deemed most important for approving EQA providers include: clinically relevant sample types and matrices and variation in target concentration (Table 5).
Table 5

Feedback on the proposal for a new national model

 

Molecular microbiology (N = 10)

Molecular hematology (N = 6)

Molecular pathology (N = 4)

Sum (N = 20)

Q1: Are sufficient and relevant parameters for which EQA is mandatory included on the annual list with of Sciensano?

 Yes

50.0

66.7

0.0

45.0

 No

30.0

33.3

75.0

40.0

 Not applicable

20.0

0.0

25.0

15.0

Q2: Assume Sciensano would give a choice of possible EQA providers per parameter and a fixed budget for participation. Would you prefer this?

 Yes

50.0

50.0

100.0

60.0

 No

40.0

50.0

0.0

35.0

 Not applicable

10.0

0.0

0.0

5.0

Q3: Which elements should certainly be taken into consideration by the committees of Sciensano when selecting EQA providers?

 Clinically relevant sample types

80.0

83.3

75.0

80.0

 Clinically relevant sample matrices

60.0

66.7

75.0

65.0

 Variation in target concentration

70.0

50.0

50.0

60.0

 Number of samples

30.0

33.3

25.0

30.0

 Clear reporting

30.0

0.0

75.0

30.0

 Availability of disease-based schemes

70.0

33.3

25.0

50.0

 Multiple rounds per year

60.0

33.3

0.0

40.0

 Experience of EQA provider

0.0

0.0

25.0

5.0

 Availability for other markers

10.0

16.7

25.0

15.0

Q4: How often do you participate?

 Annually

80.0

100.0

75.0

85.0

 Every 2 years

10.0

0.0

0.0

5.0

 Every 3 years

10.0

0.0

25.0

10.0

Data shown as percentage of laboratories

Discussion

In terms of laboratory accreditation, most requirements and guidelines listed in Supplementary Table 1 require participation in EQA schemes [8, 10, 14, 15, 16, 17]. These sources do not specify how to select an EQA provider nor do they define the frequency of participation. This study surveyed Belgian laboratories to identify their current local EQA strategy and their needs, to come to an overview what molecular laboratories look for in an EQA scheme and to formulate a proposal how to manage their EQA participation.

One major limitation of the study is that only one-third of the invited laboratories completed the survey. Possible explanations could be the extensiveness of the questionnaire, high workload of people working in molecular testing laboratories or the perception that giving their opinion is useless since participation in EQA is often considered as something mandatory. That is why we also want to raise awareness about the importance of selecting a EQA provider that fulfills the laboratories’ needs with this paper. Although there are less laboratories doing molecular testing in Wallonia (24 %) and Brussels (12 %) as compared to Flanders (63 %), there still is some overrepresentation of Flemish laboratories in our study. Nevertheless, the answers reflect the problematic points of the participation in EQA and some general trends were observed. In general, laboratories have a high willingness to participate in EQA schemes, not only because it is mandatory, but also because they want to know their performance level and improve it if necessary. The received surveys indicate that there is room for improvement in the selection of EQA schemes to better fit the laboratories’ needs. On the other hand, laboratories might not participate in the EQA program that is best suited. Although guidelines exist for organizing EQA programs (e.g., ISO/IEC 17043), there are still large differences between the accredited EQA providers. The majority of laboratories already have a selection procedure for EQA schemes in place. Participation in a particular scheme can be mandatory by law (for clinical biology and pathology) or is guided by the distribution of real patient material, the use of a syndromic approach and challenging samples.

Proposed workflow for EQA organization in a molecular laboratory

Based on the survey and an already existing guideline for clinical chemistry [18], we identified the following critical elements to consider when selecting an EQA provider:

1. Accreditation

As required by multiple sources, it is recommended to choose a provider that is ISO/IEC 17043 accredited

Note 1: If a non-accredited provider is chosen, laboratories should have a good reason and document this

Note 2: If EQA schemes (accredited or non-accredited) are not available, other forms of interlaboratory comparison (e.g., sample exchange between laboratories) are recommended

2. Sample type

Select an EQA provider that distributes samples which are fit for purpose

 If daily performance is to be evaluated, providers that send real patient material are preferred

 If the limitations of the detection method are to be evaluated, artificial samples in a matrix resembling the routine matrix could be used (e.g., paraffin-embedded cell lines)

 If reader accuracy or interpretation of the test outcome is to be evaluated, digital cases are also an option

Note: It is also advised to evaluate pre-analytical and post-analytical processes as much as possible. At the moment, a limited number of EQA schemes assessing the pre-analytical phase is available, although pre-analytical errors impact the whole upstream testing process

3. Target range

EQA providers distributing samples containing a clinically relevant range of concentrations or allelic frequencies are preferred. If only strongly positive cases are included, the laboratory will not have a clear insight on its performance

Note: National guidelines regarding clinical relevance of tested targets should be considered when participating in an international EQA scheme

4. Target variation

Select a provider that includes a variance in clinically relevant targets, either within one parameter or within a disease group (‘syndromic approach’)

Example 1: EQA schemes for RAS testing in molecular pathology that only include mutations in codons 12 or 13 of the RAS gene should be avoided

Example 2: EQA schemes for influenza virus with recently/currently circulating strains are preferred

Example 3: EQA schemes for myeloproliferative neoplasms where JAK2 p. V617F and CALR are tested in parallel can be preferred

5. Frequency

The frequency of the selected EQA scheme should best match the ‘ideal participation frequency’ of the laboratory that is identified based on a risk analysis, as described below [15]

6. Educational value

EQA schemes should be sufficiently educational. This implies that challenging cases could be included but also that the report should be clear and give sufficient explanation on the test outcomes. In addition, EQA schemes providing feedback on the interpretation of test results are preferred

7. Number of participants

If the laboratory wants to benchmark itself, it is advisable to select an EQA provider with a high number of participants. A better overview of performance related to the techniques used might then be given in the final report

To be able to follow these steps, it is essential that the commissions for clinical biology and pathology of Sciensano as well as the laboratories know which are the available EQA schemes. To help them make an informed choice, EQA providers must be transparent about the material they distribute and the service they provide.

Since there are no sources specifying the minimal required frequency of participation in EQA schemes, BELAC and Eurachem [14, 19] stated that the frequency should be based on a risk analysis considering the following elements;

1. Number of samples annually tested

A previous study has indicated that analyzing a higher number of samples is correlated with more experience and less errors [1]

2. Possibility to group tests based on disease or on technique

Example 1: Make sure that a panel for next-generation sequencing (NGS) is included once a year in an EQA program (e.g., year 1 for colon, year 2 for lung, etc.)

Example 2: Participate annually in an EQA scheme for respiratory pathogens

Note: As a laboratory you have to trust that the EQA provider will alter the variants annually, so all pathogens will be tested over a larger period of time

3. The number of involved operators and frequency of turnover in technical staff

It is good practice that each operator participates in at least one EQA scheme each year

4. The education level and general experience of the personnel

For starting operators, participation in EQA schemes could be part of their training program

5. The availability of certified reference materials

If certified reference materials are available and continuously used as internal controls, the need for frequent EQA participation might be lower

6. The complexity of the measurement technique

High complex multi-step testing methods [e.g., next-generation sequencing (NGS)] might be more error-prone then single-step tests and thus require more stringent quality control measures

7. The level of criticality of the result and its final use

Tests of which results alter the duration of a patient’s life or quality of life should be handled with more care

A re-evaluation of the defined participation frequency has to be performed in case of important changes in laboratory organization. Moreover, 3 years is considered as the maximum time interval between two EQA participations.

In Belgium, most laboratories still annually participate in EQA for each offered test. For larger testing centers and laboratories using big NGS panels, this becomes untenable. Grouping parameters per sub-discipline can help. A sub-discipline is then defined as ‘an area of technical competence defined by a minimum of one Measurement Technique, Property and Product, which are related,’ meaning tests can be performed with the same training of the personnel [15]. Furthermore, tests that are used for several years with no change in setup or interpretation may not need an annual EQA. The following EQA participation strategy is used by one of the study participants: for new tests the laboratory participates annually. If during the first 3 years no major genotyping or interpretation errors (i.e., errors which could harm patients) are made, participation is from then on only required once every 3 years. It is also considered that every laboratory technique (cfr. ‘sub-discipline’ as per above) is each year covered by an EQA, and that each technical responsible person participates once per year.

Use of EQA results

One important purpose of EQA is to educate laboratories. For laboratories, it is important, to the extent possible, to treat EQA samples the same way as patient samples. EQA providers also have a responsibility, by drafting a clear final report and by providing sufficient individual feedback. It is important that the laboratory has a procedure for communicating EQA results within the laboratory (e.g., with laboratory management or during the management review) and for taking actions. With a score varying between 2.9 and 3.0 on 4 in the three disciplines, respectively, laboratories feel quite encouraged by EQA providers to take actions. Surprisingly, in only 31 % of the surveyed laboratories in molecular microbiology and 13 % in molecular hematology, the quality manager is (co)-responsible for the follow-up of the results, whereas this is 100 % in molecular pathology laboratories.

Proposal for a national model for organizing external quality assessment

The results of the surveys done in this study showed that improvements to the current situation of mandatory EQA participation are desired.

Now, laboratories are participating in schemes that do not always coincide with their defined needs in terms of frequency of participation, sample type, target range, etc. To benefit from the nationally organized schemes, for each parameter at least 2 different schemes should be offered with a comparable degree of difficulty, if available. As such, every laboratory can decide which scheme best fits their needs (e.g., syndromic approach versus single parameter test). In addition, more parameters should be included in the list for which EQAs are nationally offered to avoid that laboratories opt for an easy EQA for the missing parameters. It is very important that the educational value of these schemes should prevail the fact that participation is mandatory. The commissions for clinical biology and pathology and the commission for oncology have with these results the information to discuss the possibilities to answer to these desires and to communicate the wishes to the EQA organizers. This publication can thus be a tool for the commissions to select the most appropriate EQA provider, if this is not available to encourage the providers to develop more appropriate EQAs.

There was a clear overlap between the three disciplines (molecular microbiology, hematology and pathology) with regard to ‘important elements for selecting an EQA scheme.’ Representative samples, syndromic approach, variation in target and concentration/allelic frequency, scoring of clinical interpretation and a simple personalized report delivered in time with a detailed comparison of different laboratories and techniques are highly requested. For each discipline, there were some extra recurrent elements such as decreasing the number of samples per round in some microbiology EQA schemes, and the need for more quantitative schemes in the hematology setting.

To limit the number of annual EQA participations for tests with an obligatory accreditation, laboratories should perform a risk analysis to determine for each parameter the frequency of EQA participation. This approach, supported by BELAC, is contradictory with the annual compulsory EQA participation by the Belgian law for tests requiring accreditation. It is therefore requested to mitigate this obligation and give the responsibility to the laboratories to define the frequency of EQA participation.

Conclusion

This study aimed to create an overview of how Belgian laboratories currently handle their participation in EQA programs, to survey whether they are satisfied and to propose a new national model for the selection of EQA providers and follow-up of laboratory performance by the different national commissions. Although participation in EQA programs is mandatory according to national and international guidelines, no clear requirements exist for the frequency of participation and the selection criteria for EQA providers. Since participation in EQA is in Belgium regulated by a national body, laboratories are currently not always participating to the most relevant scheme for their daily routine. Therefore, the authors propose a new model, offering laboratories more options. To conclude, in this paper guidance is offered to determine the frequency for participation in EQA schemes and to select the most relevant EQA provider.

Footnotes

  1. 1.

    The RIZIV/INAMI nomenclature list is an appendix to the ‘Belgian Official Gazette’ which contains a list of medical services for which a full or partial reimbursement is offered via the mandatory healthcare insurance.

Notes

Acknowledgements

The authors would like to thank Elke Van Rossen (BELAC), Vanessa Ghislain (Sciensano) and Elke Boone for sharing their expertise, their contribution to the survey and their input for the manuscript. They also thank Kirsten Vanwelden for her practical support to the survey and MolecularDiagnostics.be for financially supporting this manuscript.

Funding

Publication fees for this manuscript are funded by MolecularDiagnostics.be.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

769_2019_1410_MOESM1_ESM.docx (13 kb)
Supplementary material 1 (DOCX 12 kb)

References

  1. 1.
    Tack V, Schuuring E, Keppens C, Hart N, Pauwels P, van Krieken H, Dequeker EMC (2018) Accreditation, setting and experience as indicators to assure quality in oncology biomarker testing laboratories. Br J Cancer 119(5):605–614CrossRefGoogle Scholar
  2. 2.
    Keppens C, Tack V, Hart N, Tembuyser L, Ryska A, Pauwels P, Zwaenepoel K, Schuuring E, Cabillic F, Tornillo L, Warth A, Weichert W, Dequeker E, EQA Assessors Expert Group (2018) A stitch in time saves nine: external quality assessment rounds demonstrate improved quality of biomarker analysis in lung cancer. Oncotarget 9(29):20524–20538CrossRefGoogle Scholar
  3. 3.
    Slotved HC, Sheppard CL, Dalby T, van der Ende A, Fry NK, Morfeldt E, Nyholm O, Rokney A, Ron M, Siira L, Scott KJ, Smith A, Thom L, Toropainen M, Vestrheim DF (2017) External quality assurance for laboratory identification and capsular typing of Streptococcus pneumoniae. Sci Rep 7(1):13280CrossRefGoogle Scholar
  4. 4.
    Abdad MY, Squires RC, Cognat S, Oxenford CJ, Konings F (2017) External quality assessment for arbovirus diagnostics in the World Health Organization Western Pacific Region, 2013–2016: improving laboratory quality over the years. Western Pac Surveill Response J 8(3):27–30CrossRefGoogle Scholar
  5. 5.
    Au TH, Wang K, Stenehjem D, Garrido-Laguna I (2017) Personalized and precision medicine: integrating genomics into treatment decisions in gastrointestinal malignancies. J Gastrointest Oncol 8(3):387–404CrossRefGoogle Scholar
  6. 6.
    Del Campo JA, Parra-Sánchez M, Figueruela B, García-Rey S, Quer J, Gregori J, Bernal S, Grande L, Palomares JC, Romero-Gómez M (2018) Hepatitis C virus deep sequencing for sub-genotype identification in mixed infections: a real-life experience. Int J Infect Dis 67:114–117CrossRefGoogle Scholar
  7. 7.
    Delic S, Rose D, Kern W, Nadarajah N, Haferlach C, Haferlach T, Meggendorfer M (2016) Application of an NGS-based 28-gene panel in myeloproliferative neoplasms reveals distinct mutation patterns in essential thrombocythaemia, primary myelofibrosis and polycythaemia vera. Br J Haematol 175(3):419–426CrossRefGoogle Scholar
  8. 8.
    Wet van 19 September 2018. Koninklijk besluit tot wijziging van de artikelen 24, § 1, en 33bis, van de bijlage bij het koninklijk besluit van 14 September 1984 tot vaststelling van de nomenclatuur van de geneeskundige verstrekkingen inzake verplichte verzekering voor geneeskundige verzorging en uitkeringen. http://www.ejustice.just.fgov.be/eli/besluit/2018/09/19/2018014254/justel. Accessed 5 Apr 2019
  9. 9.
    K.B. 3 December 1999 Koninklijk besluit betreffende de erkenning van de laboratoria voor klinische biologie door de Minister tot wiens bevoegdheid de Volksgezondheid behoort. http://www.ejustice.just.fgov.be/eli/besluit/1999/12/03/1999024072/justel. Accessed 5 Apr 2019
  10. 10.
    International Organization for Standardization (ISO) (2012) Medical laboratories: requirements for quality and competence. Standard No. ISO 15189:2012. ISO, GenevaGoogle Scholar
  11. 11.
    Wet van 19 September 2018. Koninklijk besluit tot wijziging van artikel 32 van de bijlage bij het koninklijk besluit van 14 September 1984 tot vaststelling van de nomenclatuur van de geneeskundige verstrekkingen inzake verplichte verzekering voor geneeskundige verzorging en uitkeringen. http://www.ejustice.just.fgov.be/eli/besluit/1984/09/14/1984022288/justel. Accessed 5 Apr 2019
  12. 12.
    Wet van 5 December 2011 Koninklijk besluit betreffende de erkenning van de laboratoria voor pathologische anatomie door de Minister tot wiens bevoegdheid de Volksgezondheid behoort. http://www.ejustice.just.fgov.be/eli/besluit/2011/12/05/2012024054/justel. Accessed 5 Apr 2019
  13. 13.
    Wet van 14 December 1987 Koninklijk besluit houdende vaststelling van de normen waaraan de centra voor menselijke erfelijkheid moeten voldoen. http://www.ejustice.just.fgov.be/eli/besluit/1987/12/14/1987025417/justel. Accessed 5 Apr 2019
  14. 14.
    Belgian Accreditation Body (BELAC) (2011) Proficiency testing: guidelines on the level of participation and evaluation of performances in proficiency testing activities in the context of accreditation assessments. Standard No. 2-106 Rev 1-2011. BELAC, BrusselsGoogle Scholar
  15. 15.
    European Accreditation (EA) (2010) Guidance on the level and frequency of proficiency testing participation. EA 4/18TA INF. https://european-accreditation.org/publications/ea-4-18-inf/. Accessed 25 Sept 2019
  16. 16.
    International Laboratory Accreditation Cooperation (ILAC) (2014) Policy for participation in proficiency testing activities. ILAC-P9:06/2014. https://ilac.org/publications-and-resources/ilac-policy-series/. Accessed 25 Sept 2019
  17. 17.
    Joint Commission International (JCI) (2007) Joint Commission International Accreditation Standards for Hospitals, 6th edn. JCI, Oak BrookGoogle Scholar
  18. 18.
    James D, Ames D, Lopez B, Still R, Simpson W, Twomey P (2014) External quality assessment: best practice. J Clin Pathol 67:651–655CrossRefGoogle Scholar
  19. 19.
    Eurachem (2011) Selection, use and interpretation of proficiency testing (PT) schemes, 2nd ed. https://www.eurachem.org/index.php/publications/guides/usingpt. Accessed 25 Sept 2019

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Kelly Dufraing
    • 1
  • Els Lierman
    • 2
  • Anne Vankeerberghen
    • 3
  • Sabine Franke
    • 4
  • Els Dequeker
    • 1
    Email author
  1. 1.Department of Public Health and Primary Care, Biomedical Quality Assurance Research UnitUniversity of LeuvenLouvainBelgium
  2. 2.Center for Human GeneticsUniversity Hospital LeuvenLouvainBelgium
  3. 3.Clinical Laboratory of MicrobiologyOLVZ AalstAalstBelgium
  4. 4.Department of Human GeneticsUniversity Hospital (CHU Liège)LiègeBelgium

Personalised recommendations