4.1 Main Challenges and Recommendations

The organisation of a TPS is a complex and demanding process. This can be made easier if timelines, rules and specific criteria are defined early, and are followed. This requires more preparation work; however, this pays off when the TPS is running. It is worth trying to foresee possible scenarios and difficulties. In this way, reaction times to act will be shorter, and will affect the running of the TPS to a lesser extent. Possible delays need to be taken into account. For example, delays of sample dispatching due to delays in acquiring Letters of Authorisation can lead to less time available for the TPS participants to perform the tests. Therefore, the TPS participants need to be informed in advance if they will need to acquire a Letter of Authorisation (or other import permits). They also need to be given enough time to prepare for the TPS, to order the specific chemicals, and to perform the tests, while still leaving some time to repeat some tests if needed.

Even though communication with TPS participants is sometimes time consuming, it is crucial to avoid misunderstandings and to avoid the exclusion of results from the analysis. Open and transparent communication between the organisers of a TPS and the commercial kit providers can be crucial, for similar reasons. This communication can also help with the acquisition of chemicals needed for a TPS by the TPS participants, and in the case of some specific chemicals with short expiry dates (e.g., serological tests), this can prevent a possible shortage.

Based on the results of the analysis, the TPS organiser should draw the appropriate conclusions on the performances of the tests included in the TPS, and should determine if they are fit for purpose, considering the scope of the testing. Also, the TPS organiser should comment on the performances of certain tests, in terms of which uses and conditions they are applicable to, and which they are not.

As mentioned several times, open communication between the TPS organiser and the TPS participants is very important. After the completion of the entire TPS, organisers can circulate a satisfaction survey to solicit feedback from the participants. The participants can be asked to rate the experience of their participation, their satisfaction with the organising process, and the possible areas for improvement. Certainly, such surveys should be confidential, and in line with the General Data Protection Regulations (GDPR for the EU region).

Even though a TPS is not designed as an interlaboratory comparison where the proficiency of the laboratories is evaluated, after each laboratory receives the TPS report, they can analyse their own performances. These data can thus be used to identify whether some correction measurements need to be implemented to improve their performances.

The organisation of TPS also has common elements with the organisation of proficiency testing, such as the preparation of the plan, the communication with the participating laboratories, the preparation of samples, testing for stability and homogeneity, etc. The guidance given in this book can also be useful in this context.

As explained in the Introduction, the offers for TPS are still relatively limited in plant health. The authors hope that by sharing their experience and providing guidance and tips in this book, this will help to increase the numbers of laboratories that are willing to organise test performance studies for the global benefit to the plant health diagnostic community.

4.2 Summary

In this book we have used the collected experience that our TPS organisers gained through the organisation of 12 TPS, as well as previous extensive experience each participating institution gained through organisation of similar studies. This information can be used as a ‘textbook’ for organisers of future TPS, not only in the field of plant pest detection, but also in other areas of microbiology. The possibility to use the knowledge gathered in this book beyond the field of plant health, will enable the creation of new network connections and exchange, and as an outcome will continuously improve the concept and the organisation of TPS.

The organisation of a TPS and similar studies is a difficult process, with the need for financing and time. Moreover, it requires a high level of expertise from the TPS organiser to ensure a smooth process and reliable results. However, TPS studies represent the best way to obtain the most reliable validation data for a test of interest, and to study the performance of the test in different laboratories under different conditions. This can help to minimise the uncertainties in the performance characteristics of a test. In addition, TPS are also the most transparent approach to discuss tests and testing strategy with different stakeholders. It provides objective evidence for each test evaluated, despite it being carried out under specific conditions.

By organising 12 TPS on 11 prioritised pests, we aimed to reach the widest possible audience of interested stakeholders in the field of plant pest diagnostics. In all, 640 diagnostic laboratories were invited to participate in these 12 TPS, and 242 of these laboratories confirmed their participation. Five companies also participated, where their main activities included the production of commercial tests for plant pests. Furthermore, their engagement in the test selection process enabled better understanding of the possibilities to change some aspects of the test implementation, and prevented significant deviation of the results due to the changes applied. Each TPS organiser also contacted companies that did not directly participate in the Project (i.e., other manufacturers of commercial tests for plant pests), and indeed, some of their tests were also evaluated in the preliminary studies, if not in the TPS. The evaluation process involved open communication with the companies. As the intergovernmental organisation that is responsible for international cooperation in plant protection, EPPO was included to help to establish and ensure certain levels of harmonisation of all of the procedures during the preparation and organisation of these TPS. During the Project, we harmonised the organisation of the TPS as much as possible between the Project partners, with the aim to set the standards for further similar studies. This was enabled and was achieved through the development and use of common criteria and rules for the selection of tests and laboratories, and common templates (e.g., TPS invitation letters, contracts, technical sheets) and common procedures. When we established the possibility during this process to improve the procedures to better serve the purpose (based on the experience gained in the Project), we up-dated some of these documents. As a Project outcome, this book presents the rules, criteria and documents that we developed and that can serve as a blueprint for similar future studies.