Introduction

In recent years, regulatory adaptations in the United States of America (USA) and the European Union (EU) with regard to the use of in vitro diagnostic applications in the management of drug therapy have been introduced [1, 2]. This also affected the performance requirements of next generation sequencing (NGS).

Analytical validity is considered a critical step in test assessment. According to the US in vitro diagnostic (IVD) regulation and the new in vitro diagnostic medical devices regulation (IVDR) in the EU, the analytical and clinical performance of an in vitro diagnostic test to be legally marketed should first be evaluated [3, 4]. Therefore, the suitability of an in vitro diagnostic NGS test needs to be demonstrated for the intended use [5, 3]. Laboratories offering targeted NGS [6], whole exome sequencing (WES) and whole genome sequencing (WGS) as diagnostic tests are also obliged to evaluate performance characteristics and carry out a full validation prior to clinical application as laboratory developed test (LDT). In the EU, the evaluation of analytical performance of an IVD or LDT should encompass a variety of general criteria also including the collection and handling of according specimen. However, explicit criteria that meet the needs of the assessment of analytical NGS performance are not specified [3]. Furthermore, standardization for performance evaluation is currently controversial [2].

For the analytical validation of targeted NGS based on NGS panels, in a recent publication on the consensus recommendation of the Association for Molecular Pathology and College of American Pathologists, an assessment of accuracy in terms of the positive percent agreement (PPA) and positive predictive value (PPV) was recommended. Furthermore, the precision of variant detection in terms of reproducibility and repeatability, reportable range and reference range, limits of detection, analytical specificity (interfering substances) and carryover was included in the recommendation [7].

In the USA, further guidance for analytical validation of in vitro diagnostics on the basis of NGS is provided by non-binding recommendations of the Food and Drug Administration (FDA). Here, test performance characteristics such as accuracy also in terms of PPA, technical PPV and additionally negative percent agreement (NPA), precision, limit of detection, and analytical specificity in terms of interference, cross-reactivity and cross-contamination are suggested for evaluation [8].

FDA documents such as the “Summary of Safety and Effectiveness” provide publicly available analytical performance evaluations of NGS-based diagnostic tests as companion diagnostics. However, predominantly for personalized therapeutic management for drugs applied in oncology [9]. In the USA, such approved NGS-based tests can therefore be applied in clinical practice for the intended use. However, no NGS-based companion diagnostic is approved to guide therapy with regard to the pharmacogenetic profile of drug metabolizing enzymes. Large-scale analyses such as whole exome and whole genome sequencing are increasingly used as laboratory developed tests [5]. Such data is considered a valuable resource for pharmacogenomic profiling [10,11,12]. However, the added value of WES and WGS applications compared to other large scale molecular genetic tests in pharmacogenetics is controversial and limitations were reported with regard to coverage and short read-based assessment [12, 13]. The status of the validation of NGS-based applications for clinical pharmacogenomic (PGx) profiling is unclear, although value for therapeutic management has been reported [10, 14], and clinical implementation studies have been performed in several countries [15,16,17]. Also in the EU, for a few drug prescriptions, e.g. testing for cytochrome P450 2C19 (CYP2C19) variants prior to therapy with atazanavir is recommended and testing for cytochrome P450 2D6 (CYP2D6) prior to therapy with eliglustat is even required [18]. However, PGx testing is not applied as companion diagnostics yet [9]. Approaches to repurpose e.g. whole exome sequencing data obtained in clinical settings also for pharmacogenetic profiling suggest that such secondary findings may provide an extraordinary opportunity to integrate valuable information for personalized treatment. However, the feasibility evaluations reported limitations with regard to several pharmacogenes [19, 20]. Furthermore validity of these approaches needs to be established [20].

Here, on the basis of current literature identified at the platforms Pubmed and Pubmed Central, we provide an overview of the available information on the analytical validation status of applied targeted NGS including second and third generation sequencing and furthermore whole exome and whole genome sequencing for targeting relevant pharmacogenetic biomarkers. Additionally, we discuss the potential of third generation sequencing techniques for whole genome sequencing and the assessment of pharmacogenetic information provided by such techniques.

Methods

The FDA list “Nucleic acid-based tests” was applied in order to analyze the validation status of NGS applications for genetic testing of pharmacogenes coding drug metabolizing enzymes. Here, listed genes of the category “drug metabolizing enzymes” were used for further investigations [21]. The list provides genetic tests that have either been cleared or approved. Such tests are appropriate for comparison in the evaluation of the analytical validation of other testing techniques for pharmacogenetic profiling. Second generation and third generation sequencing techniques were included in the analysis.

Analytical validation on the basis of cleared or approved genetic tests

Screening for publications providing a performance comparison of FDA approved nucleic acid-based diagnostic tests concerning drug metabolizing enzymes with second and third generation sequencing applications was performed in August 2022. Many of these tests were also CE certified in the European Union. The aim was to identify whether such approved or cleared tests have been used to validate NGS based applications with a focus on relevant pharmacogenes. According to the FDA recommendations, such tests identified as appropriate by the FDA, should be applied as reference tests of choice in an analytical validation [8]. Here, the publication platforms Pubmed and Pubmed Central were screened for according publications using the keywords “test name” (Table 1, see Trade Name), associated gene(s) (Table 1, see “Gene”), next generation sequencing or whole exome sequencing or whole genome sequencing or long read sequencing or Nanopore or Pacific Biosciences and validity or validation. Publications identified for second and third generation sequencing applications were evaluated separately. Congress abstracts were not included and duplications were excluded. Publications not providing information on validation results with the according FDA cleared or approved nucleic acid-based diagnostic tests involving the according gene in the method or result section of the publication were excluded. The few publications included were analyzed for the evaluation of NGS performance on the basis of relevant analytical validation criteria recommended for NGS in current literature such as accuracy, precision, analytical specificity (endogenous and exogenous interference, cross-reactions, cross-contamination) and limit of detection [7, 8].

Table 1 Test search with the according keywords (trade name, gene(s), NGS application, validity or validation) on the basis of the FDA list “Nucleic acid-based tests” focusing on the category “drug metabolizing enzymes”
Full size table

Validation status of NGS based applications

To further evaluate the validation status of targeted NGS, WES and WGS in research and the clinic with regard to the previously analyzed pharmacogenes (Table 1), a second keyword search was performed. Cytochrome P450 2C9, 2C19 and 2D6, Vitamin K epOxide Reductase Complex Subunit 1 (VKORC1) and UDP-Glucuronosyltransferase 1 Polypeptid A1 (UGT1A1) were also the focus of this evaluation. This search was extended to any test or orthogonal method applied for performance comparison with an NGS based application to increase the yield of findings. Thereby, the keywords next generation sequencing and VKORC1 or CYP2C9 or CYP2C19 or CYP2D6 or UGT1A1 and validity or validation were used to screen the publication platforms Pubmed and Pubmed Central. The same search as for NGS was performed for WES and WGS. Title and abstract were screened for suitability of the publication in terms of information on a performance evaluation of the according method involving the according gene of interest. The included publications were analyzed for the evaluation of analytical validation criteria recommended for NGS in current literature such as limit of detection, accuracy (positive percent agreement, negative percent agreement and positive predictive value), precision (reproducibility and repeatability), and analytical specificity (endogenous and exogenous interference, cross-reactions, cross-contamination).

Results

Analytical validation of NGS- based tests applying orthogonal FDA cleared or approved genetic tests

The present keyword search (Fig. 1) aimed to identify published analytical validation studies of NGS-applications with FDA cleared or approved orthogonal tests.

Fig. 1
figure 1

PRISMA 2020 scheme of the literature search for analytical performance studies of NGS-applications with FDA cleared or approved orthogonal tests in Pubmed and Pubmed Central

Full size image

FDA cleared or approved genetic tests used for keyword search were identified as PCR- or predominantly microarray-based methods for genotyping of drug metabolizing enzymes (Table 2). A keyword search applying the platform Pubmed did not result in any findings. An additional keyword search on the platform Pubmed Central resulted in a few publications addressing some of the cleared or approved nucleic acid-based tests. The keywords validation and validity resulted in overlapping publication lists. However, few suitable publications with regard to test comparison or validation were identified (Table 2) [22, 23].

Table 2 Test search results with the according keywords (trade name, gene(s), NGS application, validity or validation). All findings for second generation sequencing (SGS) including targeted NGS, WES and WGS and for third generation sequencing (TGS) were summed up excluding duplications. The number of according publications providing performance evaluations was included in brackets.
Full size table

None of the publications in this literature search provided a comparison of WES or WGS with the listed nucleic acid-based tests. Publications of comparison experiments with next generation sequencing were identified in the keyword search for the Invader UGT1A1 Molecular Assay (Third Wave Technologies Inc., Wisconsin, USA) [24] and the xTAG CYP2D6 Kit v3 (Luminex Molecular Diagnostics, Inc., Toronto, ON, Canada) [22, 23]. The published performance comparison with the Invader UGT1A1 Molecular Assay however, was carried out with the OpenArray pharmacogenomics panel, which comprised of 4 customized TaqMan® OpenArray Genotyping Plates. NGS was included in the validation of the OpenArray panel (Thermo Fisher Scientific, Waltham, MA, USA) but was not validated with the Invader UGT1A1 Molecular Assay [24]. Therefore, this publication also did not result in an appropriate finding according to the inclusion criteria for this review. Performance comparison of the Luminex xTAG CYP2D6, with second generation sequencing was detected in only one of the identified publications [22]. Here, Carvalho Henriques et al. provided only an evaluation of concordance for CYP2D6 and CYP2C19 variant detection in their cross-validation of a large variety of different techniques. These also included NGS represented by Ion Torrent™semiconductor sequencing on the basis of the Ion AmpliSeq Pharmacogenomics Panel (Thermo Fisher Scientific, Waltham, MA, USA). Therefore, the accuracy via positive percent agreement, negative percent agreement and positive predictive value and also evaluations in terms of further important validation criteria such as precision, limit of detection, and analytical specificity were not addressed.

Performance comparison of the Luminex xTAG CYP2D6 with a third-generation sequencing application was only identified in one publication [23]. Qiao et al. provide a concordance evaluation of the long-read SMRT sequencing of CYP2D6 on the Pacific Biosciences platform with the xTAG CYP2D6 Kit v3 (Luminex Corporation, TX, USA) and in terms of CNVs, the TaqMan® real-time qPCR Copy Number Assays (Applied Biosystems, Carlsbad, CA, USA). Furthermore, quality metrics and precision of the SMRT sequencing was evaluated in terms of intra- and inter-run reproducibility on the basis of triplicates. However, sample size was low and other aspects of accuracy such as positive percent agreement, negative percent agreement and positive predictive value and other criteria of analytical validation were not addressed [23].

Validation status of NGS based applications

In the second literature search described previously, about 99% of the publications screened by title and abstract were excluded, as they did not include information on sequencing of the pharmacogenes of interest or the use of NGS applications or any aspects of performance evaluation. Duplications due to overlapping results by applying the keywords validity or validation and due to published overlapping analyses including several of the pharmacogenes of interest were excluded in the screening phase. Further publications were excluded in the assessment for eligibility due to lacking focus on the NGS applications or the pharmacogenes of interest in the performance evaluations (Fig. 2).

Fig. 2
figure 2

Scheme of the literature search for analytical performance evaluations of NGS-applications in Pubmed and Pubmed Central

Full size image

In summary, of the 15 publications included (1 duplicate assessed for WGS and WES), 13 publications on second generation sequencing including also WES and WGS and two publications with a focus on third generation sequencing of CYP2D6 were identified with performance evaluations. They provided information to target at least one of the pharmacogenes of interest (Table 3). All of these publications provide concordance evaluations of the next generation sequencing application with other genotyping methods such as TaqMan-based genotyping assays, Sanger sequencing, Agena Bioscience ADME genotyping panels, array-based applications and custom PCR-based assays (Supplementary material 1). The third generation sequencing articles identified used the Pacific Biosciences platform [23, 25]. Evaluations focusing on second generation sequencing were mainly based on the Illumina platforms, for WES and WGS the HiSeq 2000 and/or HiSeq 2500 platforms were used [26,27,28]. For targeted second generation sequencing, the HiSeq platforms (HiSeq 1500, HiSeq 2000, HiSeq 2500, HiSeq 4000) [29,30,31,32, 16] the MiSeq, NextSeq 500 and MiniSeq system was reported [33,34,35].

Table 3 Publications identified in the second literature search reporting approaches of analytical validation or concordance evaluation of NGS-based tests
Full size table

Library preparation systems used for test evaluations on the Illumina platforms also differed. The Agilent SureSelect kits were used more often than other enrichment kits (Supplementary material 1). Other platforms were used when only a few variants or genes were analyzed. The PyroMark Q24 (QIAGEN GmbH, Venlo, Netherlands)), a pyrosequencing platform, was applied in a comparison of CYP2D6*10 identification and the Ion Chef instrument along with the Ion AmpliSeq Pharmacogenomics Panel was used in a cross-validation of several molecular genetic techniques for genotyping of CYP2D6 and CYP2C19 (Supplementary material 1) [36, 22]. NGS applications assessed in a majority of the analysed publications (93%), considered the pharmacogene CYP2D6 while CYP2C9, CYP2C19, VKORC1 and UGT1A1 were less often covered (Table 4). However, in one of the analyzed publications, the NGS application that covered CYP2D6 and UGT1A1 was an oncology sequencing assay and therefore not explicitly intended to be used for pharmacogenetic profiling (Supplementary material 1) [29].

Table 4 Overview of publications evaluating genetic tests based on second and third generation sequencing that include the according pharmacogenes of interest
Full size table

In addition to concordance evaluations, about 46.67% (7 of 15) of the analyzed publications provide further data on analytical performance criteria (Table 3). Accuracy assessments (33.3%, 5 of 15) were performed, these included overall accuracy [31] or in a majority of cases, as is recommended, accuracy in terms of the PPV along with positive percent agreement (also referred to as sensitivity) and negative percent agreement (referred to as specificity) (26.67%, 4 of 15) [29, 32, 35, 33] (Table 5). The limit of detection for variant types, such as SNVs and indels, is reported in one publication and was determined on the basis of detected variant allele fractions [29]. In 3 further publications only assessments in terms of minor allele frequency or fraction [31, 28] or detection rate [35] were performed without determining a limit of detection for the analyzed variant types (Supplementary material 2). Only one publication providing an evaluation of a targeted second generation sequencing panel appropriately addressed an evaluation of precision, in terms of intra-run and inter-run precision, that also provided the most comprehensive assessment in terms of FDA validation criteria such as also accuracy and further criteria such as reportable regions along with quality of the NGS metrics. However, analytical validation was based on only three reference samples [35]. Furthermore, one publication evaluating CYP2D6 SMRT third generation sequencing reported an assessment of precision in terms of triplicate intra-run sequencing of a validation sample and inter-run sequencing [23].

Table 5 Publications identified in the second literature search, providing evaluations of performance criteria
Full size table

In summary, few studies evaluate analytical performance criteria such as limit of detection, accuracy and precision of second generation sequencing panels and fewer still focus on third generation sequencing. Analytical specificity in terms of interference, cross-reactions and cross-contamination has not been assessed or provided by any of these published evaluations. Therefore, according to the information provided on the platforms Pubmed and Pubmed Central, none of the NGS- based tests has an analytical performance evaluation that has been executed according to the current recommendations in literature and of the FDA. Although, several studies compare WES and WGS with other molecular genetic techniques in terms of variant detection in pharmacogenes coding drug metabolizing enzymes, here also no articles were identified that carried out an analytical validation in terms of evaluating performance criteria as recommended for NGS-based IVDs or LDTs [26, 30, 27, 28].

Discussion

The present literature assessment shows that published studies on the analytical validation of NGS based pharmacogenetic tests concerning drug metabolizing enzymes are scarce. Specifically, this applies for WES, WGS and also third generation sequencing, as currently represented by Single Molecule Real Time sequencing using the PacBio (Pacific Biosciences of California, Inc, California, USA) and increasingly also Oxford Nanopore (Oxford Nanopore Technologies (ONT) Inc, Oxford, UK) platforms.

According to currently available, but non-binding, FDA recommendations, the evaluations identified are not sufficiently comprehensive. Furthermore, also, relevant analytical performance metrics according to the College of American Pathologists (CAP) guidance documents and the MM09 guideline of the Clinical and Laboratory Standards Institute (CLSI) were not addressed by the evaluated publications. In addition to the aforementioned analytical performance criteria recommended by the FDA, the CAP/ CLSI analytical performance metrics also include robustness, reportable range and reference interval. Metrics such as interference and cross-reactivity are listed in the CAP/CLSI MM09 test validation worksheet as IVD (not LDT) terminology only and cross-contamination is not addressed [37] (Supplementary material 3). Required analytical performance characteristics for IVDs in general, listed by the US Medicare, Medicaid, and Clinical Laboratory Improvement Amendments (CLIA) programs § 493.1253(c), comprise similar metrics to CAP/CLSI MM09 for IVDs, however not specified for NGS applications [38]. In the EU, required general analytical performance characteristics provided by the IVDR (Annex I chapter II 9.1 (a)) also include a larger scope and correspond to the CAP and CLIA criteria and cover also the specified FDA recommendations for NGS (Supplementary material 3). The performance metrics addressed by the evaluated publications, therefore do not meet CLIA and IVDR requirements as well. Such tests therefore are not suitable to be marketed and applied for clinical pharmacogenetic profiling in the USA or the EU. For an appropriate analytical validation of an NGS based qualitative test for pharmacogenetic profiling at least accuracy, precision, limit of detection and specificity should be assessed as recommended by the FDA guidance document “for Stakeholders and Food and Drug Administration Staff” [8] and as analyzed in this literature search. Furthermore, we suggest that also the reportable range and the range of outcomes expected in a normal population (normal interval) should be determined as recommended additionally by the CAP and the CLIA.

The most extensive performance evaluations were identified for targeted NGS panels that encompass a large variety of target genes, but without a direct focus on pharmacogenetic profiling. Here, Silver A et al. (2022) provided the best analytical validation in terms of the scope of performance criteria. The evaluation included accuracy and precision as recommended by the FDA and detection rates of targeted alleles, however did not include analytical specificity in terms of interference, cross-reactions and cross-contamination (Supplementary material 2). Furthermore, the reportable range was determined, which was however not a criterion of the present literature assessment. Ramudo-Cela L et al. (2020) evaluated accuracy of a targeted NGS panel according to FDA recommendations and partly precision in terms of reproducibility as sample triplicates were applied. The NGS panels however comprise a total of 430 (Silver A et al., 2022) and 389 (Ramudo-Cela L et al., 2020) genes and too large a panel size is not recommended as it can affect the efficiency of the laboratory, due to the complexity in interpreting the outcomes and the depth of coverage required [7]. Beaubier N et al. (2019) evaluated accuracy and LOD for different variant types as recommended by the FDA, however the intended use of the panel was to detect somatic alterations and microsatellite instability in solid tumors. It is therefore not validated for pharmacogenetic profiling although CYP2D6 and UGT1A1 were included in the panel. In the publication by Klein K et al. (2019), the lowest level of concordance with FDA recommendations was found with respect to the evaluation of performance criteria for a targeted NGS panel. Minor allele frequencies were addressed, however without determining detection limits and accuracy was evaluated as overall accuracy and not as recommended by the FDA via PPA, NPA and PPV. No further criteria were evaluated. Most of the publications assessed, include profiling of the highly polymorphic gene CYP2D6 which is also a challenging locus to analyze due to complex structural variation.

A more comprehensive analysis of complex genetic information to assess the pharmacogenetic profile associated with the drug metabolism of a patient can be provided by WES and WGS [10]. In our literature search two publications were identified evaluating performance of WES or WES and WGS in terms of also the according pharmacogenes of interest (Supplementary material 2) [33, 28]. Both publications mainly focus on concordance analyses with orthogonal methods and quality metrics such as call rates or depth of coverage and not on the FDA analytical validation criteria. However, Gulilat M et al. (2019) evaluated accuracy of WES only without PPV as recommended and Yang W et al. (2016) addressed minor allele fractions only without reporting the determination of detection limits according to FDA guidance. Recommendations providing guidance on the analytical validation of WGS in clinical context were published in recent years. They indicate that traditional approaches with regard to the evaluation of performance metrics for the complete NGS based assay alone is not sufficient [39].

Large-scale massive parallel sequencing applications such as WES and WGS do not target specific genes, sequence contexts, diseases or associated variant types per se. Therefore, here the aim of analytical validation is to evaluate and determine reliable metrics for a suitable performance with regard to the largest percentage of area analyzed to assure high quality of sequencing results overall. It is intended that variant calling over the target region is sensitive and precise and also allows for the determination of regions or bases that fail requirements for appropriate variant calls [5].

Metrics considering aspects of genome complexity such as sequence content and the variety of different variant types should also be assessed, as variability in the accuracy of variant calling is also context-driven. Diagnostic accuracy is often evaluated in terms of sensitivity (PPA) and specificity (NPA). However, due to the expected large quantity of true negatives in WGS, precision is recommended as a more helpful metric for accuracy than specificity (NPA). In general, also for WGS, the assessment and extent of metrics as published in the non-binding guidance document for NGS by the FDA is recommended and the number of samples is suggested to be adapted to variant type or the analyzed region [39]. In the present literature search, the only publication evaluating performance of WGS for pharmacogenetic profiling did not address analytical performance criteria including precision as recommended. Current studies evaluating NGS-based test performance for pharmacogenetic profiling mainly provide concordance assessments with orthogonal methods (Supplementary material 2).

Compared to targeted NGS panels analytical sensitivity for exome sequencing is lower as depth of coverage is not uniform or insufficient affecting also analytical specificity and variant calling. Thus, Sanger Sequencing is needed to complete content and identify false positive calls [40].

For the profiling of pharmacogenes, so far, short read WGS has been proposed to be a more suitable NGS application than WES as it provides a more comprehensive testing without increasing costs significantly [41]. With regard to further genome wide tests such as chromosomal microarrays the performance of WGS for CNV detection has been shown to be at least comparable [39]. However, in the current literature no appropriate analytical validation of WGS for pharmacogenomic profiling has been addressed. Linderman et al. (2014) provided an evaluation of the analytical performance of WES and WGS in general using five reference samples and technical replicates, by assessing the reproducibility (intra-run, inter-run, inter-mode and inter-machine), the concordance with microarrays and Sanger Sequencing and the sensitivity. Here, for WES and WGS the focus was on sensitive and precise as possible variant calling over the target region and thereby the determination of bases for which appropriate requirements were not met. Here, the performance analysis was based on ranges recommended by The American College of Medical Genetics and Genomics [5].

However, short read based NGS currently applied for WES or WGS can be prone to coverage bias in GC rich regions also due to a dependence on PCR for template amplification and therefore may miss hidden genetic variation in numerous genomic regions with high GC content. Therefore, disruptions of relevant genes with such GC rich regions or impairments of dosage-sensitive genes leading to dosage change may remain undetected in patients assumed to have a genetic disorder. Many structural variants and indels undetected by short read based NGS were detected by long read sequencing in recent studies [42,43,44].

However, the low throughput in the application of platforms such as Nanopore ONT and PacBio prohibits cost-effective deep sequencing without enrichment of the target sequence of interest [45]. Nevertheless, barcoding and multiplexing of samples has been reported to reduce per sample expenses in CYP2D6 SMRT sequencing and enable cost comparability to commercial genotyping assays. Moreover, full gene SMRT provides further CYP2D6 variant information such as e.g. the characterization of CNVs, sub allele resolution and novel tandem arrangement which may be convenient for routine clinical testing [23].

Also in long read sequencing PCR based enrichment can introduce phasing errors [46] and amplification bias and also leads to a loss of information on native modifications. CRISPR-Cas-based enrichment strategies such as those for Nanopore Cas9-targeted sequencing (nCATS) can prevent such bias and information loss. Still, a relatively high amount of native DNA is needed and limitations in the detection of some SNVs remain. For the performance of this application, however, it is assumed that identification of SNVs will progress with accelerating improvements of algorithms for base- and variant-calling in future and will also increase performance for mutation surveillance [47].

Furthermore, many recent publications indicate that these evolving techniques hold considerable promise for clinical application [48,49,50]. However, for third generation sequencing with regard to drug metabolizing enzymes only one study providing validation results and performance comparisons with currently FDA cleared or approved nucleic acid based tests was detected via our literature screening based on the keyword search. The study detected was however based on low numbers of tested samples. Still, it provides a promising outlook for third generation sequencing technologies to detect variation also in the complex CYP2D6 locus.

Currently, WES and WGS are performed based on second generation sequencing techniques as the emerging techniques in third generation sequencing are characterized by a high error rate. However, third generation sequencing provides a promising utility for the analysis of structurally complex genomic regions and may become a helpful tool for the assessment of the whole genome for clinical use [42, 51]. Thus, they are increasingly provided by commercial service laboratories, currently however for research evaluations only [52, 53].

A limitation of the present assessment is that other suitable keywords such as performance or concordance were not applied in this literature search. Therefore, several publications focusing on analytical validation in terms of analytical performance may have been missed. Another limitation is that NGS based diagnostic tests applied as clinical assays are mainly validated and applied in laboratories as LDT and documented data on analytical validation is not publicly available [6, 54]. In 2017, the FDA published a discussion paper on the oversight of LDTs. It addressed also an increase of transparency in terms of providing publicly available information on analytical and clinical validity of LDTs. However, a final guidance was not established [55]. Due to the new in vitro diagnostic regulation in the EU it will be mandatory for laboratories to provide appropriate performance evaluations to the regulatory authorities. The establishment of transparency in terms of publicly available information on the validity of laboratory developed and applied NGS technologies is not addressed by the new IVDR in the EU [3].

Conclusions

Clinical NGS and knowledge in pharmacogenomics are evolving rapidly. However, current studies evaluating NGS-based test performance for pharmacogenetic profiling mainly provide concordance assessments with orthogonal methods without considering further relevant analytical performance characteristics recommended by the FDA or required according to the CAP and CLIA guidance and in the EU according to the IVDR. For the use of next generation sequencing applications such as WES, WGS and also third generation sequencing for pharmacogenetic profiling in the clinical setting, a more comprehensive analytical performance analysis including at least the recommended performance characteristics is needed. However, it became clear that for publication of such assessments a standard in reporting on analytical validation of NGS based tests is not in place and may be increasingly necessary to facilitate the implementation of NGS tests in clinical use. Feasibility evaluations to extract clinically important pharmacogenetic information from sequencing data obtained by diagnostic testing for other clinical conditions suggest a potential to reuse relevant information for therapy management. Therefore, regulatory guidance to establish analytical validation requirements also for such approaches may be necessary.