Background

The Food and Drug Administration (FDA) has announced new mechanisms for regulatory approval of next generation sequencing (NGS) [1]. This includes a new pathway for approval of NGS tests for tumor profiling using the New York State Department of Health (NYSDOH) as a FDA third-party reviewer of in vitro diagnostics [2]. Central to this review is the fundamental tenet that NGS tests cover biomarkers with predictive value that change over time as clinical and scientific discoveries are made. As such, the FDA now recognizes three evidence levels that support the actionability and clinical utility of NGS tests [1]. Level 1 variants are defined by FDA as essential for the safe and effective use of a corresponding therapeutic. In seeking FDA approval, NGS tests may include level 1 evidence claims for a specific drug based on support for the analytical validity of the test for each specific biomarker and a clinical study that establishes either the link between the result of that test and patient outcomes or the clinical concordance to a previously approved companion diagnostic. Level 2 variants are defined by FDA as enabling physicians to use information about their patients in accordance with supporting clinical evidence, such as professional guidelines and/or peer-reviewed publications. Level 3 variants are defined by FDA as informational or used to direct patients toward clinical trials. Such claims are supported by analytical validation, principally through a representative approach when appropriate, and clinical or mechanistic rationale for inclusion in the panel, including peer-reviewed publications or in vitro preclinical models. While many groups have implemented other classification approaches for disease-biomarker-drug evidence associations, the FDA’s approach uniquely focuses on NGS and explicitly requires the analytical validity of the gene-variants tested.

Not unlike many other major cancer-focused medical academic centers [3,4,5,6,7,8,9,10], Roswell Park Comprehensive Cancer Center (Buffalo, NY) developed a precision oncology initiative that included NGS tests approved by the New York State Department of Health (NYSDOH). In 2016, an NGS pan-cancer comprehensive genomic profiling panel called OmniSeq Comprehensive® (OCP) (OmniSeq®, Buffalo, NY) was launched, based upon the Oncomine™ Comprehensive Assay by ThermoFisher Scientific (Carpinteria, CA). To evaluate the utility of the above described FDA variant classification guidance for oncologist treatment recommendations, we interpreted and applied the three-tier evidence schema to OCP test results from a large cohort of sequentially tested advanced cancer patients, and assessed agreement between NGS test targeted therapy recommendations and subsequent physician treatment recommendations for their patients.

Methods

The OCP test uses tumor tissue to detect all classes of somatic genomic alterations in 144 cancer-associated genes. As seen in Table 1, the DNA-Seq component of the test detects somatic mutations (single nucleotide variants, insertions and deletions) and copy number variants in both oncogenes and tumor suppressor genes, while the RNA-Seq component performs rearrangement (fusion) analysis in oncogenes. DNA mutational analysis requires a minimum depth of 457 reads and uses a hot spot coverage strategy to detect gain-of-function mutations in oncogenes, while copy number analysis detects high level amplification. DNA mutational analysis also detects loss-of-function mutations in tumor suppressor genes using a complete coding sequence coverage strategy, while copy number analysis detects homozygous deletions. The RNA analysis detects fusions. The OCP test is approved for clinical use by New York State Clinical Laboratory Evaluation Program (NYS CLEP), which requires orthogonal confirmation by secondary technologies for somatic mutations. A proprietary bioinformatics pipeline filters single nucleotide polymorphisms and identifies reportable variants, including variants of unknown therapeutic significance (VUTS), based on pathogenicity using multiple public genomic content sources such as COSMIC, 1000 Genomes Project, dbSNP, SIFT, PolyPhen, and ClinVar.

Table 1 Gene-variants tested by OmniSeq Comprehensive®
Full size table

Filtered, detected variants are submitted to a comprehensive knowledgebase of therapeutic associations, which determines whether or not there are therapeutic associations at the nucleotide, codon, exon, gene, or fusion level. A laboratory information engine with reporting rules specific to oncogenes and tumor suppressor genes to determine clinical significance in the final report. Variants in tumor suppressor genes must be pathogenic or deleterious by both SIFT and PolyPhen to be reported. While OCP does not sequence matching non-tumor tissue from tested patients, it is possible that germline mutations can be identified from tumor-only sequencing results without direct analysis of germline DNA. OCP reports detected mutations in genes prescribed by the American College of Medical Genetics and Genomics (ACMG) [11] as potentially hereditary, and directs physicians to further investigate by germline testing if clinically applicable. OCP test performance characteristics were analytically validated by OmniSeq Laboratories under the requirements of the Clinical Laboratory Improvement Amendments (CLIA) of 1988, and OmniSeq, Inc. is licensed by CLIA, College of American Pathologists (CAP), and the NYS CLEP to perform high-complexity molecular diagnostic testing. As such, OCP meets the analytical requirements put forth in the FDA variant classification guidelines for NGS testing. Additional details regarding OCP methodology, clinical validity, and performance characteristics can be found in the National Center for Biotechnology Information (NCBI) Genetic Testing Registry (https://www.ncbi.nlm.nih.gov/gtr/tests/552042/overview/).

Each gene-variant detected and reported by OCP tests previously performed between June 2017 to June 2017 (n = 657) was mapped to one or more levels of evidence based on interpretation of the FDA guidance for actionable variant classification, as follows: Level 1: variants listed on current FDA and/or European Medical Association (EMA) approved targeted therapy labels required for drug administration; Level 2: variants described in publicly available professional practice guidelines described as having evidence of response, resistance, or non-response to targeted therapeutics. Evidence sources to describe level 2 variants included professional practice guidelines established by the National Comprehensive Cancer Network (NCCN), and the European Society for Medical Oncology (ESMO). A complete list of level 1 and 2 therapeutic variant associations is shown in Table 2. Level 3: variants used as inclusion criteria or direct therapeutic targets of agents in active clinical development. Automated and manual review of investigational studies at https://clinicaltrials.gov was used for the identification of level 3 variants. As of this writing, there were 160 unique targeted agents, in use alone or in combination in over 350 recruiting precision medicine trials with OCP variants that act as direct therapeutic targets of the investigational agent, inclusion criteria, or both. Off-label use of variants listed on current FDA and/or EMA approved targeted therapy drug labels required for administration qualify were also defined as level 3 evidence since many targeted therapy trials are engaged in expanding existing indications to other tumor types.

Table 2 Level 1 and 2 evidence therapeutic variant associations tested by OmniSeq Comprehensive® (June 2017)
Full size table

Age, gender, stage of disease, and tumor type were retrieved from test requisition data. Medical records were reviewed for patients with at least 60 days of post-test follow up time available in order to collect treatment status at the time of test order and physician treatment recommendations made for patients within 120 days after NGS results became available. Physicians’ post-test patient treatment recommendations were classified as targeted therapy, precision medicine clinical trial, immunotherapy, hormonal therapy, chemotherapy/radiation, surgery, transplant, or non-therapeutic (hospice, surveillance, or palliative care), and were compared to NGS report recommendations for agreement.

Results

Patients

Patients with 29 tumor types (Table 3) were tested, with breast cancer, colorectal carcinoma, lung carcinoma, melanoma, ovarian carcinoma, prostate cancer, and sarcoma being most common with each representing 5% or more of the total tests. Median age of all patients tested (64) and gender (55% males, 45% females) was within expectations for a cancer therapeutic management-focused test. Most patients had advanced (Stage III/IV) disease (540/657; 88%), but for a patient subset, stage was not reported when the NGS test was ordered (77/657; 12%). The majority of tests (76%; 497/657) were for patients currently undergoing treatment and/or had at least one prior treatment when the NGS test was ordered. Repeat testing was mostly restricted to lung cancer patients with resistance to prior targeted therapies.

Table 3 Tested patient characteristics (n = 646)
Full size table

NGS test results and physician treatment recommendations by FDA evidence level

A total of 2777 genomic alterations were identified in 657 tests with an average of 4.2 mutations per test. Frequently identified mutations were level 3 (1532; 55%), followed by VUTS (1045; 38%), level 2 (107; 4%), and level 1 (93; 3%).

Level 1 variants

A total of 92 variants with companion diagnostic level 1 supporting evidence as the most actionable result were detected in 14% (89/657) of tests as the most actionable evidence across 5 tumor types (Table 4). For the majority of these tests (94%; 84/89), a single genomic alteration was identified, with and no more than two level 1 variants reported in the remaining 5 tests. All tests with a level 1 variant had at least one other level 2 or level 3 alteration. The most frequent level 1 variants detected were identified in KRAS (colorectal cancer), EGFR (lung cancer), and BRAF (melanoma). Additional tumor types with at least one level 1 variant detected included breast and ovarian cancer.

Table 4 Frequency of detected variants by FDA level of supporting evidence (as of June 2017)
Full size table

The most common physician treatment recommendation for tests with level 1 evidence variant results was targeted therapy for (26/89; 29%) of tests (48% excluding mutations with contraindications: KRAS, NRAS), followed by chemotherapy/radiation for 23%. The majority of these patients initiated chemotherapy/radiation prior to the test being performed with the presumed intention of planning for future treatment. Immunotherapy was a frequent recommendation (14/89; 16%), related to lack of EGFR or ALK alterations in NSCLC. Non-therapeutic recommendations (hospice or palliative care) were infrequent when variants with level 1 evidence were detected (2% of tests). Clinical trial recommendations were also uncommon (5/89; 6%) (Fig. 1).

Fig. 1
figure 1

Physician treatment recommendations by the highest level of supporting clinical evidence to treat by targeted therapy for each test. Physician treatment recommendations for targeted therapies were most frequent for patient tests detecting variants supported by level 1 companion diagnostic evidence for targeted therapy (26/89; 29%), followed by tests with variants supported by level 2 practice guideline evidence as the most actionable result (11/100; 11%), and tests with variants supported by level 3 clinical trial/off-label evidence as the most actionable result (15/419; 4%). Recommendations for chemotherapy and/or radiation were more common for tests with variants supported by level 3 evidence (193/419; 46%) than for patient test results with level 2 (32/100; 32%) or level 1 (23/89; 26%) evidence. Recommendations for clinical trials were infrequent for test results across all 3 evidence levels for patients with companion diagnostic level 1 (5/89; 6%), level 2 (2/100; 2%), and level 3 (28/419; 7%) evidence. Recommendations for immunotherapy were relatively frequent for tests with targeted therapy level 1 (14/89; 16%), level 2 (25/100; 25%) or level 3 (49/419; 12%) supporting evidence

Full size image

Level 2 variants

A total of 109 variants with level 2 professional practice guideline supporting evidence for targeted therapy as the most actionable findings were detected in 100/657 (15%) of tests across five tumors types (colorectal carcinoma, lung cancer, melanoma, sarcoma, thyroid carcinoma). Similar to tests with a level 1 results, the majority of these tests (95/100; 97%) documented a single level 2 variant, and no more than two level 2 variants were reported for any given test. All tests with a level 2 variant also harbored at least one level 3 alteration. The most frequent level 2 variants were KRAS mutations in lung cancer. The second most frequent level 2 results encompassed atypical activating EGFR mutations in lung cancer patients, supporting use of EGFR inhibitors for certain alterations. The remaining level 2 variant results were distinct from level 1 results, and included a variety of alterations, all at a prevalence < 5% for that particular tumor type or a single case result (Table 4).

The most frequent physician treatment recommendation in the presence of level 2 evidence was chemotherapy/radiation (32/100; 32%), followed by immunotherapy for 25/100 tests (25%). Targeted therapy was a less frequent recommendation (11/98; 11%) with non-therapeutic recommendations being slightly more frequent (14/100; 14%). Clinical trial recommendations were also uncommon in this group (2/100; 2%) (Fig. 1).

Level 3 variants

There were 419 tests (65%) with level 3 genomic alterations as the most actionable variants. These results supported potential enrollment in either precision medicine clinical trials or use of off-label therapy, and were the most common result overall: mutations (1532/2777; 55%), tests (579/657; 88%), and all genes tested (67/144; 47%). Genes implicated by a level 3 mutation in at least 5% of all tests included TP53, ATM, CDKN2A, BRCA2, PTEN, BRCA1, PIK3CA, TSC2, KDR, NF1, MET, RB1, and PTCH1 in descending order of prevalence (Table 5).

Table 5 Level 3 variant frequency by gene and number of tests
Full size table

For the 419 tests for which level 3 variants were identified as the most actionable, the rate of clinical trial treatment recommendations by physicians was 7% (28/419). Commonly recommended in this group was chemotherapy/radiation (193/419; 46%), followed by immunotherapy (49/419; 11%). Targeted therapy recommendations were uncommon (15/419; 4%). Non-therapeutic options were recommended to 10% (41/419) of the cases belonging to this group, and not remarkably different from patients with a level 2 result (14%) (Fig. 1).

Overall, a total of 111 mutations with off-label indications were identified in 90 tests with no level 1 or 2 evidence. The majority of these off-label indications were BRCA1 or BRCA2 mutations (90/111; 81%), which were documented across 20 different tumor types. Less frequent mutations linked to off-label indications included activating single nucleotide variants or indels (insertions/deletions) in ERBB2 (HER-2), KIT, BRAF, and RET, as well as MET copy number gain or exon 14 skip. Off-label targeted therapy recommendations were infrequent (11/111; 10%).

VUTS

For a subset of tests (49/657; 7%), no variants were classified as having clinical significance based on FDA variant classification guidance. It is worthy to note that this group had the highest recommendation rates for chemotherapy/radiation (26/49; 53%), as well as and non-therapeutic options (8/49; 16%) compared to tests with variants supported by level 1, 2 or 3 evidence. Targeted therapy was never recommended in this group.

Tumor type-specific results

Of 29 tumor types tested, 24 did not harbor level 1 or 2 variants. These included mesotheliomas, sarcomas, bladder, brain, cervical, endocrine, endometrial, gastroesophageal, eye, genital, gallbladder, head and neck, kidney, renal pelvis, liver, bile duct, neuroendocrine, non-melanoma skin, pancreatic, prostate, small bowel, stomach, testicular, thymic, thyroid, and unknown primary cancers. Most of these tumor types also did not have biomarker-directed indications approved for targeted therapy, either on label or in professional practice guidelines. None of the 34 prostate cancers tested expressed a level 1 or 2 alteration. Of the 24 tumor types with no actionable level 1 or 2 variants, only 2 (gastroesophageal carcinomas and gastrointestinal stromal tumors) harbored mutations that met the list of genomic variants identified as therapeutic association with a high level of evidence (Table 4).

The remaining 5 tumor types with at least one level 1 or 2 variant included melanoma, breast, colorectal, lung, and ovarian cancers accounted for 61% (402/657) of the total tests performed, and do have biomarker-directed indications for targeted therapy, accurately reflecting the prevalence of these tumors and their current position in the precision medicine hierarchy per oncologist test usage.

Discussion

This study demonstrates the application of FDA-guidance for NGS variant classification in oncology therapeutic management, and compares the test results to treatment recommendations the way healthcare providers typically do (i.e., retrospectively) to obtain reimbursements. In particular, this study provides evidence from a current clinical practice where physicians formulate treatment recommendations based on a cancer-focused comprehensive NGS assay in the context of new guidelines from FDA. One conclusion from this study is that physicians do not recommend (or at least do not document recommending) clinical trials as often as they are indicated by NGS testing. We recognize that capturing treating physicians clinical trial recommendations solely by medical record review may generate an underestimation of the actual physician intent (i.e. patient ineligibility based on pre-screening evaluation) or patient-physician interaction in which the highest level of evidence is associated with an investigational study (level 3 evidence). In the group of tests with no on-label indications for targeted therapy, chemotherapy/radiation were the most frequent recommendations, suggesting that traditional approaches to cancer are still highly regarded by both oncologists and patients, or at least are still regarded as the most viable option given all the complex factors that contribute to decision making. This study also suggests that physicians are using NGS results appropriately to recommend targeted therapy. Targeted therapy was recommended for 48% of tests with level 1 evidence, 11% of tests with level 2 evidence, 3.6% of tests with level 3 evidence, and never in the presence of VUTS. This may have been positively influenced by the fact that testing and recommendations were performed in a NCI-designated comprehensive cancer center with access to a molecular tumor board, which can aid in treatment decision recommendations.

The majority of tumor types tested in this study have no level 1 or 2 therapeutic drug associations, implying that maximal benefit could originate from either a clinical trial or an off-label recommendation, both of which were infrequently recommended by treating physician, at least during the data collection period in this study. There is doubt that following such patients in a registry will provide any benefit when physicians do not strongly support enrollment in clinical trials but generally recommend chemotherapy/radiation. One factor that may offset such a perceived lack of benefit from NGS testing for targeted therapy is immunotherapy. Immunotherapy was recommended by physicians in 16% of tests with level 1 evidence, 25% of tests with level 2 evidence, 12% of tests with level 3 evidence, and 6% of tests documenting VUTS. This suggests that physicians are often using NGS results to exclude targeted therapy prior to recommending immunotherapy.

Many centers in the US are implementing clinical oncology pathways and collecting evidence as we did, which may be more accurate and representative than the current approach of multiple laboratories. Such pathways should aim at aligning care to national guidelines or, when appropriate, define the rationale for deviation from guidelines [12]. Another key objective for these pathways should be to support clinical trials, and to identify the barriers to enrollment. Some pathway programs require review of relevant clinical trials as the first treatment choice before allowing selection of “standard” therapies [13]. Pathway systems also provide a registry that identifies patients starting a new cancer treatment, what that treatment is, if the patient accepted to join a clinical trial, and how treatment compares to national guidelines. Pathway systems may provide the basis for proper identification of patients eligible for NGS and related clinical trials. Databases generated in the context of these pathways may also provide a robust foundation for the required National Institute of Health Genetic Testing Registry (NIH GTR) as per the new FDA guidelines for NGS.

There are multiple limitations to the present study, many of which related to the fact that it was retrospective and involved no direct interaction with physicians other than test ordering, test reporting, or molecular tumor board review. Recommendations were captured from clinical documentation which may be incomplete (e.g. not specifically documenting that a patient with is not eligible for clinical trial due to exclusion criteria) and may not record past discussions with ordering physicians. This may partially explain the low documented rate of recommendations to clinical trials, which may not fully represent the actual clinical practice. Eligibility for a clinical trial often includes multiple factors beyond molecular parameters, such as number and types of prior therapies, comorbidities, etc. Physician recommendations were captured within 120 days post-NGS test for patients that had at least 60 days of follow up time. Several clinical trials, such as the NCI-MATCH, require that patients have explored standard therapeutic options first and recommendation for a clinical trial may occur much later in a patient’s treatment course, often after multiple episodes of care.

Conclusions

NGS is a well-established technology, yet it lacks standardized regulatory approval. The majority of tumors do not have on-label, variant-directed targeted therapy indications, however, our data show that physicians generally understand NGS results and apply them appropriately, and that the FDA has provided valid guidance a pathway for medical policy coverage for at least a subset of tumor types. As precision medicine advances toward regulatory approval in the standard of care setting, coverage by both the Centers for Medicaid and Medicare (CMS) and other payers will require a consistent reproducible method of presenting evidence.