FormalPara Key Points

Alignment between regulatory bodies and the Federal Joint Committee regarding the relevance of primary endpoints to patients in oncology clinical trials is required early on in clinical development in order to facilitate the generation of appropriate data.

Evaluation of additional benefit should be performed taking into account the condition and the disease stage. Endpoints in the three dimensions of mortality, morbidity and health-related quality of life should be selected and their relevance weighted carefully.

1 Introduction

Since the introduction of the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany in 2011, the Federal Joint Committee (FJC) is charged with the early benefit assessment (EBA) of medicines entering the German market [1]. The health technology assessment (HTA) process itself and the subsequent pricing of pharmaceuticals in Germany are important for reimbursement decisions in many European countries.

The aims of drug evaluation by the FJC and regulatory authorities clearly differ. While regulatory bodies focus on a balanced benefit–risk profile of a new medicine, the FJC assesses the additional benefit compared with the best available comparative treatment. In order to determine the additional benefit of a new medicine, the FJC re-examines all clinical data. Although the focus of the assessments by regulatory bodies (benefit/risk) and the FJC (additional benefit of a new medicine vs. best available treatment) differs, the underlying methods and standards of evidence-based medicine apply to both procedures.

However, early experience based on systematic reviews of primary endpoints [2] and all endpoints used in oncology drug assessments [3] indicate that certain endpoints and analyses commonly used to support regulatory approval are not necessarily considered as relevant to patients in the EBA decision process. Based on the results of these systematic analyses, we aim to reflect the FJC’s practice of addressing ‘patient-relevant benefit’ and compare it with the clinical assessments of the European Medicines Agency (EMA) and US Food and Drug Administration (FDA).

2 Regulatory View on Clinically Relevant Endpoints in Oncology

Regulatory bodies have issued guidance on clinically meaningful endpoints in oncology clinical trials. The FDA suggests that overall survival (OS) remains the gold standard for demonstration of clinical benefit [4]. Alternative endpoints in randomized controlled trials include time to progression (TTP) and progression-free survival (PFS), while in single-arm trials, response rates (overall response rate or complete response) may also be used (Fig. 1). Improvement in disease-related symptoms, as a more recently developed endpoint, is considered a clinical benefit and might be suitable for regulatory approval [4, 5]. The EMA largely supports the view of the FDA. Acceptable primary endpoints include cure rate, OS, PFS or disease-free survival (DFS) (Fig. 1) [6]. More recently, the EMA has issued a reflection paper on patient-reported outcomes (PROs) suggesting that PROs are an umbrella term encompassing also health-related quality of life (HRQoL). Those PROs are generally considered to carry add-on value over conventional efficacy and safety data from a licensing perspective [7, 8].

Fig. 1
figure 1

Comparison of acceptance of oncology endpoints in regulatory and benefit assessment. DFS disease-free survival, FJC Federal Joint Committee, EORTC QLQ European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire, HRQoL health-related quality of life, OS overall survival, PFS progression-free survival, TTP time to progression

Full size image

3 The FJC Framework on Patient-Relevant Endpoints in Oncology

The AMNOG is embedded in the German Social Code (SGB V §35a and §103b) [14]. According to this legal framework, the additional benefit over an appropriate comparative therapy may be based on the following three benefit categories: mortality, morbidity and HRQoL (Fig. 1) [15, 16]. The FJC rules of procedure provide some further guidance [16]:

  • Mortality: longer survival (e.g. OS).

  • Morbidity: improvement in the state of health (e.g. recovery from or abatement of the disease, long-term freedom from or reduction in symptoms), reduction of the duration of the disease, reduction in side-effects.

  • HRQoL: improvement in the quality of life.

Within available early benefit decisions, the FJC determines patient-relevant benefit according to those three dimensions. However, the findings of the FJC indicate that their view on patient-relevant endpoints largely deviates from the regulatory bodies’ view on clinically relevant endpoints [2]. Recent analyses from the German Institute for Quality and Efficiency in Health Care (IQWiG) state that only around half of the available outcomes for drugs in the EBA process have been included as patient-relevant endpoints in the institute’s assessments [17, 18].

4 EUnetHTA View on Patient-Relevant Endpoints

In response to controversial discussions regarding the use of endpoints in approval and HTA [2, 3, 8–11], the European Network for Health Technology Assessment (EUnetHTA) developed guidelines on relevant clinical endpoints for relative effectiveness assessment (REA). There is consensus that patient-relevant endpoints used in HTA should be a valid measure of clinical benefit due to treatment. The endpoints, which can be broadly categorized into mortality, morbidity and HRQoL, should describe how a patient feels, functions and survives. Along with mortality and morbidity, HRQoL is, therefore, one of the major REA endpoints [12, 13]. The EUnetHTA further emphasizes that the relevance of endpoints for REA depends on the disease, population, treatment and decision context [12]. Preference is clearly given to long-term or final endpoints, whenever possible [12, 13]. Notably, even if a trial is usually only powered for a primary endpoint, the additional benefit of a new medicine will be assessed in comparison to an adequate comparator on all endpoints relevant to the disease or its treatment [12, 13]. This lack of consideration of the hierarchy of endpoints is a hallmark of the FJC’s appraisals.

5 Benefit Category 1: Mortality—OS as Gold Standard

Regulatory authorities view OS as the gold standard for demonstrating clinical benefit of oncology medicines [4–6]. However, it can be confounded by subsequent therapies. Furthermore, survival analysis requires long-term follow-up and large sample sizes. In some oncological conditions, a long period of assessment would be required to reasonably analyze OS.

In a recent comparison of clinical trial endpoints for oncology medicines accepted by the EMA and considered as patient-relevant in the FJC benefit assessment, a high level of agreement between the two authorities with regard to endpoints covering the benefit category of mortality was found. OS was recognized as the preferred endpoint to form the basis for EBA decisions in oncology [2]. One reason for the dominance of mortality in supporting additional benefit may be the fact that unquestionable OS is regarded as the ultimate ‘true’ endpoint [19].

Two ways of indicating a therapeutic effect on survival for a therapy option are used: the absolute (median survival time) and the relative value [hazard ratio (HR)] [20]. Median survival time is the time point at which the percentage of ‘survivors’ reaches 50 %, which might require a long follow-up. The HR is calculated from the hazard rates, which quantify the likelihood that a patient will experience a ‘hazardous event’, such as disease progression or death, during a defined time interval, within each treatment arm [20]. Less time is needed to determine the HR, which makes it possible to provide new promising treatments to the patients as quickly as possible and which is of great importance in the case of life-threatening diseases with high medical need (e.g. metastatic melanoma). However, median OS differences in months between the treatment arms are primarily considered meaningful by the FJC; HRs are less well-accepted in the EBA decisions. Although information on absolute extension of survival is valuable, it may be very difficult to obtain under certain circumstances considering the chronic nature of certain oncological conditions or, for example, in indolent diseases with inherently long survival times. In this context, it is also important to consider that reaching a pre-defined efficacy boundary, typically based on HR, before the planned end of the study might result in a recommendation for crossover. This will lead to biased OS data after crossover and makes it impossible to determine absolute differences in treatment effects.

6 Benefit Category 2: Morbidity—The Field of Controversy

In the absence of OS data, primary endpoints such as cure rate, TTP, PFS and DFS are considered appropriate and clinically meaningful by regulatory authorities [5, 6]. These endpoints are by definition not confounded by subsequent treatments, and they usually require a smaller sample size and may be more rapidly assessed [4].

An analysis of patient-relevant endpoints in oncology proved the suitability of PFS of patients (i.e. survival of patients without disease worsening) as a morbidity-related, independent endpoint. In the context of disease complications, a significant improvement of PFS with acceptable side effects was seen as a valid, clinically and patient-relevant morbidity benefit, especially in maintenance treatment and palliative situations [3].

There is a lack of alignment between the EMA and the FJC with respect to morbidity endpoints. Well-established and clinically relevant morbidity endpoints used as the basis for regulatory approval are considered ‘not relevant to patients’ and are, therefore, mostly excluded by the FJC from their appraisals (Fig. 1) [2].

The IQWiG report ‘Validity of surrogate endpoints in oncology’ does consider OS as the ultimate ‘true’ endpoint, and all parameters of tumour response, including response rate and progression-related outcomes, are seen as surrogates for OS. Thus, morbidity is not a self-standing dimension and morbidity outcomes are only acceptable when validated against mortality [19]. The only true endpoint in morbidity that is frequently assessed is pain [21]. However, in particular, asymptomatic increase in tumour progression related to morbidity that is not yet symptomatic guides clinical behaviour mainly in early stages of the disease, but it is not considered patient relevant.

According to the FJC, the patient relevance of PFS is not proven. Imaging techniques to determine disease progression in accordance with the Response Evaluation Criteria In Solid Tumors (RECIST) were regarded by the FJC as not being relevant to patients. For example, in the assessment of axitinib for renal cell carcinoma, imaging methods were used to identify metastases in the spine, which was not considered as relevant to patients by the FJC. In the assessment of pertuzumab, the manufacturer suggested that at least a change of therapeutic regimen after disease progression should be considered relevant to patients; an argument that was not reflected in the FJC’s final appraisal [21]. Generally speaking, almost all asymptomatic primary endpoints are not considered to be patient relevant by the FJC, which does, to a certain extent, challenge the ethical basis of the conduct of those clinical trials.

The FJC splits PFS into a mortality and symptom component. However, as mortality is covered within OS and symptoms are often assessed by HRQoL instruments, the question remains what the FJC considers as a ‘true’ endpoint covering the morbidity dimension in oncology. For example, in the recent assessment of obinutuzumab, six morbidity-related endpoints are listed within the regulatory summary of product characteristics (SmPC) [22], none of which are accepted as being relevant to patients by the FJC [21].

Clearly, this controversy needs to be resolved and the rationale for different views on this dimension should undergo scientific (and public) discussion. The acceptability of PFS might also depend on the disease stage (e.g. adjuvant setting: PFS acceptable; metastatic setting: PFS alone insufficient) [18, 22]. Moreover, it has to be recognized that there are clinical circumstances under which conclusions about OS based on PFS results are questionable. For example, PFS may lead to longer OS simply because patients are able to tolerate treatment for a longer period of time and not because of superior efficacy.

7 Benefit Category 3: HRQoL—A Field to be Explored Further

So far, measures of HRQoL have not been used as primary efficacy endpoints for the regulatory approval of oncology drugs (Fig. 1) [4]. In addition, changes in signs or symptoms must clearly distinguish between disease symptoms and drug toxicity.

The recent EMA reflection paper suggests that PROs are an umbrella term covering single-dimensional and multi-dimensional measures of symptoms, HRQoL, health status, adherence to treatment and satisfaction with treatment. PROs include any outcome evaluated directly by the patients themselves and based on patients’ perception of a disease and its treatment(s). HRQoL is a multi-domain concept that represents patients’ general perceptions of the effect of illness and treatment on physical, psychological and social aspects of life. HRQoL instruments attempt to measure complex aspects of life that may potentially be modified by therapeutic interventions [7]. Whereas objective clinical measurements may not necessarily correlate to patients’ own feelings of well-being, such measurements may capture the personal and social context of the disease and treatment experience. According to the EMA, PROs carry add-on value over conventional efficacy and safety data from a licensing perspective [7]. Moreover, PROs are of increasing importance in the interpretation of survival gains, for example, in end-stage disease. In those situations, a key treatment goal is the relief of symptoms [8].

There is no standard approach regarding collection, analysis or interpretation of PRO data in clinical trials [23]. Careful thought must go into designing and implementing PRO measures in oncology trials in order to investigate a well-formulated, pre-defined hypothesis [7]. When assessing HRQoL, generic instruments should be considered and complemented with disease-specific measures if available [13].

A high level of acceptance of HRQoL instruments by the FJC, but poor contribution of HRQoL results to the final EBA decision, was found due to limited acceptance of the conducted analyses (Fig. 1) [2]. Methodological challenges, such as study-specific adaptions of questionnaires or an inadequate return rate, have prevented notable consideration in benefit assessments to date [24].

The results of HRQoL instruments were repeatedly disregarded by the FJC; whereas single items (e.g. evaluation of a specific symptom) were accepted in the morbidity benefit category [2].

Measures of patients’ preferences, usually referred to as utility measures, are currently only poorly reflected in the FJC’s benefit decisions.

Further study is required to address methodological issues related to HRQoL, for example:

  1. 1.

    A sound conceptual framework is needed to address the expected impact of a disease on patient symptoms [25]. This framework should also provide clarification regarding the different (albeit interrelated) concepts of PROs, HRQoL and utility measurement.

  2. 2.

    The relevance, reliability and responsiveness of the specific instrument/assessment, e.g. in orphan indications, should be determined.

  3. 3.

    Adequacy of study design, including ex ante HRQoL hypotheses and methods for appropriate handling of data collection (e.g. frequency and duration) as well as statistical methods regarding missing data, should be determined.

  4. 4.

    The rationale for the anticipated magnitude of effect should be addressed: statistical significance should correlate with clinical meaningfulness; confounding explanations should be addressed.

8 Further Aspect: Weighting of Benefit Categories According to Disease Indication

In an ‘ideal’ EBA scenario, additional benefit would be based on positive findings in all three dimensions: mortality, morbidity and HRQoL. However, the relative importance of benefit categories varies between tumour entities and disease stages. For example, in an aggressive cancer with a high rate of mortality, for example, melanoma (median OS of less than 1 year [26]), an increase in survival time is of the utmost relevance to the patient. In contrast, in a chronic but less life-threatening disease, such as advanced basal cell carcinoma, or in early, for example, neo-adjuvant, therapy settings, disease morbidity (e.g. delay in progression, pathological complete response) becomes more important. For chronic indications where no curative treatment is available, an improvement in morbidity or HRQoL can be of high importance to patients and be even more meaningful than OS [27].

Evaluation of additional benefit should, hence, be performed on a disease-specific basis because of the complex and heterogeneous nature of various conditions, and the three dimensions of mortality, morbidity and HRQoL should be weighted appropriately. To date, there has been no weighting of patient relevance of the various available endpoints by the FJC according to the respective course and stage of diseases.

9 Recommendations

The following recommendations should be considered in future EBAs to ensure adequate representation of patient-relevant endpoints across all three dimensions:

  1. 1.

    Agreement between regulatory bodies and the FJC on clinically meaningful endpoints that are relevant to patients and related benefit categories is needed early on in clinical development. This particularly applies to primary study endpoints.

  2. 2.

    The FJC should provide a comprehensive list of ‘true’ oncology endpoints covering the dimension of morbidity.

  3. 3.

    Clinical trial programmes should attempt to capture and consequently monitor endpoints related to as many benefit categories as possible.

  4. 4.

    Weighting of benefit categories should reflect disease-specific conditions and stage of disease (especially in complex indications like oncology).