Opinion statement
Treatment guidelines for colorectal cancer (CRC) are primarily based on the results of randomized clinical trials (RCTs), the gold standard methodology to evaluate safety and efficacy of oncological treatments. However, generalizability of trial results is often limited due to stringent eligibility criteria, underrepresentation of specific populations, and more heterogeneity in clinical practice. This may result in an efficacy-effectiveness gap and uncertainty regarding meaningful benefit versus treatment harm. Meanwhile, conduct of traditional RCTs has become increasingly challenging due to identification of a growing number of (small) molecular subtypes. These challenges—combined with the digitalization of health records—have led to growing interest in use of real-world data (RWD) to complement evidence from RCTs. RWD is used to evaluate epidemiological trends, quality of care, treatment effectiveness, long-term (rare) safety, and quality of life (QoL) measures. In addition, RWD is increasingly considered in decision-making by clinicians, regulators, and payers. In this narrative review, we elaborate on these applications in CRC, and provide illustrative examples. As long as the quality of RWD is safeguarded, ongoing developments, such as common data models, federated learning, and predictive modelling, will further unfold its potential. First, whenever possible, we recommend conducting pragmatic trials, such as registry-based RCTs, to optimize generalizability and answer clinical questions that are not addressed in registrational trials. Second, we argue that marketing approval should be conditional for patients who would have been ineligible for the registrational trial, awaiting planned (non) randomized evaluation of outcomes in the real world. Third, high-quality effectiveness results should be incorporated in treatment guidelines to aid in patient counseling. We believe that a coordinated effort from all stakeholders is essential to improve the quality of RWD, create a learning healthcare system with optimal use of trials and real-world evidence (RWE), and ultimately ensure personalized care for every CRC patient.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Most novel therapies become available following a successful prospective phase III RCT, the standard approach to assess treatment efficacy and safety. Although the randomized design optimizes internal validity, generalizability of oncological RCT results can be limited [1]. Most landmark trials use stringent eligibility criteria excluding a large portion of patients who will be treated in clinical practice, such as patients with multiple comorbidities, brain metastases, or poor performance status. Furthermore, elderly patients, and ethnic and racial minorities are found to be underrepresented in CRC trials [2, 3•]. Besides differences in patient populations, patients in trials receive more attention and are treated by specialized doctors in academic centers. Meanwhile,conduct of traditional RCTs to establish drug efficacy has become increasingly challenging due to identification of a growing number of predictive molecular subtypes. Trial enrollment, for instance, is challenging in rare populations such as NTRK fusion positive CRC (Fig. 1). Hence, attempts to accelerate patient access to novel drugs in case of unmet clinical need have led to authorization of therapies based on single-arm trials and surrogate end points such as response rate [4, 5]. It is recognized that all these challenges result in uncertainty regarding meaningful benefit opposed to treatment harm and societal cost once novel treatments are implemented in clinical practice. Hence, catalyzed by the digitization of healthcare, the use of RWE to complement trials has gained much interest in the oncology community.
RWE has been defined as evidence derived from analysis of RWD collected through the routine course of clinical care from a variety of sources other than traditional trials [6]. RWD on CRC is increasingly being collected in large-scale databases and registries. These provide opportunity for large population-based studies and pragmatic trials such as the registry-based RCT and studies that employ the trials-within-cohorts (TwiCs) design [7, 8]. This year, the European Organisation for Research and Treatment of Cancer (EORTC) published their position on the role of these designs in clinical cancer research [9••]. Such studies seem more representative of clinical practice due to inclusion of larger and more heterogeneous populations. Conversely, methodologic pitfalls inherent to use of RWD result in lack of trust and hesitance to base decisions solely on RWE [10]. Regulatory bodies like the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) are actively working towards further establishment of the value of RWE in supporting regulatory decision-making across all stages of drug development [11, 12].
By utilizing high-quality RWD, we believe it to be possible to learn from every patient with CRC in order to provide precision medicine to our future patients. Since RWE has globally gained attention from scientists, industry, payers, and regulators in recent years, this narrative review will provide an insight in its contributions to CRC treatment. In addition, we discuss remaining barriers and future perspectives to unlock RWE’s full potential, focusing on the medical oncology perspective.
Adapted from Punt CJA, Koopman M, and Vermeulen L. Nat Rev Clin Oncol 14, 235–246 (2017) [13].*Limited to FDA- and/or EMA-approved treatments. MSI, microsatellite instability; mut, mutant; wt, wild-type.
Landscape of molecularly targeted treatments* for metastatic CRC.
Using RWD for treatment effect evaluation
Trends in population outcomes
The first cancer registries were once developed to study cancer incidence and survival and are still used for this purpose today. The identification of high-risk populations, regional disparities, and potential risk factors has been crucial for early detection and prevention of CRC. Healthcare policy decisions and initiatives are also informed by population outcome trends. For instance, in the 1990s, prognosis of CRC patients in Denmark was found to be inferior to neighboring countries. These findings led to initiation of national cancer plans and a subsequent increase in short- and long-term survival, closing the identified gap [14, 15].
Naturally, the goal of development of new oncological therapies is to improve survival while maintaining quality of life. Many population-based studies in Europe and the United States (US) have established improved survival rates for patients with CRC over the last decades [16,17,18,19]. For metastatic CRC (mCRC), median OS (mOS) in RCTs on first-line systemic treatment has almost doubled now exceeding 30 months [20, 21]. RWD on mCRC from the US Surveillance, Epidemiology and End Results (SEER) registry confirmed a meaningful increase in mOS from 12 months in 1986 to 21 months in 2015 [22•]. As the RWD availability in the SEER registry did not allow more detailed evaluation, the researchers recently conducted an additional single-center analysis and suggested increased application of liver metastasis resection, use of immunotherapy, and use of third-line chemotherapy to be the drivers behind this upward survival trend [23]. It is important to recognize that survival trends are not necessarily attributable to the accumulative effect of treatment advances alone. Mortality rates are greatly influenced by incidence rates [24], and survival may be affected by lead time bias due to earlier diagnosis after implementation of population screening and intensive follow-up programs. Also, improved diagnostic imaging leads to stage migration which influences stage stratified survival rates [25].
Evaluation of treatment effectiveness and safety
Treatment effect can be more thoroughly evaluated using RWD sources with detailed information. While efficacy describes treatment performance in an ideal setting such as an RCT, effectiveness refers to performance in the real-world setting [26••]. The application of strict eligibility criteria in trials has led to study populations that do not resemble CRC patients in clinical practice, thereby limiting generalizability of results [3, 27]. Outcomes in systemic treatment trials are regularly found to be superior to outcomes of systemic treatments in the real world, resulting in less absolute benefit and higher levels of toxicity [28,29,30,31]. Regulatory bodies like the FDA and EMA are not responsible for ensuring that new therapies provide meaningful benefit(s), but rather that they are safe and not inferior to the standard of care. Both the American Society of Clinical Oncology (ASCO) and European Society of Medical Oncology (ESMO) have developed frameworks to assess the value of cancer therapies [32, 33]. Nevertheless, international consensus is lacking, and every country applies its own reimbursement policies [34]. Given the rapid rise of healthcare expenditure in oncology, and the realization that novel drugs do not always provide meaningful benefit [5, 35, 36], post-approval benefit-risk (re-)assessment is warranted and has led to interest in using high-quality RWD for health technology assessments (HTA) [37]. Moreover, RWD can be used to fill some of the post-registration evidence gaps with which clinicians are faced.
For instance, encorafenib-cetuximab was recently approved for pretreated patients with BRAFV600E mutated mCRC following the results of the BEACON trial, which demonstrated a survival benefit of 3.4 months with a mOS of 9.3 months [38]. International treatment guidelines have since included this targeted treatment. However, despite application of strict eligibility criteria in the trial, the guideline recommendation is generalized to all patients with pretreated BRAFV600E mutated mCRC, and does not elaborate on the uncertainty of benefit in patients who were not represented in the BEACON trial [39, 40]. Boccacino and colleagues found that patients treated with encorafenib-cetuximab in an Italian nominal use program had approximately 2 months shorter median OS [41]. As this nominal use program applied eligibility criteria closely resembling those of BEACON, an additional population-based study was conducted in the Netherlands which discovered that over a third of all patients treated with encorafenib-cetuximab in routine clinical care would have been ineligible for the BEACON trial [42]. These ineligible patients demonstrated significantly inferior mOS of only 6 months. Patients with a poor performance status (WHO ≥ 2) and/or symptomatic brain metastases had such a short survival time that the likelihood of meaningful benefit was deemed negligible.
We recognize that the current restrictive design of RCTs may not represent the entire patient population in which the findings will be applicable. Therefore, we agree with guideline developers’ decision to initially generalize treatment recommendations beyond the landmark trial population. However, to avoid futile or possibly even harmful treatment, we advocate that (non) randomized studies using high-quality RWD should be conducted by default to establish or refute treatment effect in populations for whom RCT evidence was not provided. This can refine selection of treatment-eligible patients to reach the eventual goal of personalized medicine. Moreover, as the efficacy-effectiveness gap is highly relevant for patient counseling, we argue that high-quality population-based effectiveness results should be incorporated in treatment guidelines [26, 43•].
RWD can also provide information on a treatment outcome or safety issue that was not assessed in the pivotal trial, including (rare) long-term safety or the understudied but highly relevant QoL measures [33, 44, 45]. This information contributes to ongoing drug safety surveillance and informs benefit-risk assessments. For example, the RECOURSE trial demonstrated a modest mOS benefit for trifluridine/tipiracil of 1.8 months compared to placebo, which led to approval and recommendation in international guidelines [36, 39]. As QoL was not assessed, a prospective evaluation of QoL and OS was performed in patients treated with trifluridine/tipiracil in routine practice using data of the Prospective Dutch Colorectal Cancer (PLCRC) cohort [46], and in a population equal to the RECOURSE trial [47, 48]. Both studies found QoL to be maintained during treatment, thus supporting trifluridine/tipiracil use in clinical practice. More recently, following trial evidence of efficacy and safety of the oral fluoropyrimidine S-1 in Western patients [49,50,51], additional descriptive RWE on long-term safety and cardiotoxicity recurrence supported EMA approval [52,53,54]. The ESMO guideline now recommends switching to S-1 in patients with mCRC who experience hand-foot syndrome or cardiovascular toxicity while being treated with capecitabine or 5-FU [39, 55].
Comparative effectiveness research
Ideally, causal questions are answered in an RCT. This methodologic design ensures balanced patient groups with respect to both known and unknown risk factors and therefore provides the least biased evidence regarding treatment effect. RWD is increasingly used to assess causal questions, more commonly referred to as comparative effectiveness research (CER). Limitations of CER are well described and include missing data, misclassification, confounding, selection, immortal time, and treatment indication bias [56]. Treatment selection in clinical care is influenced by many characteristics including patient and physician preferences resulting in imbalanced treatment groups [57]. Advanced statistical methods are developed to correct for bias in CER, such as propensity score matching [58, 59]. Yet only variables that are measured and available for analysis can be used for such methods, which leaves the potential risk of residual confounding. There are different scenarios in which CER can be applied (Table 1). A previous publication [60••] has thoroughly described two examples of CER within the adjuvant CRC treatment setting with misleading results [61, 62]. In these cases, prior RCTs provided no evidence of treatment efficacy; however, CER performed in similar populations did suggest a treatment effect. Given that it is highly implausible that a treatment is ineffective under ideal circumstances but effective in clinical practice, CER does not provide valuable evidence in this scenario.
As discussed previously, when a landmark RCT has provided evidence of treatment efficacy, questions remain regarding treatment effect in the underreported and the trial-ineligible patient population. Pragmatic trials can be used to answer these questions; however, randomization is only considered ethical in the case of equipoise, i.e., the existence of genuine uncertainty regarding the superiority of one treatment over the other. Since treatment efficacy is most likely not limited to the trial-eligible population, performing a post-marketing pragmatic trial may be considered unethical. In this scenario, carefully designed and analyzed CER could help establish or refute treatment effectiveness in subgroups for whom RCT evidence was not provided. This can refine selection of treatment-eligible patients and reach the eventual goal of personalized medicine. Population-based CER may also inform on the overall value (benefit versus harm) of a new treatment option in clinical practice. It must be recognized that RWD are often sourced from electronic health records (eHRs) with unstandardized data. Since relevant variables—such as the experienced level of toxicity or patient performance status—are not always documented, available RWD may be of insufficient quality to yield actionable RWE.
Besides the setting in which RCT evidence is already available, CER is also performed while awaiting RCT results (Table 1). For instance, the indication for primary tumor resection (PTR) in patients with synchronous mCRC and an asymptomatic primary tumor has long been a topic of debate. Prospective evaluation was complicated due to poor acceptance of randomization by both patients and clinicians, and for a long time the only available evidence was provided by retrospective (pooled) analyses of RCTs suggesting improved survival with upfront PTR. Two propensity score–adjusted observational studies, each with a sample size greater than 10,000, were published, yet with contradictory results [63, 64]. Hence, the final answer had to be provided by prospective RCTs which have since confirmed no superiority of upfront PTR over chemotherapy alone [65, 66]. One could argue that CER results in this setting might decrease equipoise and endanger the feasibility of ongoing RCTs. Therefore, we must emphasize that quality of CER should be critically assessed, and its limitations should be acknowledged when interpreting results. Nevertheless, when effect sizes are large, and the risk of residual confounding is considered limited, CER could provide valuable and timely evidence in this scenario.
The last scenario in which CER can be conducted is the setting in which a traditional RCT is not considered ethical or feasible. For instance, when there is insufficient equipoise, or when requiring sufficient sample size or follow-up is unfeasible [67]. Randomization should, however, remain the gold standard to address causality. Hence, whenever possible, we recommend conducting pragmatic trials, such as registry-based RCTs, to optimize generalizability and answer clinical questions that are not addressed in registrational trials, e.g., optimal dosage or treatment sequence. These are recognized as efficient and cost-effective tools that combine the power of prospective randomization with the strengths of large-scale clinical registries. Such RCTs are and have been successfully conducted in CRC, examples being the RECTAL-BOOST trial in patients with locally advanced rectal cancer [68] and the MEDOCC-CrEATE trial in stage II colon cancer [69].
Using RWD for precision oncology
mCRC is a heterogenous disease characterized by a fast-increasing number of distinct molecular subgroups with different prognosis and response to treatment (Fig. 1). As new drugs are being developed for rare genetic subpopulations such as patients with NTRK fusions, RET fusions, KRASG12C mutation, or ERBB2 amplified tumors, conduct of phase III RCTs has become increasingly challenging. To address unmet clinical needs, accelerated and conditional marketing approval has been introduced based on single-arm trials, pan-tumor indications, and/or surrogate endpoints. Recently, Schroder et al. successfully replicated a control arm from the IMBLAZE370 trial in mCRC using RWD, suggesting the feasibility of matched comparisons with external controls [70]. RWD is increasingly provided to regulatory bodies as context for interpretation of single-arm phase II studies [71, 72]. In 2018, Overman et al. demonstrated an encouraging 1-year OS rate of 85% for ipilimumab-nivolumab combination treatment in pretreated dMMR mCRC patients [73]. Although these single-arm results led to FDA approval, EMA approval was delayed due to lack of a control arm. RWD from the French AGEO study, the Dutch PLCRC cohort, and the US Flatiron database were analyzed demonstrating inferior survival with systemic chemotherapy [74, 75]. It is unclear from public records whether these additional data supported the regulatory approval of ipilimumab-nivolumab [76]; however, data recently presented at the ESMO annual conference demonstrates that EMA considered RWD a supportive source of efficacy- and safety-related evidence in 20% of oncology targeted drug indications from 2018 to 2022 [77].
Post-approval, RWD can be used to identify predictive biomarkers for treatment response. It was a small retrospective study which first suggested KRAS to be a negative predictive marker for cetuximab efficacy in mCRC [78], a finding that ultimately led to restriction of anti-EGFR therapy to patients with KRAS wild-type mCRC. Furthermore, a RWD discovery cohort, including whole-genome sequencing data, identified KRASG12 mutations as a potential predictive biomarker for trifluridine/tipiracil resistance. Subsequently, this exploratory finding was validated in both a large real-world cohort, and in the population treated within the RECOURSE trial [79•].
Using RWD for patient counseling
As survival results from clinical trials are not translatable to all patients in clinical practice [26••], population-based RWD currently provides more reliable estimates for subgroups of patients with comparable prognostic characteristics. Since patients prefer to discuss realistic scenarios, including best-, typical-, and worst-case median survival times [80], Hamers and colleagues recently evaluated survival scenarios for various treatment subgroups using data of over 27,000 patients with mCRC from the Netherlands Cancer Registry (NCR) [43•]. It must be emphasized that such RWE is not intended to be used to inform treatment decisions but rather to estimate patient outcomes given prevailing treatment choices. These estimations can, however, inform further care and advanced care planning, and empower patients to make informed decisions [81•].
With the goal to provide precision medicine to every patient, research efforts are increasingly focused on development of patient-level prediction models using historical data. Most prediction models provide diagnostic or prognostic probabilities using a score or risk stratification algorithm and aim to assist clinicians to identify patients who require diagnostic tests or treatment. Examples within CRC are (1) prediction tools developed to identify individuals at increased risk of CRC, which could optimize cancer screening [82], (2) models to predict recurrence of disease in the adjuvant and the oligometastatic setting, which could guide frequency of diagnostic imaging and decisions regarding postoperative adjuvant chemotherapy [83,84,85], and (4) models that estimate probability of survival at a specific moment in time, which could aid in decisions regarding surgery or salvage treatment [86, 87]. Nevertheless, to our knowledge, use in clinical practice is limited and there are no CRC prediction models yet that have undergone impact analysis to determine whether they indeed improve outcomes when used in clinical practice [88, 89]. Before adopting a prediction rule and evaluating impact, external validation should be performed to assess whether a prediction model is accurate and applicable to a specific setting [90]. The proportion of such external validation studies is currently small. The lack of standardization of dataset formats and variable nomenclature provides an obstacle since data curation can be very time-consuming. In the past years, external validation of several promising prediction models in the metastatic setting using population-based RWD unfortunately demonstrated suboptimal predictive performance. However, opportunities for improvement were identified for instance by including additional predictors [91, 92]. Since both the availability of high-quality RWD and methods to analyze large datasets are improving, we expect impactful prediction and decision models in the future. These could be implemented in clinical care by creating patients-like-me dashboards that can be used in the consultation room to facilitate shared decision-making.
Using RWD to optimize treatment delivery
In addition to the development of novel therapies, the outcomes of CRC patients can also be improved by making better use of the therapies that we already have. To this end, RWD may be used to evaluate quality of care, for example, by looking at treatment adoption, guideline adherence, and access to care. In the adjuvant CRC setting, guideline adherence was previously shown to be limited [93]. Results of the IDEA trial led to the recommendation of 3 instead of 6 months of combination chemotherapy [94]. Population-based data has since demonstrated rapid implementation of these recommendations with improved guideline-concordant treatment [95]. In the metastatic setting, the evolving therapeutic landscape has led to a continuum of care in which the optimal sequence of treatment is currently unknown. A key principle is to strive to ensure that patients receive all effective agents for which they are eligible. Hence, we have used RWD to evaluate treatment patterns, practice variation, and adoption of new treatment options in the Netherlands [57, 96, 97]. These results received nationwide attention, were discussed intensively, and have resulted in practice changes. Since examples of application of RWD to evaluate care for CRC are abundant, Table 2 provides additional examples.
For interpretation of RWE, it is relevant to consider the large differences between countries in both drug access and adoption [34]. In Europe, EMA provides marketing authorization to pharmaceutical companies after assessment of drug safety and efficacy. After authorization, individual countries apply their own processes and requirements to decide on reimbursement of a registered drug. For instance, in the Czech Republic, reimbursement of regorafenib by public health insurance is or was conditional on the contribution of data to the CORECT registry with the goal to evaluate effectiveness [98]. The ongoing PROMETCO study, an international prospective longitudinal cohort, evaluates key differences between countries in the management and outcomes of mCRC [99].
Remaining barriers and future perspectives
We have repeatedly highlighted the potential of high-quality RWE. The pursuit of actionable high-quality evidence is logical but has its challenges. Most RWE currently derives from RWD of a single center, data source, or country which strongly limits analytic possibilities. However, sharing and combining RWD is challenging for multiple reasons. First, there is the variability between RWD sources in both format and terminology. Second, a unique identifier is needed to enable proper linkage and avoid patient duplication. Third, use and sharing of RWD is protected by rather strict legal and ethical requirements to protect patient privacy and requires patient consent. It is important to realize that many patients are in favor of secondary use of their clinical data and biological samples. Hence, the EU Data Protection Regulation Recital 157 allows population-based cancer registries to operate with a “no-consent” policy under the supervision of relevant public health bodies [100]. As most countries outside the EU are not recognized to have equivalent data protection procedures in place, there is at present no practical way to share health data for research purposes, resulting in suspended and delayed international research projects [100]. Given these challenges, transparency is imperative to translate results to different settings, reproduce RWE, and compare outcomes. Reporting of RWE is however often of limited quality due to insufficiently described outcome and variable definitions, study populations, healthcare settings, and analytic procedures. To improve reporting quality of oncology RWE studies, the ESMO Real World Data and Digital Health working group developed the ESMO Guidance for Reporting Oncology real-World evidence (GROW) [101••]. These standards will not only improve the quality of reporting, but also serve as a basis for the development of study conduct assessment, and ultimately facilitate the incorporation of reliable RWE in clinical treatment guidelines.
To solve aforementioned data-related challenges, common data models (CDM) have been designed over the last decade with the aim to standardize the structure and content of observational data sources. The CDM developed by the Observational Medical Outcomes Partnership (OMOP) is recognized as most promising and is maintained and deployed by the international Observational Health Data Sciences and Informatics (OHDSI) collaboration [102, 103]. Besides transformation of registries and databases into a common format with common representation of terminology, definitions and coding scheme, this model can be used to perform systematic analyses using an open-source library of analytic procedures, which are developed by OHDSI today. We believe this will greatly aid in reproducibility, i.e., timely external validation of predictive models in different settings, and increased trust in RWE resulting from transparency. In 2022, 453 large datasets from 41 countries had been converted to the OMOP-CDM representing 12% of the world’s population [104]. As these numbers are increasing rapidly, this provides much opportunity for the near future.
Another exciting development—which relies heavily on the availability of high-quality RWD—is the application of artificial intelligence (AI) in healthcare. Machine learning (ML) and deep learning techniques are believed to have the potential to accelerate oncological drug discoveries and personalize healthcare [105•]. Multiple recent reviews have highlighted the value and current applications of AI in CRC which lies outside the scope of this review [106,107,108]. AI algorithms need large volumes of data to train and obtain the best results. There is a large treasure of unstructured data stored in EHRs, which is still largely unused for research. Natural language processing, a form of ML, is now able to analyze these data which could improve predictive modelling accuracy [108]. Importantly, quantity does not make up for suboptimal quality of documentation which is complicated further by lack of eHR interoperability. Hence, what is really needed is harmonization of eHRs and arranging them to serve not only clinical but also research purposes. Many countries aim to establish one country-wide eHR system with comprehensive sharing of records from multiple providers [109]. To overcome the challenge regarding patient privacy, multiple observational research groups are working on a privacy-by-design approach using federated learning; a ML technique that performs an analysis across multiple decentralized data sources [110•]. The aggregated outcomes and model parameters from the decentralized sources are combined in a central server that provides the researcher with one result based on complex mathematics. This method does not require exchanging of raw and sensitive patient data. Although AI techniques are promising and start to impact diagnostic imaging in clinical practice, application in CRC treatment is still in the experimental stage and faces many challenges. Most important to realize is that they are not yet able to make accurate causal inference and are therefore not equipped to recommend the optimal treatment for an individual CRC patient [105•].
To conclude, in the trial design phase, we recommend to carefully consider pragmatic trial designs to increase generalizability whenever suitable; in the drug regulatory phase to provide conditional marketing approval for treatment of patients who would have been ineligible for the registrational trial—awaiting planned evaluation of outcomes in the real-world; and lastly regarding the clinical application, effectiveness results of high-quality RWE studies should be incorporated in treatment guidelines to support optimal patient counseling. We emphasize that both RWD and results from RCTs are needed to improve care for patients with CRC. A coordinated effort among all stakeholders, i.e., healthcare professionals, patient advocates, HTA bodies and payers, regulators, epidemiologists, and statisticians, is needed to achieve high-quality primary data and ensure high-quality secondary use. Supported by further sophistication of sources and analytical methods, we believe it to be possible to use RWD to answer questions of all stakeholders, reduce oncological healthcare costs, and, most importantly, improve patient care and outcomes.
References and Recommended Reading
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?” The Lancet. 2005;365(9453):82–93.
Ayaz A, Naqvi SAA, Islam M, Ikram W, Raza U, Riaz A, et al. Source of funding and enrollment disparities in the inclusion of minorities in colorectal clinical trials: a systematic review and meta-analysis. Journal of Clinical Oncology. 2023;41(4_suppl):23–23.
• Batra A, Kong S, Cheung WY. Eligibility of real-world patients with stage ii and iii colon cancer for adjuvant chemotherapy trials. Clin Colorectal Cancer. 2020;19(4):e226-34. Study that evaluates and reflects on the eligibility of real-world patients for colorectal cancer adjuvant chemotherapy trials.
Subbiah V, Solit DB, Chan TA, Kurzrock R. The FDA approval of pembrolizumab for adult and pediatric patients with tumor mutational burden (TMB) ≥10: a decision centered on empowering patients and their physicians. Ann Oncol. 2020;31(9):1115–8.
• Del Paggio JC, Berry JS, Hopman WM, Eisenhauer EA, Prasad V, Gyawali B, et al. Evolution of the randomized clinical trial in the era of precision oncology. JAMA Oncol. 2021;7(5):728. Retrospective cohort study that describes trends over time in the design, funding, and results of randomized controlled trials in oncology.
Cave A, Kurz X, Arlett P. Real-world data for regulatory decision making: challenges and possible solutions for Europe. Clin Pharmacol Ther. 2019;106(1):36–9.
Relton C, Torgerson D, O’Cathain A, Nicholl J. Rethinking pragmatic randomised controlled trials: introducing the “cohort multiple randomised controlled trial” design. BMJ. 2010;340(mar191):c1066–c1066.
Li G, Sajobi TT, Menon BK, Korngut L, Lowerison M, James M, et al. Registry-based randomized controlled trials- what are the advantages, challenges, and areas for future research? J Clin Epidemiol. 2016;80:16–24.
•• Saesen R, Van Hemelrijck M, Bogaerts J, Booth CM, Cornelissen JJ, Dekker A, et al. Defining the role of real-world data in cancer clinical research: the position of the European Organisation for Research and Treatment of Cancer. Eur J Cancer. 2023;186:52–61. Perspective of the European Organisation for Research and Treatment of Cancer (EORTC) on use of real-world data in cancer clinical research.
Saesen R, Kantidakis G, Marinus A, Lacombe D, Huys I. How do cancer clinicians perceive real-world data and the evidence derived therefrom? Findings from an international survey of the European Organisation for Research and Treatment of Cancer. Front Pharmacol. 2022;24:13.
US Food and Drug Administration. Framework for FDA’s real world evidence program. 2018. Available at: https://www.fda.gov/media/120060/download. Accessed 12 Sep 2023.
European Medicines Agency. Real-world evidence framework to support EU regulatory decision-making. Reference EMA/289699/2023. 2023. Available at: https://www.ema.europa.eu/en/documents/report/real-world-evidence-framework-support-eu-regulatory-decision-making-report-experience-gained-regulator-led-studiesseptember-2021-february-2023_en.pdf. Accessed 12 Sep 2023.
Punt CJA, Koopman M, Vermeulen L. From tumour heterogeneity to advances in precision treatment of colorectal cancer. Nat Rev Clin Oncol. 2017;14(4):235–46.
Iversen LH, Green A, Ingeholm P, Østerlind K, Gögenur I. Improved survival of colorectal cancer in Denmark during 2001–2012 – the efforts of several national initiatives. Acta Oncol (Madr). 2016;55(sup2):10–23.
Iversen LH, Ingeholm P, Gögenur I, Laurberg S. Major reduction in 30-day mortality after elective colorectal cancer surgery: a nationwide population-based study in Denmark 2001–2011. Ann Surg Oncol. 2014;21(7):2267–73.
Brouwer NPM, Bos ACRK, Lemmens VEPP, Tanis PJ, Hugen N, Nagtegaal ID, et al. An overview of 25 years of incidence, treatment and outcome of colorectal cancer patients. Int J Cancer. 2018;143(11):2758–66.
Siegel RL, Wagle NS, Cercek A, Smith RA, Jemal A. Colorectal cancer statistics, 2023. CA Cancer J Clin. 2023;73(3):233–54.
Holleczek B, Rossi S, Domenic A, Innos K, Minicozzi P, Francisci S, et al. On-going improvement and persistent differences in the survival for patients with colon and rectum cancer across Europe 1999–2007 – results from the EUROCARE-5 study. Eur J Cancer. 2015;51(15):2158–68.
Lundberg FE, Birgisson H, Johannesen TB, Engholm G, Virtanen A, Pettersson D, et al. Survival trends in patients diagnosed with colon and rectal cancer in the nordic countries 1990–2016: the NORDCAN survival studies. Eur J Cancer. 2022;172:76–84.
• Biller LH, Schrag D. Diagnosis and treatment of metastatic colorectal cancer: a review. JAMA. 2021;325(7):669–85. Comprehensive review on treatment options for metastatic colorectal cancer.
Cremolini C, Antoniotti C, Rossini D, Lonardi S, Loupakis F, Pietrantonio F, et al. TRIBE2: upfront FOLFOXIRI plus bevacizumab and reintroduction after progression versus mFOLFOX6 plus bevacizumab followed by FOLFIRI plus bevacizumab in the treatment of patients with metastatic colorectal cancer (TRIBE2): a multicentre, open-label, phase. Lancet Oncol. 2020;21(4):497–507.
• Shen C, Tannenbaum D, Horn R, Rogers J, Eng C, Zhou S, et al. Overall survival in phase 3 clinical trials and the surveillance, epidemiology, and end results database in patients with metastatic colorectal cancer, 1986–2016. JAMA Netw Open. 2022;5(5):e2213588. Systematic review on the factors associated with clinically relevant improvement (defined as 2 months) in overall survival of mCRC patients in trials, and the association with outcomes reflected in SEER registry data.
Zeineddine FA, Zeineddine MA, Yousef A, Gu Y, Chowdhury S, Dasari A, et al. Survival improvement for patients with metastatic colorectal cancer over twenty years. NPJ Precis Oncol. 2023;7(1):16.
Ellis L, Woods LM, Estève J, Eloranta S, Coleman MP, Rachet B. Cancer incidence, survival and mortality: explaining the concepts. Int J Cancer. 2014;135(8):1774–82.
Feinstein AR, Sosin DM, Wells CK. The Will Rogers Phenomenon. N Engl J Med. 1985;312(25):1604–8.
•• Templeton AJ, Booth CM, Tannock IF. Informing patients about expected outcomes: the efficacy-effectiveness gap. Journal of Clinical Oncology. 2020;38(15):1651–4. Commentary on the existence and relevance of the efficacy-effectiveness gap.
Jin S, Pazdur R, Sridhara R. Re-evaluating eligibility criteria for oncology clinical trials: analysis of investigational new drug applications in 2015. J Clin Oncol. 2017;35(33):3745–52.
Westgeest HM, Uyl-de Groot CA, van Moorselaar RJA, de Wit R, van den Bergh ACM, Coenen JLLM, et al. Differences in trial and real-world populations in the Dutch Castration-resistant Prostate Cancer Registry. Eur Urol Focus. 2018;4(5):694–701.
Templeton AJ, Vera-Badillo FE, Wang L, Attalla M, De Gouveia P, Leibowitz-Amit R, et al. Translating clinical trials to clinical practice: outcomes of men with metastatic castration resistant prostate cancer treated with docetaxel and prednisone in and out of clinical trials. Ann Oncol. 2013;24(12):2972–7.
Mol L, Koopman M, van Gils CWM, Ottevanger PB, Punt CJA. Comparison of treatment outcome in metastatic colorectal cancer patients included in a clinical trial versus daily practice in The Netherlands. Acta Oncol (Madr). 2013;52(5):950–5.
Truong J, Lee EK, Trudeau ME, Chan KKW. Interpreting febrile neutropenia rates from randomized, controlled trials for consideration of primary prophylaxis in the real world: a systematic review and meta-analysis. Ann Oncol. 2016;27(4):608–18.
Schnipper LE, Davidson NE, Wollins DS, Blayney DW, Dicker AP, Ganz PA, et al. Updating the American Society of Clinical Oncology Value Framework: revisions and reflections in response to comments received. J Clin Oncol. 2016;34(24):2925–34.
Cherny NI, Dafni U, Bogaerts J, Latino NJ, Pentheroudakis G, Douillard JY, et al. ESMO-Magnitude of Clinical Benefit Scale version 1.1. Annals of Oncology. 2017;28(10):2340–66.
Vogler S, Alexander M, Dedet G, Lam J, Bak Pedersen H. World Health Organisation - Medicines Reimbursement Policies in Europe. 2018. Available at: https://iris.who.int/bitstream/handle/10665/342220/9789289053365-eng.pdf?isAllowed=y&sequence=1. Accessed 22 Oct 2023.
Grothey A, Van CE, Sobrero A, Siena S, Falcone A, Ychou M, et al. Regorafenib monotherapy for previously treated metastatic colorectal cancer (CORRECT): an international, multicentre, randomised, placebo-controlled, phase 3 trial. The Lancet. 2013;381(9863):303–12.
Mayer RJ, Van Cutsem E, Falcone A, Yoshino T, Garcia-Carbonero R, Mizunuma N, et al. Randomized trial of TAS-102 for refractory metastatic colorectal cancer. N Engl J Med. 2015;372(20):1909–19.
Makady A, ten Ham R, de Boer A, Hillege H, Klungel O, Goettsch W. Policies for use of real-world data in health technology assessment (HTA): a comparative study of six HTA agencies. Value in Health. 2017;20(4):520–32.
Tabernero J, Grothey A, Van Cutsem E, Yaeger R, Wasan H, Yoshino T, et al. Encorafenib plus cetuximab as a new standard of care for previously treated BRAF V600E–mutant metastatic colorectal cancer: updated survival results and subgroup analyses from the BEACON study. J Clin Oncol. 2021;39(4):273–84.
• Cervantes A, Adam R, Roselló S, Arnold D, Normanno N, Taïeb J, et al. Metastatic colorectal cancer: ESMO Clinical Practice Guideline for diagnosis, treatment and follow-up. Annals of Oncology. 2023;34(1):10–32. The most recent ESMO treatment guideline for metastatic colorectal cancer.
• Morris VK, Kennedy EB, Baxter NN, Benson AB, Cercek A, Cho M, et al. Treatment of metastatic colorectal cancer: ASCO guideline. Journal of Clinical Oncology. 2023;41(3):678–700. The most recent ASCO treatment guideline for metastatic colorectal cancer.
Boccaccino A, Borelli B, Intini R, Antista M, Bensi M, Rossini D, et al. Encorafenib plus cetuximab with or without binimetinib in patients with BRAF V600E-mutated metastatic colorectal cancer: real-life data from an Italian multicenter experience. ESMO Open. 2022;7(3): 100506.
van Nassau SCMW, Zwart K, van der Baan F, Koopman M, van Gestel AJ, Snaebjornsson P, et al. Overall survival of patients with BRAF-mutant metastatic colorectal cancer treated with encorafenib-cetuximab in a real-world nationwide study in the Netherlands. Journal of Clinical Oncology. 2023;41(16_suppl):3589–3589.
• Hamers PAH, Elferink MAG, Stellato RK, Punt CJA, May AM, Koopman M, Vink GR. Informing metastatic colorectal cancer patients by quantifying multiple scenarios for survival time based on real-life data. Int J Cancer. 2021;148(2):296–306. https://doi.org/10.1002/ijc.33200. Real-world study that evaluates scenarios for survival of patients with mCRC. Such scenarios can be used to communicate life-expectancy to patients.
Lombardi P, Marandino L, De Luca E, Zichi C, Reale ML, Pignataro D, et al. Quality of life assessment and reporting in colorectal cancer: a systematic review of phase III trials published between 2012 and 2018. Crit Rev Oncol Hematol. 2020;146: 102877.
Davis C, Naci H, Gurpinar E, Poplavska E, Pinto A, Aggarwal A. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009–13. BMJ. 2017;4: j4530.
Burbach JPM, Kurk SA, Coebergh van den Braak RRJ, Dik VK, May AM, Meijer GA, et al. Prospective Dutch colorectal cancer cohort: an infrastructure for long-term observational, prognostic, predictive and (randomized) intervention research. Acta Oncol (Madr). 2016;55(11):1273–80.
Hamers PAH, Vink GR, Elferink MAG, Stellato RK, Dijksterhuis WPM, Punt CJA, et al. Quality of life and survival of metastatic colorectal cancer patients treated with trifluridine-tipiracil (QUALITAS). Clin Colorectal Cancer. 2022;21(2):154–66.
Bachet JB, Wyrwicz L, Price T, Cremolini C, Phelip JM, Portales F, et al. Safety, efficacy and patient-reported outcomes with trifluridine/tipiracil in pretreated metastatic colorectal cancer: results of the PRECONNECT study. ESMO Open. 2020;5(3): e000698.
Kwakman JJM, Simkens LHJ, van Rooijen JM, van de Wouw AJ, ten Tije AJ, Creemers GJM, et al. Randomized phase III trial of S-1 versus capecitabine in the first-line treatment of metastatic colorectal cancer: SALTO study by the Dutch Colorectal Cancer Group. Ann Oncol. 2017;28(6):1288–93.
Kwakman JJM, van Werkhoven E, Simkens LHJ, van Rooijen JM, van de Wouw YAJ, ten Tije AJ, et al. Updated survival analysis of the randomized phase iii trial of S-1 versus capecitabine in the first-line treatment of metastatic colorectal cancer by the Dutch Colorectal Cancer Group. Clin Colorectal Cancer. 2019;18(2):e229–30.
Derksen JWG, Smit KC, May AM, Punt CJA. Systematic review and non-inferiority meta-analysis of randomised phase II/III trials on S-1-based therapy versus 5-fluorouracil- or capecitabine-based therapy in the treatment of patients with metastatic colorectal cancer. Eur J Cancer. 2022;166:73–86.
• Punt CJA, Kwakman JJM, Mol L, Roodhart J, Hendriks M, Speetjens F, et al. Long-term safety data on S-1 administered after previous intolerance to capecitabine-containing systemic treatment for metastatic colorectal cancer. Clin Colorectal Cancer. 2022;21(3):229–35. Real-world study on S-1 long-term safety in mCRC that was supportive for EMA approval.
• Osterlund P, Kinos S, Pfeiffer P, Salminen T, Kwakman JJM, Frödin JE, et al. Continuation of fluoropyrimidine treatment with S-1 after cardiotoxicity on capecitabine- or 5-fluorouracil-based therapy in patients with solid tumours: a multicentre retrospective observational cohort study. ESMO Open. 2022;7(3):100427. International real-world study on S-1 safety that was supportive for EMA approval.
European Medicines Agency (EMA). Teysuno EPAR assessment report - variation. 2022. Available at: https://www.ema.europa.eu/en/documents/variationreport/teysuno-h-c-001242-ii-0045-epar-assessment-report-variation_en.pdf. Accessed 7 Sep 2023.
Punt CJA, Heinemann V, Maughan T, Cremolini C, Van Cutsem E, McDermott R, et al. Fluoropyrimidine-induced hand-foot syndrome and cardiotoxicity: recommendations for the use of the oral fluoropyrimidine S-1 in metastatic colorectal cancer. ESMO Open. 2023;8(2): 101199.
Woolacott N, Corbett M, Jones-Diette J, Hodgson R. Methodological challenges for the evaluation of clinical effectiveness in the context of accelerated regulatory approval: an overview. J Clin Epidemiol. 2017;90:108–18.
van Nassau SC, Bond MJ, Scheerman I, van Breeschoten J, Kessels R, Valkenburg-van Iersel LB, et al. Trends in use and perceptions about triplet chemotherapy plus bevacizumab for metastatic colorectal cancer. JAMA Netw Open. 2021;4(9): e2124766.
Hershman DL, Wright JD. Comparative effectiveness research in oncology methodology: observational data. J Clin Oncol. 2012;30(34):4215–22.
Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: The International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force Report—Part II. Value in Health. 2009;12(8):1053–61.
•• Karim S, Booth CM. Effectiveness in the absence of efficacy: cautionary tales from real-world evidence. Journal of Clinical Oncology. 2019;37(13):1047–50. Commentary that describes scenarios (in colorectal cancer) in which comparative effectiveness research can be hazardous if interpreted as conclusive.
Freischlag K, Sun Z, Adam MA, Kim J, Palta M, Czito BG, et al. Association between incomplete neoadjuvant radiotherapy and survival for patients with locally advanced rectal cancer. JAMA Surg. 2017;152(6):558.
Casadaban L, Rauscher G, Aklilu M, Villenes D, Freels S, Maker AV. Adjuvant chemotherapy is associated with improved survival in patients with stage II colon cancer. Cancer. 2016;122(21):3277–87.
Alawadi Z, Phatak UR, Hu CY, Bailey CE, You YN, Kao LS, et al. Comparative effectiveness of primary tumor resection in patients with stage IV colon cancer. Cancer. 2017;123(7):1124–33.
Lam-Boer J’t, Van der Geest LG, Verhoef C, Elferink ME, Koopman M, de Wilt JH. Palliative resection of the primary tumor is associated with improved overall survival in incurable stage IV colorectal cancer: a nationwide population-based propensity-score adjusted study in the Netherlands. Int J Cancer. 2016;139(9):2082–94.
Kanemitsu Y, Shitara K, Mizusawa J, Hamaguchi T, Shida D, Komori K, et al. Primary tumor resection plus chemotherapy versus chemotherapy alone for colorectal cancer patients with asymptomatic, synchronous unresectable metastases (JCOG1007; iPACS): a randomized clinical trial. J Clin Oncol. 2021;39(10):1098–107.
Rahbari NN, Biondo S, Feißt M, Bruckner T, Rossion I, Luntz S, et al. Randomized clinical trial on resection of the primary tumor versus no resection prior to systemic therapy in patients with colon cancer and synchronous unresectable metastases. Journal of Clinical Oncology. 2022;40(17):3507–3507.
Lodeweges JE, Klinkenberg TJ, Ubbels JF, Groen HJM, Langendijk JA, Widder J. Long-term outcome of surgery or stereotactic radiotherapy for lung oligometastases. J Thorac Oncol. 2017;12(9):1442–5.
Couwenberg AM, Burbach JPM, Berbee M, Lacle MM, Arensman R, Raicu MG, et al. Efficacy of dose-escalated chemoradiation on complete tumor response in patients with locally advanced rectal cancer (RECTAL-BOOST): a phase 2 randomized controlled trial. International Journal of Radiation Oncology*Biology*Physics. 2020;108(4):1008–18.
Schraa SJ, van Rooijen KL, van der Kruijssen DEW, Rubio Alarcón C, Phallen J, Sausen M, et al. Circulating tumor DNA guided adjuvant chemotherapy in stage II colon cancer (MEDOCC-CrEATE): study protocol for a trial within a cohort study. BMC Cancer. 2020;20(1):790.
Schröder C, Lawrance M, Li C, Lenain C, Mhatre SK, Fakih M, et al. Building external control arms from patient-level electronic health record data to replicate the randomized IMblaze370 control arm in metastatic colorectal cancer. JCO Clin Cancer Inform. 2021;5:450–8.
Backenroth D, Royce T, Pinheiro J, Samant M, Humblet O. Considerations for pooling real-world data as a comparator cohort to a single arm trial: a simulation study on assessment of heterogeneity. BMC Med Res Methodol. 2023;23(1):193.
Mulder J, Teerenstra S, van Hennik PB, Pasmooij AMG, Stoyanova-Beninska V, Voest EE, et al. Single-arm trials supporting the approval of anticancer medicinal products in the European Union: contextualization of trial results and observed clinical benefit. ESMO Open. 2023;8(2): 101209.
Overman MJ, Lonardi S, Wong KYM, Lenz HJ, Gelsomino F, Aglietta M, et al. Durable clinical benefit with nivolumab plus ipilimumab in DNA mismatch repair–deficient/microsatellite instability–high metastatic colorectal cancer. J Clin Oncol. 2018;36(8):773–9.
Wensink GE, Elferink MAG, May AM, Mol L, Hamers PAH, Bakker SD, et al. Survival of patients with deficient mismatch repair metastatic colorectal cancer in the pre-immunotherapy era. Br J Cancer. 2021;124(2):399–406.
Tougeron D, Sueur B, Zaanan A, Fouchardiére C, Sefrioui D, Lecomte T, et al. Prognosis and chemosensitivity of deficient MMR phenotype in patients with metastatic colorectal cancer: an AGEO retrospective multicenter study. Int J Cancer. 2020;147(1):285–96.
European Medicines Agency (EMA). Opdivo EPAR assessment report - variation. 2021. Available at: https://www.ema.europa.eu/en/documents/variationreport/opdivo-h-c-3985-ws-1840-epar-assessment-report-variation_en.pdf. Accessed 1 Sep 2023.
Derksen JWG, Martins Branco D, Pellat A, van Nassau SCMW, Valachis A, Aggarwal A, et al. 1702P Real-world evidence contributions to European medicines agency’s safety and efficacy evaluations of oncology targeted therapies between 2018–2022. Ann Oncol. 2023;34:S930.
Lièvre A, Bachet JB, Le Corre D, Boige V, Landi B, Emile JF, et al. KRAS mutation status is predictive of response to cetuximab therapy in colorectal cancer. Cancer Res. 2006;66(8):3992–5.
• van de Haar J, Ma X, Ooft SN, van der Helm PW, Hoes LR, Mainardi S, et al. Codon-specific KRAS mutations predict survival benefit of trifluridine/tipiracil in metastatic colorectal cancer. Nat Med. 2023;29(3):605–14. Study that uses real-world data to identify biomarkers of response and resistance to trifluridine/ tipiracil mCRC and validates findings in both a realworld cohort and phase III trial.
Kiely BE, McCaughan G, Christodoulou S, Beale PJ, Grimison P, Trotman J, et al. Using scenarios to explain life expectancy in advanced cancer: attitudes of people with a cancer experience. Support Care Cancer. 2013;21(2):369–76.
• Booth CM, Sengar M, Goodman A, Wilson B, Aggarwal A, Berry S, et al. Common Sense Oncology: outcomes that matter. Lancet Oncol. 2023;24(8):833–5. Commentary outlining the ‘“Common Sense Oncology”’ (CSO) movement which was recently established with the goal to ensure that oncology trials and treatments are focused on outcomes that matter to patients.
Mülder DT, van den Puttelaar R, Meester RGS, O’Mahony JF, Lansdorp-Vogelaar I. Development and validation of colorectal cancer risk prediction tools: a comparison of models. Int J Med Inform. 2023;178: 105194.
Spelt L, Andersson B, Nilsson J, Andersson R. Prognostic models for outcome following liver resection for colorectal cancer metastases: a systematic review. European Journal of Surgical Oncology (EJSO). 2012;38(1):16–24.
• Kleppe A, Skrede OJ, De Raedt S, Hveem TS, Askautrud HA, Jacobsen JE, et al. A clinical decision support system optimising adjuvant chemotherapy for colorectal cancers by integrating deep learning and pathological staging markers: a development and validation study. Lancet Oncol. 2022;23(9):1221–32. A risk stratification system using a deep learning-–based marker analyzing H&E-stained tissue sections (DoMore-v1-CRC) which could potentially help improve selection of adjuvant chemotherapy after complete resection of colorectal cancer.
Oyaga-Iriarte E, Insausti A, Sayar O, Aldaz A. Prediction of irinotecan toxicity in metastatic colorectal cancer patients based on machine learning models with pharmacokinetic parameters. J Pharmacol Sci. 2019;140(1):20–5.
Pietrantonio F, Miceli R, Rimassa L, Lonardi S, Aprile G, Mennitto A, et al. Estimating 12-week death probability in patients with refractory metastatic colorectal cancer: the Colon Life nomogram. Ann Oncol. 2017;28(3):555–61.
Vogelsang RP, Bojesen RD, Hoelmich ER, Orhan A, Buzquurz F, Cai L, et al. Prediction of 90-day mortality after surgery for colorectal cancer using standardized nationwide quality-assurance data. BJS Open. 2021;5(3):zrab023. https://doi.org/10.1093/bjsopen/zrab023
Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1-73.
Wallace E, Smith SM, Perera-Salazar R, Vaucher P, McCowan C, Collins G, et al. Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs). BMC Med Inform Decis Mak. 2011;14(11):62.
Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–9.
Bolhuis K, Wensink GE, Elferink MAG, Bond MJG, Dijksterhuis WPM, Fijneman RJA, et al. External Validation of two established clinical risk scores predicting outcome after local treatment of colorectal liver metastases in a nationwide cohort. Cancers (Basel). 2022;14(10):2356.
Hamers PAH, Wensink GE, van Smeden M, Vink GR, Smabers LP, Lunenberg RA, et al. External validation of the colon life nomogram for predicting 12-week mortality in Dutch metastatic colorectal cancer patients treated with trifluridine/tipiracil in daily practice. Cancers (Basel). 2022;14(20):5094.
Chagpar R, Xing Y, Chiang YJ, Feig BW, Chang GJ, You YN, et al. Adherence to stage-specific treatment guidelines for patients with colon cancer. J Clin Oncol. 2012;30(9):972–9.
Grothey A, Sobrero AF, Shields AF, Yoshino T, Paul J, Taieb J, et al. Duration of adjuvant chemotherapy for stage iii colon cancer. N Engl J Med. 2018;378(13):1177–88.
van Rooijen KL, Derksen JWG, Verkooijen HM, Vink GR, Koopman M. Translation of IDEA trial results into clinical practice: analysis of the implementation of a new guideline for colon cancer. Int J Cancer. 2022;151(8):1270–9.
Keikes L, van Oijen MGH, Lemmens VEPP, Koopman M, Punt CJA. Evaluation of guideline adherence in colorectal cancer treatment in The Netherlands: a survey among medical oncologists by the Dutch Colorectal Cancer Group. Clin Colorectal Cancer. 2018;17(1):58–64. https://doi.org/10.1016/j.clcc.2017.10.007.
Keikes L, Koopman M, Stuiver MM, Lemmens VEPP, van Oijen MGH, Punt CJA. Practice variation on hospital level in the systemic treatment of metastatic colorectal cancer in The Netherlands: a population-based study. Acta Oncol (Madr). 2020;59(4):395–403. https://doi.org/10.1080/0284186X.2020.1722320.
Novakova-Jiresova A, Kopeckova K, Boublikova L, Chloupkova R, Melichar B, Petruzelka L, et al. Regorafenib for metastatic colorectal cancer: an analysis of a registry-based cohort of 555 patients. Cancer Manag Res. 2020;12:5365–72.
Koopman M, Pinto C, Bodoky G, Garcia-Carbonero R, Marti FM, Bachet JB. Rationale and design of the PROMETCO study: a real-world, prospective, longitudinal cohort on the continuum of care of metastatic colorectal cancer from a clinical and patient perspective. Future Oncol. 2022;18(11):1313–20.
Casali PG, Vyas M. Data protection and research in the European Union: a major step forward, with a step back. Ann Oncol. 2021;32(1):15–9.
•• Castelo-Branco L, Pellat A, Martins-Branco D, Valachis A, Derksen JWG, Suijkerbuijk KPM, et al. ESMO Guidance for Reporting Oncology real-World evidence (GROW). Ann Oncol. 2023;34(12):1097–112. https://doi.org/10.1016/j.annonc.2023.10.001. Specific guidance, including an easy-to-use checklist, for reporting of oncology real-world evidence studies in peer-reviewed journals: the ESMO Guidance for Reporting Oncology Real-World Evidence (ESMO-GROW).
Garza M, Del Fiol G, Tenenbaum J, Walden A, Zozus MN. Evaluating common data models for use with a longitudinal community registry. J Biomed Inform. 2016;64:333–41.
Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ, et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574–8.
Observational Health Data Sciences and Informatics. Where the OHDSI Community Has Been and Where We are Going. Updated 2022. Available at: https://www.ohdsi.org/wp-content/uploads/2022/10/OHDSI-OurJourney-2022.pdf. Accessed 11 Sep 2023.
• El Naqa I, Karolak A, Luo Y, Folio L, Tarhini AA, Rollison D, et al. Translation of AI into oncology clinical practice. Oncogene. 2023;42(42):3089–97. https://doi.org/10.1038/s41388-023-02826-z. Review that highlights the challenges impeding artificial intelligence clinical translation in oncology.
Abdul Rahman H, Ottom MA, Dinov ID. Machine learning-based colorectal cancer prediction using global dietary data. BMC Cancer. 2023;23(1):144.
• Mitsala A, Tsalikidis C, Pitiakoudis M, Simopoulos C, Tsaroucha AK. Artificial intelligence in colorectal cancer screening, diagnosis and treatment. a new era. Current Oncology. 2021;28(3):1581–607. Review on the applications of artificial intelligence in colorectal cancer care.
Causa Andrieu P, Golia Pernicka JS, Yaeger R, Lupton K, Batch K, Zulkernine F, et al. Natural language processing of computed tomography reports to label metastatic phenotypes with prognostic significance in patients with colorectal cancer. JCO Clin Cancer Inform. 2022;6:e2200014. https://doi.org/10.1200/CCI.22.00014
Oderkirk J. OECD Health Working Paper No. 99 - Findings of the 2016 OECD HCQI Study of Electronic Health Record System Development and Data Use. 2017. Available from: http://www.oecd.org/els/health-systems/health-working-papers.htm. Accessed 12 Sep 2023.
• Rieke N, Hancox J, Li W, Milletarì F, Roth HR, Albarqouni S, et al. The future of digital health with federated learning. NPJ Digit Med. 2020;3(1):119. Perspective paper on federated learning as a solution for the future of digital health.
Patel AK, Barghout V, Yenikomshian MA, Germain G, Jacques P, Laliberté F, et al. Real-world adherence in patients with metastatic colorectal cancer treated with trifluridine plus tipiracil or regorafenib. Oncologist. 2020;25(1):e75-84.
Birkett RT, Chamely E, Concors SJ, Bleier JI, Aarons CB, Shanmugan S, et al. Overuse and limited benefit of chemotherapy for stage ii colon cancer in young patients. Clin Colorectal Cancer. 2019;18(4):292–300.
Meyer Y, Olthof PB, Grünhagen DJ, de Hingh I, de Wilt JHW, Verhoef C, et al. Treatment of metachronous colorectal cancer metastases in the Netherlands: a population-based study. Eur J Surg Oncol. 2022;48(5):1104–9.
van de Schootbrugge-Vandermeer HJ, Lansdorp-Vogelaar I, de Jonge L, van Vuuren AJ, Dekker E, Spaander MCW, et al. Socio-demographic and cultural factors related to non-participation in the Dutch colorectal cancer screening programme. Eur J Cancer. 2023;190: 112942.
Schraa S, Laclé MM, Zwart K, Gort EH, Koopman M, de Leng W, et al. 364P Prevalence, treatment and survival of NTRK gene fusions in microsatellite instable metastatic colorectal cancer patients. Ann Oncol. 2022;33:S703.
Booth CM, Nanji S, Wei X, Mackillop WJ. Management and outcome of colorectal cancer liver metastases in elderly patients. JAMA Oncol. 2015;1(8):1111.
Murphy CC, Harlan LC, Warren JL, Geiger AM. Race and insurance differences in the receipt of adjuvant chemotherapy among patients with stage iii colon cancer. J Clin Oncol. 2015;33(23):2530–6.
Author information
Authors and Affiliations
Contributions
Writing—original draft preparation: S.N. and J.D.
Writing—review and editing; discussion of content: G.B., F.B., J.R., G.V., C.P., A.M., M.K.
Preparation of figure 1: S.N., C.P., and M.K.
All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
S.N. reports receiving support for travel from Servier via institution outside the submitted work. G.B. reports a patent for biomarker DDX3, receiving institutional grants from Bayer, Terumo and Servier, and honoraria for lectures from Pierre Fabre, Terumo and Bayer outside the submitted work. F.B. reports receiving an institutional grant from Personal Genome Diagnostics (PGDx) outside the submitted work. J.R. reports receiving institutional grants from Bristol Myers Squibb, Pierre Fabre, Servier, HUB 4 organoids, Clear Biotech, GSK, Cilis biotech, DoMore diagnostics, receiving consulting fees via institution from Bayer, Bristol Myers Squibb, Merck-Serono, Pierre Fabre, Servier, GSK, Amgen, has received payment via institution for lectures from Bristol Myers Squibb, Pierre Fabre and Servier, support for meetings and/or travel from Servier, participates on a Data Safety Monitoring Board or Advisory Board for PELVEX and MENDIT, and non-remunerated activity in the ONCODE clinical advisory board, KWF scientific board, and Hubrecht Organoid Biobank foundation management board, all outside the submitted work. G.V. reports receiving institutional grants from PGDx, Delfi diagnostics, Merck, Servier, Bayer, Bristol Myers Squibb, Sirtex and Pierre Fabre outside the submitted work. C.P. reports receiving consulting fees via institution from Nordic Pharma, outside the submitted work.
M.K. reports institutional scientific grants from Bayer, Bristol Myers Squibb, Merck, PGDx, Pierre Fabre, Roche, Sirtex and Servier, has an advisory role for Eisai, Nordic Farma, Merck-Serono, Pierre Fabre, and Servier, is principal investigator of the Prospective Dutch CRC (PLCRC) cohort and the international cohort study PROMETCO with Servier as sponsor, and reports non-remunerated activity as vice-chair for the Dutch Colorectal Cancer Group, chair of the ESMO Real World Data and Digital Health Working Group, and involvement in several colorectal cancer clinical trials as PI or co-investigator, all outside the submitted work. A.M. and J.D. declare no conflict of interest.
Human and Animal Rights and Informed Consent
This article does not contain any experiments with human or animal subjects performed by any of the authors.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
van Nassau, S.C.M.W., Bol, G.M., van der Baan, F.H. et al. Harnessing the Potential of Real-World Evidence in the Treatment of Colorectal Cancer: Where Do We Stand?. Curr. Treat. Options in Oncol. 25, 405–426 (2024). https://doi.org/10.1007/s11864-024-01186-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11864-024-01186-4