In the current issue of Annals of Surgical Oncology, Aquina et al. evaluated textbook oncologic outcomes (TOOs) across several cancer sites using data from the US National Cancer Database (NCDB) for over 800,000 patients.1 Using adjusted hospital TOO rates, the study established a direct association with overall survival by cancer site, thereby concluding that TOO is a valid metric that can be used to compare the overall quality of cancer care and drive improvement at the hospital level.

Textbook outcomes based on composite measures provide a more comprehensive picture of the postoperative journey than traditional risk-adjusted morbidity and mortality rates.2 While some metrics will trend in the same direction, such as postoperative complications and length of stay,3, 4 others may not. For example, evaluating length of stay, without considering readmission rates may make it challenging to draw conclusions about the efficacy of early discharge programs.5

A TOO incorporates not only the postoperative outcomes related to surgical morbidity (length of stay and readmission) but also the ‘ideal’ oncologic result (R0 resection, adequate lymph node yield, and appropriate adjuvant chemotherapy). This is particularly relevant in cancer surgery because the same factor can influence quality metrics in different directions. For example, in rectal cancer surgery, a total mesorectal excision (TME) may be associated with an improved R0 resection and better lymph node yield but a higher rate of anastomotic leak.6 TOOs start to unravel the complexity that unfolds when surgeons strive for the ‘ideal’ outcome, defined as the most complete resection with the lowest morbidity.

Risk adjustment is paramount when comparing outcomes and the more detailed the clinical and demographic data, the more relevant the estimation of quality provided by the metric.7 As surgeons, these factors are well-recognized and form the basis for our preoperative consent discussion. They can be reproduced with machine learning, where multifactorial prediction models for 30-day mortality and length of stay have an area under the receiver operating characteristic curve (AUC) of >0.8 for some cancer surgeries.8 Methodological excellence is apparent in the current study by Aquina et al., where additional risk adjustment, controlling for patient demographics, comorbidity status, tumor characteristics, and pathological stage, was undertaken because of evidence of residual confounding on bivariate analysis of patient and oncologic factors by adjusted hospital TOO rate quintile. While this additional adjustment controls for those factors captured in the NCDB data, it still falls short of acknowledging the complexity of technical, social, and biological factors that have all been correlated to cancer outcomes. Importantly, TOOs also fail to include any measure of QoL or patient experience, something which, while complicated to collect, provides a more meaningful interpretation of the cancer journey and has been repeatedly identified as being valued by patients.9

Comprehensive measures with detailed risk adjustment, such as the one described in the study by Aquino et al., are certainly an improvement over traditional unidimensional metrics. The most important question is, ‘What can hospitals and providers change and improve upon and how can this metric inform that?’ Specifically, what factors are modifiable and which are non-modifiable? In the present study, the overall TOO for colon cancer is approximately 67%, versus rectal cancer at <34%, despite the fact that the population demographic, surgeons, and hospital processes are likely identical for these two disease sites. This suggests that TOOs, and survival itself, are heavily influenced by anatomy and cancer biology, and are therefore not easily modifiable.

While TOOs provide a more complete picture of the perioperative cancer journey, it is important to reflect on the value of a TOO as a single quality metric, its limits, and the significant downside if we are not thoughtful about how this metric is used. At the provider level, TOOs may be designed to improve perioperative care or technical skills but, as described by Jerry Muller in ‘The Tyranny of Metrics’,10 their use could have unintended negative consequences. For example, the implementation of cardiac surgery report cards in New York State led to the phenomenon of risk aversion, where, in order to improve postoperative outcomes, surgeons decline to operate on high-risk patients. On the other hand, overly aggressive care may be recommended if oncologists feel compelled to encourage patients to receive adjuvant chemotherapy, even if the risks do not outweigh the benefits. TOOs assume that we look after textbook patients, failing to recognize the complexity of clinical decision making and provide deference to expertise, which may create situations of moral conflict for providers. Moreover, when communicated to patients, these types of metrics can often widen the disparity gap by directing those with the time, financial means, and social supports to visit a center with a higher reported TOO—all factors that are difficult to measure but that are associated with an improved oncologic outcome.11, 12

The study by Aquino et al. concludes that “TOO can be utilized to compare the overall quality of cancer care across hospitals”, but this may not be the case. As described above, TOOs will only provide information on the subset of cancer patients who are deemed eligible and selected for surgery and not all patients within the cancer system. Moreover, as hospitals work to pursue the Institute for Healthcare Improvements (IHI) Quadruple Aim,13 i.e. enhance patient experience, improve population health, reduce costs, and improve the work life of health care providers, TOOs fall short of helping them achieve these goals. At the 29th Annual National Forum on Quality Improvement in Healthcare, Dr. Don Berwick, President Emeritus at the IHI, acknowledged that healthcare is in trouble and competition is the problem.14

From a system perspective, it may be time to invest less in measurement and more in fostering the development of high reliability organizations, where coordinated efforts are needed to improve outcomes and where the focus is on how individuals within the organization interact with one another.15 In this philosophy, whose roots emerge from industries where failure has drastic consequences, such as nuclear power and naval aircraft operations, it is recognized that each patient is different and overstandardization can increase risks. In the complex business of caring for cancer patients, we need to resist the urge to oversimplify. A high reliability organization fosters strong relationships, built on mutual trust, that can maintain reliable performance, even in the setting of ongoing change. While the success of implementation will undoubtedly be harder to measure, it is nonetheless an endeavor worth pursuing.