National databases within the field of oncology capture detailed data on incident cancer diagnoses across a wide proportion of the population in the USA and represent a valuable resource for health service researchers as they study trends in cancer incidence, prevalence, treatment patterns, and disparities in oncologic outcomes.1,2 To fill gaps left by randomized controlled trials—the highest level of evidence—researchers increasingly rely on these databases to evaluate the effectiveness of cancer treatments.1 Additionally, these databases can be harnessed to help shape public policy by orienting prevention efforts and health care planning.

In this issue of Annals of Surgical Oncology, Rubenstein and colleagues compare the strengths and limitations of four national databases NSQIP, NIS, SEER, and NCDB in the context of breast cancer epidemiological research. Cancer incidence varies among states because of heterogeneity due to risk, genetic, and demographic factors.3 By evaluating each database for its clinical focus and research strengths as well as limitations, the authors highlight the unique qualities of each database and the need to enhance the value of our databases as breast cancer management becomes increasingly complex.

The authors note that, while the National Surgical Quality Improvement Program (NSQIP) provides the most detail about surgical procedures and outcomes, it does not provide facility information and has the smallest sample size of the four databases. Additionally, NSQIP provides data only for patients ≥ 18 years of age, while the other databases include patients of all ages.4 NSQIP also does not provide socioeconomic, education, or insurance information, thus limiting its utility in exploring disparities in access to care and oncologic outcomes.4 With its extensive list of comorbidities, preoperative laboratory values, and American Society of Anesthesiology (ASA) classification, as well as detailed operative and complications data, the authors conclude that NSQIP is best suited for studies evaluating surgical trends and perioperative outcomes.

The NIS, an inpatient hospitalization database, skews toward an older population. As such, a larger percentage of their dataset was ≥ 80 years of age compared with the other datasets. Although NIS does provide an extensive list of comorbidities and demographics, the number of patients in the NIS population during the study period decreased from 41,481 to 35,409, reflecting recent trends of earlier diagnosis and increasing rates of outpatient surgery. NIS only collects data during a single admission; therefore, as the authors note, costs related to complications that may occur after discharge would not be noted. Given that it is the only database that captures total charges and ownership of inpatient facilities as well as the quarter in which a discharge occurs, the authors argue that NIS is most useful for questions related to the cost of breast cancer treatment and temporal trends in inpatient burden.

The Surveillance, Epidemiology, and End Results (SEER) Program is sponsored by the Surveillance Research Program in the National Cancer Institute’s (NCI) Division of Cancer Control and Population Sciences (DCCPS). SEER collects cancer survival and incidence information from population-based cancer registries, encompassing 48% of the US population.5 In addition, SEER coverage includes 42.0% of white patients, 44.7% of African American patients, 66.3% of Hispanic patients, 59.9% of American Indian and Alaska Native patients, 70.7% of Asian patients, and 70.3% of Hawaiian/Pacific Islander patients.5 SEER provides more detailed groupings for race compared with NSQIP and NIS, and provides median household income on the basis of patient zip code. SEER provides detailed information on tumor characteristics, including histology, grade, behavior, site, size of tumor, and number of tumors. While the database includes surgical treatment and systemic and radiation treatment, the authors point out that SEER does not differentiate between endocrine therapy, immunotherapy, and chemotherapy, nor does SEER provide any information on patient comorbidities.4 As an epidemiological tool designed to monitor cancer incidence and mortality, the authors conclude that SEER can help to elucidate population-based trends in diagnosis and treatment in breast cancer.

The National Cancer Database (NCDB), a program jointly financed by the American College of Surgeons Commission on Cancer and the American Cancer Society, has, since its inception in 1988, accrued more than 34 million records from hospital cancer records in the USA and is the largest of the four databases.6 The data reported to the NCDB come from centers accredited by the Commission on Cancer, ranging from small, community-based hospitals to National Cancer Institute-designated Comprehensive Cancer Centers. Unlike SEER, it is not a strict population-based database, but with its robust data on adjuvant therapies, the authors suggest that the NCDB can be used to identify trends in treatment and adherence to standard of care.6 Importantly, previous studies have cautioned that COC-approved institutions differ from non-COC-approved institutions in size, geographic location, level of specialization of cancer care, and accreditation from other oversight agencies such as the Joint Commission and National Cancer Institute; as a result, it is important to note that results may not always be generalizable and that disparities in care detected in NCDB studies are likely underestimating the magnitude of differences.7

Acknowledging that use of these four databases surveyed in this manuscript is well suited for specific types of research, there are glaring limitations across all four databases. None of the databases provide specific information regarding BRCA1/2 or other genetic mutation status. For researchers who wish to specifically examine the outcomes of genetic mutation carriers, this is problematic. Moreover, neither SEER or NCDB provide information on tumor recurrences or disease-free survival, which significantly limit their utility, particularly at extremes of age. In a recent comparative effectiveness study, the authors collected 141 randomized clinical trials (RCTs) across eight tumor types and replicated these studies using data from the NCDB.1 They created patient cohorts within the NCDB that matched the RCT patient study populations with respect to age, diagnosis, and specific therapies. Unfortunately, the authors found that using comparative effectiveness research with cancer registry data often produces survival outcomes discordant with those of randomized clinical trials. Propensity-weighted hazard ratios for overall survival from NCDB-based analyses fell outside the 95% confidence intervals (CIs) of their RCT counterparts 36% of the time (with 64% falling within). Furthermore, observational studies led to a different inference regarding therapeutic efficacy 55% of the time (i.e., point estimates that were either in a different direction, nonsignificant in CER versus significant in RCT, or significant in CER but nonsignificant in RCT).1 Influence of selection bias, as well as misclassification and underreporting in cancer registries, have been implicated in these findings. Additionally, questions regarding the generalizability of the experience of cancer treatment in the context of a clinical trial compared with the real-world setting and generally better performance status among trial participants may also inform these observed differences in outcomes.1

While randomized controlled trials remain the gold standard for determining efficacy of treatments, these four national databases can be used to perform high-quality epidemiologic research. As Rubenstein and colleagues detail, each of the four databases has strengths and limitations and has a potential role in elucidating trends in treatment and outcomes in breast surgical oncology.