Skip to main content
SpringerLink
Log in

Strengths and Limitations of Registries in Surgical Oncology Research

  • Review Article
  • Published:
Journal of Gastrointestinal Surgery

Abstract

Over the past two decades, there has been a dramatic increase in studies based on large multi-institutional tumor registries. Applications of such databases span various research themes including epidemiology, oncology, surgical techniques, perioperative outcomes, and prognosis. Although these databases are acquired relatively easily, offer larger sample sizes and improved generalizability compared with institutional data, acknowledging limitations within analysis and cautious interpretation of data is important. Questionable conclusions can result when insufficient attention is paid to issues such as data quality and depth, potential sources of bias and missing data. This article reviews research themes and important limitations of these databases. The contemporary reporting of these issues in the literature and an increased awareness among surgical oncologists of potential applications and limitations will ensure that studies in the surgical oncology literature achieve high standards of methodological quality and clinical utility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Duggan, M. A.; Anderson, W. F.; Altekruse, S.; Penberthy, L.; Sherman, M. E., The Surveillance, Epidemiology, and End Results (SEER) Program and Pathology: Toward Strengthening the Critical Relationship. Am J Surg Pathol 2016, 40 (12), e94-e102.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Warren, J. L.; Klabunde, C. N.; Schrag, D.; Bach, P. B.; Riley, G. F., Overview of the SEER-Medicare data: content, research applications, and generalizability to the United States elderly population. Med Care 2002, 40 (8 Suppl), IV-3-18.

  3. Bilimoria, K. Y.; Bentrem, D. J.; Ko, C. Y.; Ritchey, J.; Stewart, A. K.; Winchester, D. P.; Talamonti, M. S., Validation of the 6th edition AJCC Pancreatic Cancer Staging System: report from the National Cancer Database. Cancer 2007, 110 (4), 738-44.

    Article  PubMed  Google Scholar 

  4. Merkow, R. P.; Rademaker, A. W.; Bilimoria, K. Y., Practical Guide to Surgical Data Sets: National Cancer Database (NCDB). JAMA Surg 2018.

  5. Siegel, R. L.; Miller, K. D.; Jemal, A., Cancer statistics, 2019. CA Cancer J Clin 2019, 69 (1), 7-34.

    Article  PubMed  Google Scholar 

  6. DeSantis, C. E.; Miller, K. D.; Goding Sauer, A.; Jemal, A.; Siegel, R. L., Cancer statistics for African Americans, 2019. CA Cancer J Clin 2019, 69 (3), 211-233.

    Article  PubMed  Google Scholar 

  7. Miller, K. D.; Nogueira, L.; Mariotto, A. B.; Rowland, J. H.; Yabroff, K. R.; Alfano, C. M.; Jemal, A.; Kramer, J. L.; Siegel, R. L., Cancer treatment and survivorship statistics, 2019. CA Cancer J Clin 2019, 69 (5), 363-385.

    Article  PubMed  Google Scholar 

  8. Hanna, N.; Sun, M.; Meyer, C. P.; Nguyen, P. L.; Pal, S. K.; Chang, S. L.; de Velasco, G.; Trinh, Q. D.; Choueiri, T. K., Survival Analyses of Patients With Metastatic Renal Cancer Treated With Targeted Therapy With or Without Cytoreductive Nephrectomy: A National Cancer Data Base Study. J Clin Oncol 2016, 34 (27), 3267-75.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Sun, Z.; Adam, M. A.; Kim, J.; Nussbaum, D. P.; Benrashid, E.; Mantyh, C. R.; Migaly, J., Determining the Optimal Timing for Initiation of Adjuvant Chemotherapy After Resection for Stage II and III Colon Cancer. Dis Colon Rectum 2016, 59 (2), 87-93.

    Article  PubMed  Google Scholar 

  10. Salazar, M. C.; Rosen, J. E.; Wang, Z.; Arnold, B. N.; Thomas, D. C.; Herbst, R. S.; Kim, A. W.; Detterbeck, F. C.; Blasberg, J. D.; Boffa, D. J., Association of Delayed Adjuvant Chemotherapy With Survival After Lung Cancer Surgery. JAMA Oncol 2017, 3 (5), 610-619.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Kamarajah, S. K.; Sonnenday, C. J.; Cho, C. S.; Frankel, T. L.; Bednar, F.; Lawrence, T. S.; Nathan, H., Association of Adjuvant Radiotherapy With Survival After Margin-negative Resection of Pancreatic Ductal Adenocarcinoma: A Propensity-matched National Cancer Database (NCDB) Analysis. Ann Surg 2019.

  12. Yang, C. J.; Kumar, A.; Klapper, J. A.; Hartwig, M. G.; Tong, B. C.; Harpole, D. H., Jr.; Berry, M. F.; D'Amico, T. A., A National Analysis of Long-term Survival Following Thoracoscopic Versus Open Lobectomy for Stage I Non-small-cell Lung Cancer. Ann Surg 2019, 269 (1), 163-171.

    Article  PubMed  Google Scholar 

  13. Skancke, M.; Schoolfield, C.; Umapathi, B.; Amdur, R.; Brody, F.; Obias, V., Minimally Invasive Surgery for Rectal Adenocarcinoma Shows Promising Outcomes Compared to Laparotomy, a National Cancer Database Observational Analysis. J Laparoendosc Adv Surg Tech A 2019, 29 (2), 218-224.

    Article  PubMed  Google Scholar 

  14. Gabriel, E.; Thirunavukarasu, P.; Attwood, K.; Nurkin, S. J., National disparities in minimally invasive surgery for pancreatic tumors. Surg Endosc 2017, 31 (1), 398-409.

    Article  PubMed  Google Scholar 

  15. Klein, G.; Wang, H.; Elshabrawy, A.; Nashawi, M.; Gourley, E.; Liss, M.; Kaushik, D.; Wu, S.; Rodriguez, R.; Mansour, A. M., Analyzing National Incidences and Predictors of Open Conversion During Minimally Invasive Partial Nephrectomy for cT1 Renal Masses. J Endourol 2020.

  16. Hoehn, R. S.; Nassour, I.; Adam, M. A.; Winters, S.; Paniccia, A.; Zureikat, A. H., National Trends in Robotic Pancreas Surgery. J Gastrointest Surg 2020.

  17. Thirunavukarasu, P.; Gabriel, E.; Attwood, K.; Kukar, M.; Hochwald, S. N.; Nurkin, S. J., Nationwide analysis of short-term surgical outcomes of minimally invasive esophagectomy for malignancy. Int J Surg 2016, 25, 69-75.

    Article  PubMed  Google Scholar 

  18. Arnold, B. N.; Thomas, D. C.; Narayan, R.; Blasberg, J. D.; Detterbeck, F. C.; Boffa, D. J.; Kim, A. W., Robotic-Assisted Lobectomies in the National Cancer Database. J Am Coll Surg 2018, 226 (6), 1052-1062 e15.

    Article  PubMed  Google Scholar 

  19. Raoof, M.; Nota, C.; Melstrom, L. G.; Warner, S. G.; Woo, Y.; Singh, G.; Fong, Y., Oncologic outcomes after robot-assisted versus laparoscopic distal pancreatectomy: Analysis of the National Cancer Database. J Surg Oncol 2018, 118 (4), 651-656.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Birkmeyer, J. D.; Siewers, A. E.; Finlayson, E. V.; Stukel, T. A.; Lucas, F. L.; Batista, I.; Welch, H. G.; Wennberg, D. E., Hospital volume and surgical mortality in the United States. N Engl J Med 2002, 346 (15), 1128-37.

    Article  PubMed  Google Scholar 

  21. Ghaferi, A. A.; Birkmeyer, J. D.; Dimick, J. B., Hospital volume and failure to rescue with high-risk surgery. Med Care 2011, 49 (12), 1076-81.

    Article  PubMed  Google Scholar 

  22. Potosky, A. L.; Warren, J. L.; Riedel, E. R.; Klabunde, C. N.; Earle, C. C.; Begg, C. B., Measuring complications of cancer treatment using the SEER-Medicare data. Med Care 2002, 40 (8 Suppl), IV-62-8.

  23. Zhu, P.; Du, X. L.; Zhu, J. J.; Esquenazi, Y., Improved survival of glioblastoma patients treated at academic and high-volume facilities: a hospital-based study from the National Cancer Database. J Neurosurg 2019, 132 (2), 491-502.

    Article  PubMed  Google Scholar 

  24. Joshi, S. S.; Handorf, E. A.; Zibelman, M.; Plimack, E. R.; Uzzo, R. G.; Kutikov, A.; Smaldone, M. C.; Geynisman, D. M., Treatment Facility Volume and Survival in Patients with Metastatic Renal Cell Carcinoma: A Registry-based Analysis. Eur Urol 2018, 74 (3), 387-393.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Chen, Y. W.; Mahal, B. A.; Muralidhar, V.; Nezolosky, M.; Beard, C. J.; Den, R. B.; Feng, F. Y.; Hoffman, K. E.; Martin, N. E.; Orio, P. F.; Nguyen, P. L., Association Between Treatment at a High-Volume Facility and Improved Survival for Radiation-Treated Men With High-Risk Prostate Cancer. Int J Radiat Oncol Biol Phys 2016, 94 (4), 683-90.

    Article  PubMed  Google Scholar 

  26. Kaltenmeier, C.; Shen, C.; Medich, D. S.; Geller, D. A.; Bartlett, D. L.; Tsung, A.; Tohme, S., Time to Surgery and Colon Cancer Survival in the United States. Ann Surg 2019.

  27. Bleicher, R. J.; Ruth, K.; Sigurdson, E. R.; Beck, J. R.; Ross, E.; Wong, Y. N.; Patel, S. A.; Boraas, M.; Chang, E. I.; Topham, N. S.; Egleston, B. L., Time to Surgery and Breast Cancer Survival in the United States. JAMA Oncol 2016, 2 (3), 330-9.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Samson, P.; Patel, A.; Garrett, T.; Crabtree, T.; Kreisel, D.; Krupnick, A. S.; Patterson, G. A.; Broderick, S.; Meyers, B. F.; Puri, V., Effects of Delayed Surgical Resection on Short-Term and Long-Term Outcomes in Clinical Stage I Non-Small Cell Lung Cancer. Ann Thorac Surg 2015, 99 (6), 1906-12; discussion 1913.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Rygalski, C. J.; Zhao, S.; Eskander, A.; Zhan, K. Y.; Mroz, E. A.; Brock, G.; Silverman, D. A.; Blakaj, D.; Bonomi, M. R.; Carrau, R. L.; Old, M. O.; Rocco, J. W.; Seim, N. B.; Puram, S. V.; Kang, S. Y., Time to Surgery and Survival in Head and Neck Cancer. Ann Surg Oncol 2020.

  30. Kompelli, A. R.; Li, H.; Neskey, D. M., Impact of Delay in Treatment Initiation on Overall Survival in Laryngeal Cancers. Otolaryngol Head Neck Surg 2019, 160 (4), 651-657.

    Article  PubMed  Google Scholar 

  31. Wong, S. L.; Revels, S. L.; Yin, H.; Stewart, A. K.; McVeigh, A.; Banerjee, M.; Birkmeyer, J. D., Variation in hospital mortality rates with inpatient cancer surgery. Ann Surg 2015, 261 (4), 632-6.

    Article  PubMed  Google Scholar 

  32. Wright, J. D.; Huang, Y.; Ananth, C. V.; Tergas, A. I.; Duffy, C.; Deutsch, I.; Burke, W. M.; Hou, J. Y.; Neugut, A. I.; Hershman, D. L., Influence of treatment center and hospital volume on survival for locally advanced cervical cancer. Gynecol Oncol 2015, 139 (3), 506-12.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Chapman, B. C.; Paniccia, A.; Hosokawa, P. W.; Henderson, W. G.; Overbey, D. M.; Messersmith, W.; McCarter, M. D.; Gleisner, A.; Edil, B. H.; Schulick, R. D.; Gajdos, C., Impact of Facility Type and Surgical Volume on 10-Year Survival in Patients Undergoing Hepatic Resection for Hepatocellular Carcinoma. J Am Coll Surg 2017, 224 (3), 362-372.

    Article  PubMed  Google Scholar 

  34. Sheetz, K. H.; Dimick, J. B.; Ghaferi, A. A., Impact of Hospital Characteristics on Failure to Rescue Following Major Surgery. Ann Surg 2016, 263 (4), 692-7.

    Article  PubMed  Google Scholar 

  35. Ghaferi, A. A.; Birkmeyer, J. D.; Dimick, J. B., Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg 2009, 250 (6), 1029-34.

    Article  PubMed  Google Scholar 

  36. McMurry, T. L.; Shah, P. M.; Samson, P.; Robinson, C. G.; Kozower, B. D., Treatment of stage I non-small cell lung cancer: What's trending? J Thorac Cardiovasc Surg 2017, 154 (3), 1080-1087.

    Article  PubMed  Google Scholar 

  37. Bilimoria, K. Y.; Bentrem, D. J.; Ko, C. Y.; Stewart, A. K.; Winchester, D. P.; Talamonti, M. S., National failure to operate on early stage pancreatic cancer. Ann Surg 2007, 246 (2), 173-80.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Luryi, A. L.; Chen, M. M.; Mehra, S.; Roman, S. A.; Sosa, J. A.; Judson, B. L., Hospital readmission and 30-day mortality after surgery for oral cavity cancer: Analysis of 21,681 cases. Head Neck 2016, 38 Suppl 1, E221-6.

    Article  PubMed  Google Scholar 

  39. Wegner, R. E.; Verma, V.; Hasan, S.; Schiffman, S.; Thakkar, S.; Horne, Z. D.; Kulkarni, A.; Williams, H. K.; Monga, D.; Finley, G.; Kirichenko, A. V., Incidence and risk factors for post-operative mortality, hospitalization, and readmission rates following pancreatic cancer resection. J Gastrointest Oncol 2019, 10 (6), 1080-1093.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Barber, E. L.; Rossi, E. C.; Gehrig, P. A., Surgical readmission and survival in women with ovarian cancer: Are short-term quality metrics incentivizing decreased long-term survival? Gynecol Oncol 2017, 147 (3), 607-611.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Kamarajah, S. K.; Burns, W. R.; Frankel, T. L.; Cho, C. S.; Nathan, H., Validation of the American Joint Commission on Cancer (AJCC) 8th Edition Staging System for Patients with Pancreatic Adenocarcinoma: A Surveillance, Epidemiology and End Results (SEER) Analysis. Ann Surg Oncol 2017, 24 (7), 2023-2030.

    Article  PubMed  Google Scholar 

  42. Kamarajah, S. K.; Frankel, T. L.; Sonnenday, C.; Cho, C. S.; Nathan, H., Critical evaluation of the American Joint Commission on Cancer (AJCC) 8th edition staging system for patients with Hepatocellular Carcinoma (HCC): A Surveillance, Epidemiology, End Results (SEER) analysis. J Surg Oncol 2018, 117 (4), 644-650.

    Article  CAS  PubMed  Google Scholar 

  43. Kamarajah, S. K., Evaluation of the AJCC 8th Edition Staging System for Pathologically Versus Clinically Staged Intrahepatic Cholangiocarcinoma (iCCA): a Time to Revisit a Dogma? A Surveillance, Epidemiology, and End Results (SEER) Analysis. J Gastrointest Cancer 2018.

  44. Fong, Y.; Wagman, L.; Gonen, M.; Crawford, J.; Reed, W.; Swanson, R.; Pan, C.; Ritchey, J.; Stewart, A.; Choti, M., Evidence-based gallbladder cancer staging: changing cancer staging by analysis of data from the National Cancer Database. Ann Surg 2006, 243 (6), 767-71; discussion 771-4.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Weksler, B.; Kennedy, K. F.; Sullivan, J. L., Using the National Cancer Database to create a scoring system that identifies patients with early-stage esophageal cancer at risk for nodal metastases. J Thorac Cardiovasc Surg 2017, 154 (5), 1787-1793.

    Article  PubMed  Google Scholar 

  46. Nathan, H.; Hyder, O.; Mayo, S. C.; Hirose, K.; Wolfgang, C. L.; Choti, M. A.; Pawlik, T. M., Surgical therapy for early hepatocellular carcinoma in the modern era: a 10-year SEER-medicare analysis. Ann Surg 2013, 258 (6), 1022-7.

    Article  PubMed  Google Scholar 

  47. Paniccia, A.; Hosokawa, P.; Henderson, W.; Schulick, R. D.; Edil, B. H.; McCarter, M. D.; Gajdos, C., Characteristics of 10-Year Survivors of Pancreatic Ductal Adenocarcinoma. JAMA Surg 2015, 150 (8), 701-10.

    Article  PubMed  Google Scholar 

  48. Little, R. J. A.; Rubin, D. B., Statistical analysis with missing data. 2nd ed. Hoboken NJ ed.; Wiley: 2002.

  49. Schafer, J. L., Multiple imputation: a primer. Stat Methods Med Res 1999, 8 (1), 3-15.

    Article  CAS  PubMed  Google Scholar 

  50. Nathan, H.; Pawlik, T. M.; Wolfgang, C. L.; Choti, M. A.; Cameron, J. L.; Schulick, R. D., Trends in survival after surgery for cholangiocarcinoma: a 30-year population-based SEER database analysis. J Gastrointest Surg 2007, 11 (11), 1488-96; discussion 1496-7.

    Article  PubMed  Google Scholar 

  51. Nathan, H.; Pawlik, T. M., Staging of intrahepatic cholangiocarcinoma. Curr Opin Gastroenterol 2010, 26 (3), 269-73.

    Article  CAS  PubMed  Google Scholar 

  52. Nathan, H.; Pawlik, T. M., Limitations of claims and registry data in surgical oncology research. Ann Surg Oncol 2008, 15 (2), 415-23.

    Article  PubMed  Google Scholar 

  53. de Jong, M. C.; Nathan, H.; Sotiropoulos, G. C.; Paul, A.; Alexandrescu, S.; Marques, H.; Pulitano, C.; Barroso, E.; Clary, B. M.; Aldrighetti, L.; Ferrone, C. R.; Zhu, A. X.; Bauer, T. W.; Walters, D. M.; Gamblin, T. C.; Nguyen, K. T.; Turley, R.; Popescu, I.; Hubert, C.; Meyer, S.; Schulick, R. D.; Choti, M. A.; Gigot, J. F.; Mentha, G.; Pawlik, T. M., Intrahepatic cholangiocarcinoma: an international multi-institutional analysis of prognostic factors and lymph node assessment. J Clin Oncol 2011, 29 (23), 3140-5.

    Article  PubMed  Google Scholar 

  54. Mantripragada, K. C.; Hamid, F.; Shafqat, H.; Olszewski, A. J., Adjuvant Therapy for Resected Gallbladder Cancer: Analysis of the National Cancer Data Base. J Natl Cancer Inst 2017, 109 (2).

  55. Mitin, T.; Enestvedt, C. K.; Jemal, A.; Sineshaw, H. M., Limited Use of Adjuvant Therapy in Patients With Resected Gallbladder Cancer Despite a Strong Association With Survival. J Natl Cancer Inst 2017, 109 (7).

  56. Delgado-Rodriguez, M.; Llorca, J., Bias. J Epidemiol Community Health 2004, 58 (8), 635-41.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Rubin, D. B., Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997, 127 (8 Pt 2), 757-63.

    Article  CAS  PubMed  Google Scholar 

  58. Braitman, L. E.; Rosenbaum, P. R., Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med 2002, 137 (8), 693-5.

    Article  PubMed  Google Scholar 

  59. Grose, E.; Wilson, S.; Barkun, J.; Bertens, K.; Martel, G.; Balaa, F.; Abou Khalil, J., Use of Propensity Score Methodology in Contemporary High-Impact Surgical Literature. J Am Coll Surg 2020, 230 (1), 101-112 e2.

    Article  PubMed  Google Scholar 

  60. Kamarajah, S. K.; Bednar, F.; Cho, C. S.; Nathan, H., Survival benefit with adjuvant radiotherapy after resection of distal cholangiocarcinoma: A propensity-matched National Cancer Database analysis. Cancer 2020.

  61. Mathes, T.; Buehn, S.; Prengel, P.; Pieper, D., Registry-based randomized controlled trials merged the strength of randomized controlled trails and observational studies and give rise to more pragmatic trials. J Clin Epidemiol 2018, 93, 120-127.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Surgery, Queen Elizabeth Hospital Birmingham, Birmingham, UK

    Sivesh K Kamarajah

  2. Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK

    Sivesh K Kamarajah

  3. Department of HPB and Transplant Surgery, The Freeman Hospital, Newcastle upon Tyne, Tyne and Wear, UK

    Sivesh K Kamarajah

  4. Department of Surgery, University of Michigan, Ann Arbor, MI, USA

    Hari Nathan

  5. University of Michigan, 2210A Taubman Health Care Center, 1500 E Medical Center Dr, SPC 5343, Ann Arbor, MI, 48109-5343, USA

    Hari Nathan

Authors
  1. Sivesh K Kamarajah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hari Nathan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hari Nathan.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kamarajah, S.K., Nathan, H. Strengths and Limitations of Registries in Surgical Oncology Research. J Gastrointest Surg 25, 2989–2996 (2021). https://doi.org/10.1007/s11605-021-05094-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11605-021-05094-y

Keywords

Search

Navigation