Advertisement

Quality of MBSAQIP data: bad luck, or lack of QA plan?

Abstract

Background

National clinical registries are commonly used in clinical research, quality improvement, and health policy. However, little is known about methodological challenges associated with these registry analyses that could limit their impact and compromise patient safety. This study examined the quality of Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MSBASQIP) data to assess its usability potential and improve data collection methodologies.

Methods

We developed a single flat file (n = 168,093) using five subsets (Main, BMI, Readmission, Reoperation, and Intervention) of the 2015 MBSAQIP Participant User Data File (PUF). Logic and validity tests included (1) individual profiles of patient’s body mass index (BMI) changes over time, (2) individual patient care pathways, and (3) correlation analysis between variable pairs associated with the same clinical encounters.

Results

8888 (5.3%) patients did not have postoperative weight/BMI data; 20% of patients had different units for preoperative and postoperative weights. Postoperative weight measurements ranged between − 71 and 132% of preoperative weight. There were 325 (3.7%) hospital readmissions reported on the day of or day after MBS. The self-reporting of “emergency” vs. “planned” interventions did not correlate with the type of procedure and its indication. Up to 20% of data could potentially be unused for analysis due to data quality issues.

Conclusions

Our analysis revealed various data quality issues in the 2015 MBSAQIP PUF related to completeness, accuracy, and consistency. Since information on where the surgery was performed is lacking, it is not possible to conclude whether these issues represent data errors, patient outliers, or inappropriate care. Including automated data checks and biomedical informatics oversight, standardized coding for complications, additional de-identified facility and provider information, and training/mentorship opportunities in data informatics for all researchers who get access to the data have been shown to be effective in improving data quality and minimizing patient safety concerns.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Shiloach M, Frencher SK Jr, Steeger JE, Rowell KS, Bartzokis K, Tomeh MG, Richards KE, Ko CY, Hall BL (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 210:6–16

  2. 2.

    Telem DA, Dimick JB (2018) Practical guide to surgical data sets: Metabolic and Bariatric Surgery Accreditation and Quality Program (MBSAQIP). JAMA Surg 153:766–767

  3. 3.

    Brethauer SA, Kim J, el Chaar M, Papasavas P, Eisenberg D, Rogers A, Ballem N, Kligman M, Kothari S, Committee ACI (2015) Standardized outcomes reporting in metabolic and bariatric surgery. Surg Obes Relat Dis 11:489–506

  4. 4.

    Ehrenstein V, Nielsen H, Pedersen AB, Johnsen SP, Pedersen L (2017) Clinical epidemiology in the era of big data: new opportunities, familiar challenges. Clin Epidemiol 9:245–250

  5. 5.

    Kruse CS, Goswamy R, Raval Y, Marawi S (2016) Challenges and opportunities of big data in health care: a systematic review. JMIR Med Inform 4:e38

  6. 6.

    Ardila-Gatas J, Sharma G, Lloyd SJ, Khorgami Z, Tu C, Schauer PR, Brethauer SA, Aminian A (2019) A nationwide safety analysis of discharge on the first postoperative day after bariatric surgery in selected patients. Obes Surg 29:15–22

  7. 7.

    Berger ER, Clements RH, Morton JM, Huffman KM, Wolfe BM, Nguyen NT, Ko CY, Hutter MM (2016) The impact of different surgical techniques on outcomes in laparoscopic sleeve gastrectomies: the first report from the metabolic and bariatric surgery accreditation and quality improvement program (MBSAQIP). Ann Surg 264:464–473

  8. 8.

    Chaar ME, Lundberg P, Stoltzfus J (2018) Thirty-day outcomes of sleeve gastrectomy versus Roux-en-Y gastric bypass: first report based on Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Surg Obes Relat Dis 14:545–551

  9. 9.

    Clapp B, Hahn J, Dodoo C, Guerra A, de la Rosa E, Tyroch A (2018) Evaluation of the rate of marginal ulcer formation after bariatric surgery using the MBSAQIP database. Surg Endosc. https://doi.org/10.1007/s00464-018-6468-6

  10. 10.

    Daigle CR, Brethauer SA, Tu C, Petrick AT, Morton JM, Schauer PR, Aminian A (2018) Which postoperative complications matter most after bariatric surgery? Prioritizing quality improvement efforts to improve national outcomes. Surg Obes Relat Dis 14:652–657

  11. 11.

    Dang JT, Switzer N, Delisle M, Laffin M, Gill R, Birch DW, Karmali S (2019) Predicting venous thromboembolism following laparoscopic bariatric surgery: development of the BariClot tool using the MBSAQIP database. Surg Endosc 33:821–831

  12. 12.

    Dang JT, Switzer NJ, Sun WYL, Raghavji F, Birch DW, Karmali S (2018) Evaluating the safety of intragastric balloon: an analysis of the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1340–1347

  13. 13.

    Demeusy A, Sill A, Averbach A (2018) Current role of staple line reinforcement in 30-day outcomes of primary laparoscopic sleeve gastrectomy: an analysis of MBSAQIP data, 2015-2016 PUF. Surg Obes Relat Dis 14:1454–1461

  14. 14.

    El Chaar M, Stoltzfus J (2018) Assessment of sleeve gastrectomy surgical technique: first look at 30-day outcomes based on the MBSAQIP database. J Am Coll Surg 227:564–572

  15. 15.

    El Chaar M, Stoltzfus J, Melitics M, Claros L, Zeido A (2018) 30-Day outcomes of revisional bariatric stapling procedures: first report based on MBSAQIP data registry. Obes Surg 28:2233–2240

  16. 16.

    Fazl Alizadeh R, Li S, Inaba CS, Dinicu AI, Hinojosa MW, Smith BR, Stamos MJ, Nguyen NT (2019) Robotic versus laparoscopic sleeve gastrectomy: a MBSAQIP analysis. Surg Endosc 33:917–922

  17. 17.

    Feng X, Andalib A, Brethauer SA, Schauer PR, Aminian A (2018) How safe is bariatric surgery in patients with class I obesity (body mass index 30–35 kg/m(2))? Surg Obes Relat Dis 15:253–260

  18. 18.

    Gondal AB, Hsu CH, Khoubyari R, Ghaderi I (2019) Development of a bariatric surgery specific risk assessment tool for perioperative myocardial infarction. Surg Obes Relat Dis 15:462–468

  19. 19.

    Haskins IN, Ju T, Whitlock AE, Rivas L, Amdur RL, Lin PP, Vaziri K (2018) Older age confers a higher risk of 30-day morbidity and mortality following laparoscopic bariatric surgery: an analysis of the metabolic and bariatric surgery quality improvement program. Obes Surg 28:2745–2752

  20. 20.

    Inaba CS, Koh CY, Sujatha-Bhaskar S, Zhang L, Nguyen NT (2018) Same-day discharge after laparoscopic Roux-en-Y Gastric Bypass: an analysis of the metabolic and bariatric surgery accreditation and quality improvement program database. J Am Coll Surg 226:868–873

  21. 21.

    Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Safety of laparoscopic sleeve gastrectomy and Roux-en-Y gastric bypass in elderly patients—analysis of the MBSAQIP. Surg Obes Relat Dis 14:1276–1282

  22. 22.

    Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Application of HARM score to measure surgical quality and outcomes in bariatric patients. Obes Surg 28:2815–2819

  23. 23.

    Janik MR, Rogula TG, Mustafa RR, Alhaj Saleh A, Khaitan L (2019) Safety of revision sleeve gastrectomy compared to Roux-Y Gastric bypass after failed gastric banding: analysis of the MBSAQIP. Ann Surg 269:299–303

  24. 24.

    Jurgensen JA, Reidt W, Kellogg T, Mundi M, Shah M, Collazo Clavell ML (2019) Impact of patient attrition from bariatric surgery practice on clinical outcomes. Obes Surg 29:579–584

  25. 25.

    Kanters AE, Shubeck SP, Varban OA, Dimick JB, Telem DA (2018) Incidence and efficacy of stent placement in leak management after bariatric surgery: an MBSAQIP analysis. Ann Surg. https://doi.org/10.1097/SLA.0000000000003023

  26. 26.

    Kumar SB, Hamilton BC, Wood SG, Rogers SJ, Carter JT, Lin MY (2018) Is laparoscopic sleeve gastrectomy safer than laparoscopic gastric bypass? a comparison of 30-day complications using the MBSAQIP data registry. Surg Obes Relat Dis 14:264–269

  27. 27.

    Lundberg PW, Stoltzfus J, El Chaar M (2019) 30-day outcomes of robot-assisted versus conventional laparoscopic sleeve gastrectomy: first analysis based on MBSAQIP. Surg Obes Relat Dis 15:1–7

  28. 28.

    Lundberg PW, Wolfe S, Seaone J, Stoltzfus J, Claros L, El Chaar M (2018) Robotic gastric bypass is getting better: first results from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1240–1245

  29. 29.

    Rahman U, Docimo S, Pryor AD, Bates A, Obeid NR, Spaniolas K (2018) Routine contrast imaging after bariatric surgery and the effect on hospital length of stay. Surg Obes Relat Dis 14:517–520

  30. 30.

    Sebastian R, Howell MH, Chang KH, Adrales G, Magnuson T, Schweitzer M, Nguyen H (2018) Robot-assisted versus laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy: a propensity score-matched comparative analysis using the 2015–2016 MBSAQIP database. Surg Endosc 33:1600–1612

  31. 31.

    Sharma G, Strong AT, Tu C, Brethauer SA, Schauer PR, Aminian A (2018) Robotic platform for gastric bypass is associated with more resource utilization: an analysis of MBSAQIP dataset. Surg Obes Relat Dis 14:304–310

  32. 32.

    Zafar SN, Miller K, Felton J, Wise ES, Kligman M (2019) Postoperative bleeding after laparoscopic Roux en Y gastric bypass: predictors and consequences. Surg Endosc 33:272–280

  33. 33.

    Zafar SN, Shockcor N, Prasad N, Wise ES, Jackson HT, Kligman M (2019) Laparoscopic sleeve gastrectomy has a lower risk of postoperative bleeding than laparoscopic Roux-en-Y gastric bypass. Surg Laparosc Endosc Percutan Tech 29:53–57

  34. 34.

    Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (Published January 2017) User Guide for the 2015 Participant Use Data File (PUF)

  35. 35.

    Department of Health and Human Services Office for Human Research Protections (2018) 45 CFR 46-Protection of Human Subjects, 2018 Requirements: 104 Exempt research. Department of Health and Human Services Office for Human Research Protections, Rockville

  36. 36.

    Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (October 2016) Standards Manual version 2.0. Resources for Optimal Care of the Metabolic and Bariatric Surgery Patient. 2016

  37. 37.

    American Society for Metabolic and Bariatric Surgery (2018) Who is a Candidate for Bariatric Surgery?. American Society for Metabolic and Bariatric Surgery, Gainsville

  38. 38.

    Fryar CD, Kruszon-Moran D, Gu Q, Ogden CL (2018) Mean body weight, height, waist circumference, and body mass index among adults: United States, 1999–2000 through 2015–2016. Natl Health Stat Report 1:16

  39. 39.

    Aminian A, Chang J, Brethauer SA, Kim JJ, American Society for M, Bariatric Surgery Clinical Issues C (2018) ASMBS updated position statement on bariatric surgery in class I obesity (BMI 30-35 kg/m(2)). Surg Obes Relat Dis 14:1071–1087

  40. 40.

    AHIMA e-HIM Work Group on EHR Data Content (2006) Data standard time: data content standardization and the HIM role. J AHIMA 77:26–32

  41. 41.

    Amster A, Jentzsch J, Pasupuleti H, Subramanian KG (2015) Completeness, accuracy, and computability of National Quality Forum-specified eMeasures. J Am Med Inform Assoc 22:409–416

  42. 42.

    Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250:363–376

  43. 43.

    Huser V, Kahn MG, Brown JS, Gouripeddi R (2018) Methods for examining data quality in healthcare integrated data repositories. Pac Symp Biocomput 23:628–633

  44. 44.

    Ko C (2012) The critical importance of good data to improving quality. Patient Safety & Quality Healthcare

  45. 45.

    Kupzyk KA, Cohen MZ (2015) Data validation and other strategies for data entry. West J Nurs Res 37:546–556

  46. 46.

    Marc DT, Beattie J, Herasevich V, Gatewood L, Zhang R (2016) Assessing metadata quality of a federally sponsored health data repository. AMIA Annu Symp Proc 2016:864–873

  47. 47.

    Nouraei SA, Hudovsky A, Virk JS, Saleh HA (2017) Impact of developing a multidisciplinary coded dataset standard on administrative data accuracy for septoplasty, septorhinoplasty and nasal trauma surgery. J Laryngol Otol 131:341–346

  48. 48.

    Raval MV, Pawlik TM (2018) Practical guide to surgical data sets: national surgical quality improvement program (NSQIP) and Pediatric NSQIP. JAMA Surg 153:764–765

  49. 49.

    Royal College of Surgeons of England (2018) Improving surgical data. Royal College of Surgeons of England, London

  50. 50.

    American College of Surgeons National Surgical Quality Improvement Program (2017) User guide for the 2016 ACS NSQIP procedure targeted participant use data file (PUF). Chicago, IL

  51. 51.

    Research Data Assistance Center (ResDAC) (2019) Learn about CMS data. University of Minnesota School of Public Health, Minneapolis

  52. 52.

    Ferguson TB Jr, Hammill BG, Peterson ED, DeLong ER, Grover FL, Committee STSND (2002) A decade of change-risk profiles and outcomes for isolated coronary artery bypass grafting procedures, 1990–1999: a report from the STS National Database Committee and the Duke Clinical Research Institute. Society of Thoracic Surgeons. Ann Thorac Surg 73:480–489 (discussion 489–490)

  53. 53.

    Welke KF, Ferguson TB Jr, Coombs LP, Dokholyan RS, Murray CJ, Schrader MA, Peterson ED (2004) Validity of the society of thoracic surgeons national adult cardiac surgery database. Ann Thorac Surg 77:1137–1139

Download references

Acknowledgements

The authors would like to thank Monami Majumder and Evan Kessler for their assistance with the preparation of this manuscript.

Funding

This manuscript was in part funded by Surgical Outcomes and Research (UB SOAR) program at the Department of Surgery, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.

Author information

Correspondence to K. Noyes.

Ethics declarations

Disclosures

The earlier version of the study was presented at the 2019 SAGES Annual Meeting. Dr. Hoffman is a paid consultant for Ethicon US, LLC (not related to the study). Dr. Steven Schwaitzberg is a paid consultant for Nu View Surgical, Acuity Bio, Activ Surgical, Human Extensions, Levitra Magnetics and Arch Therapeutics (not related to the study). K. Noyes and A. A. Myneni declare that they have no conflict of interest or financial ties to disclose. The American College of Surgeons Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program and the centers participating in the ACS MBSAQIP are the source of the data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Noyes, K., Myneni, A.A., Schwaitzberg, S.D. et al. Quality of MBSAQIP data: bad luck, or lack of QA plan?. Surg Endosc 34, 973–980 (2020). https://doi.org/10.1007/s00464-019-06884-x

Download citation

Keywords

  • MBSAQIP PUF
  • Data quality
  • Bariatric surgery
  • Surgical outcomes