Skip to main content

Advertisement

SpringerLink
Log in

Quality of MBSAQIP data: bad luck, or lack of QA plan?

  • 2019 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

National clinical registries are commonly used in clinical research, quality improvement, and health policy. However, little is known about methodological challenges associated with these registry analyses that could limit their impact and compromise patient safety. This study examined the quality of Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MSBASQIP) data to assess its usability potential and improve data collection methodologies.

Methods

We developed a single flat file (n = 168,093) using five subsets (Main, BMI, Readmission, Reoperation, and Intervention) of the 2015 MBSAQIP Participant User Data File (PUF). Logic and validity tests included (1) individual profiles of patient’s body mass index (BMI) changes over time, (2) individual patient care pathways, and (3) correlation analysis between variable pairs associated with the same clinical encounters.

Results

8888 (5.3%) patients did not have postoperative weight/BMI data; 20% of patients had different units for preoperative and postoperative weights. Postoperative weight measurements ranged between − 71 and 132% of preoperative weight. There were 325 (3.7%) hospital readmissions reported on the day of or day after MBS. The self-reporting of “emergency” vs. “planned” interventions did not correlate with the type of procedure and its indication. Up to 20% of data could potentially be unused for analysis due to data quality issues.

Conclusions

Our analysis revealed various data quality issues in the 2015 MBSAQIP PUF related to completeness, accuracy, and consistency. Since information on where the surgery was performed is lacking, it is not possible to conclude whether these issues represent data errors, patient outliers, or inappropriate care. Including automated data checks and biomedical informatics oversight, standardized coding for complications, additional de-identified facility and provider information, and training/mentorship opportunities in data informatics for all researchers who get access to the data have been shown to be effective in improving data quality and minimizing patient safety concerns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Shiloach M, Frencher SK Jr, Steeger JE, Rowell KS, Bartzokis K, Tomeh MG, Richards KE, Ko CY, Hall BL (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 210:6–16

    PubMed  Google Scholar 

  2. Telem DA, Dimick JB (2018) Practical guide to surgical data sets: Metabolic and Bariatric Surgery Accreditation and Quality Program (MBSAQIP). JAMA Surg 153:766–767

    PubMed  Google Scholar 

  3. Brethauer SA, Kim J, el Chaar M, Papasavas P, Eisenberg D, Rogers A, Ballem N, Kligman M, Kothari S, Committee ACI (2015) Standardized outcomes reporting in metabolic and bariatric surgery. Surg Obes Relat Dis 11:489–506

    Google Scholar 

  4. Ehrenstein V, Nielsen H, Pedersen AB, Johnsen SP, Pedersen L (2017) Clinical epidemiology in the era of big data: new opportunities, familiar challenges. Clin Epidemiol 9:245–250

    PubMed  PubMed Central  Google Scholar 

  5. Kruse CS, Goswamy R, Raval Y, Marawi S (2016) Challenges and opportunities of big data in health care: a systematic review. JMIR Med Inform 4:e38

    PubMed  PubMed Central  Google Scholar 

  6. Ardila-Gatas J, Sharma G, Lloyd SJ, Khorgami Z, Tu C, Schauer PR, Brethauer SA, Aminian A (2019) A nationwide safety analysis of discharge on the first postoperative day after bariatric surgery in selected patients. Obes Surg 29:15–22

    PubMed  Google Scholar 

  7. Berger ER, Clements RH, Morton JM, Huffman KM, Wolfe BM, Nguyen NT, Ko CY, Hutter MM (2016) The impact of different surgical techniques on outcomes in laparoscopic sleeve gastrectomies: the first report from the metabolic and bariatric surgery accreditation and quality improvement program (MBSAQIP). Ann Surg 264:464–473

    PubMed  Google Scholar 

  8. Chaar ME, Lundberg P, Stoltzfus J (2018) Thirty-day outcomes of sleeve gastrectomy versus Roux-en-Y gastric bypass: first report based on Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Surg Obes Relat Dis 14:545–551

    PubMed  Google Scholar 

  9. Clapp B, Hahn J, Dodoo C, Guerra A, de la Rosa E, Tyroch A (2018) Evaluation of the rate of marginal ulcer formation after bariatric surgery using the MBSAQIP database. Surg Endosc. https://doi.org/10.1007/s00464-018-6468-6

    Article  PubMed  Google Scholar 

  10. Daigle CR, Brethauer SA, Tu C, Petrick AT, Morton JM, Schauer PR, Aminian A (2018) Which postoperative complications matter most after bariatric surgery? Prioritizing quality improvement efforts to improve national outcomes. Surg Obes Relat Dis 14:652–657

    PubMed  Google Scholar 

  11. Dang JT, Switzer N, Delisle M, Laffin M, Gill R, Birch DW, Karmali S (2019) Predicting venous thromboembolism following laparoscopic bariatric surgery: development of the BariClot tool using the MBSAQIP database. Surg Endosc 33:821–831

    PubMed  Google Scholar 

  12. Dang JT, Switzer NJ, Sun WYL, Raghavji F, Birch DW, Karmali S (2018) Evaluating the safety of intragastric balloon: an analysis of the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1340–1347

    PubMed  Google Scholar 

  13. Demeusy A, Sill A, Averbach A (2018) Current role of staple line reinforcement in 30-day outcomes of primary laparoscopic sleeve gastrectomy: an analysis of MBSAQIP data, 2015-2016 PUF. Surg Obes Relat Dis 14:1454–1461

    PubMed  Google Scholar 

  14. El Chaar M, Stoltzfus J (2018) Assessment of sleeve gastrectomy surgical technique: first look at 30-day outcomes based on the MBSAQIP database. J Am Coll Surg 227:564–572

    PubMed  Google Scholar 

  15. El Chaar M, Stoltzfus J, Melitics M, Claros L, Zeido A (2018) 30-Day outcomes of revisional bariatric stapling procedures: first report based on MBSAQIP data registry. Obes Surg 28:2233–2240

    PubMed  Google Scholar 

  16. Fazl Alizadeh R, Li S, Inaba CS, Dinicu AI, Hinojosa MW, Smith BR, Stamos MJ, Nguyen NT (2019) Robotic versus laparoscopic sleeve gastrectomy: a MBSAQIP analysis. Surg Endosc 33:917–922

    PubMed  Google Scholar 

  17. Feng X, Andalib A, Brethauer SA, Schauer PR, Aminian A (2018) How safe is bariatric surgery in patients with class I obesity (body mass index 30–35 kg/m(2))? Surg Obes Relat Dis 15:253–260

    PubMed  Google Scholar 

  18. Gondal AB, Hsu CH, Khoubyari R, Ghaderi I (2019) Development of a bariatric surgery specific risk assessment tool for perioperative myocardial infarction. Surg Obes Relat Dis 15:462–468

    PubMed  Google Scholar 

  19. Haskins IN, Ju T, Whitlock AE, Rivas L, Amdur RL, Lin PP, Vaziri K (2018) Older age confers a higher risk of 30-day morbidity and mortality following laparoscopic bariatric surgery: an analysis of the metabolic and bariatric surgery quality improvement program. Obes Surg 28:2745–2752

    PubMed  Google Scholar 

  20. Inaba CS, Koh CY, Sujatha-Bhaskar S, Zhang L, Nguyen NT (2018) Same-day discharge after laparoscopic Roux-en-Y Gastric Bypass: an analysis of the metabolic and bariatric surgery accreditation and quality improvement program database. J Am Coll Surg 226:868–873

    PubMed  PubMed Central  Google Scholar 

  21. Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Safety of laparoscopic sleeve gastrectomy and Roux-en-Y gastric bypass in elderly patients—analysis of the MBSAQIP. Surg Obes Relat Dis 14:1276–1282

    PubMed  Google Scholar 

  22. Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Application of HARM score to measure surgical quality and outcomes in bariatric patients. Obes Surg 28:2815–2819

    PubMed  PubMed Central  Google Scholar 

  23. Janik MR, Rogula TG, Mustafa RR, Alhaj Saleh A, Khaitan L (2019) Safety of revision sleeve gastrectomy compared to Roux-Y Gastric bypass after failed gastric banding: analysis of the MBSAQIP. Ann Surg 269:299–303

    PubMed  Google Scholar 

  24. Jurgensen JA, Reidt W, Kellogg T, Mundi M, Shah M, Collazo Clavell ML (2019) Impact of patient attrition from bariatric surgery practice on clinical outcomes. Obes Surg 29:579–584

    PubMed  Google Scholar 

  25. Kanters AE, Shubeck SP, Varban OA, Dimick JB, Telem DA (2018) Incidence and efficacy of stent placement in leak management after bariatric surgery: an MBSAQIP analysis. Ann Surg. https://doi.org/10.1097/SLA.0000000000003023

    Article  Google Scholar 

  26. Kumar SB, Hamilton BC, Wood SG, Rogers SJ, Carter JT, Lin MY (2018) Is laparoscopic sleeve gastrectomy safer than laparoscopic gastric bypass? a comparison of 30-day complications using the MBSAQIP data registry. Surg Obes Relat Dis 14:264–269

    PubMed  Google Scholar 

  27. Lundberg PW, Stoltzfus J, El Chaar M (2019) 30-day outcomes of robot-assisted versus conventional laparoscopic sleeve gastrectomy: first analysis based on MBSAQIP. Surg Obes Relat Dis 15:1–7

    PubMed  Google Scholar 

  28. Lundberg PW, Wolfe S, Seaone J, Stoltzfus J, Claros L, El Chaar M (2018) Robotic gastric bypass is getting better: first results from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1240–1245

    PubMed  Google Scholar 

  29. Rahman U, Docimo S, Pryor AD, Bates A, Obeid NR, Spaniolas K (2018) Routine contrast imaging after bariatric surgery and the effect on hospital length of stay. Surg Obes Relat Dis 14:517–520

    PubMed  Google Scholar 

  30. Sebastian R, Howell MH, Chang KH, Adrales G, Magnuson T, Schweitzer M, Nguyen H (2018) Robot-assisted versus laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy: a propensity score-matched comparative analysis using the 2015–2016 MBSAQIP database. Surg Endosc 33:1600–1612

    PubMed  Google Scholar 

  31. Sharma G, Strong AT, Tu C, Brethauer SA, Schauer PR, Aminian A (2018) Robotic platform for gastric bypass is associated with more resource utilization: an analysis of MBSAQIP dataset. Surg Obes Relat Dis 14:304–310

    PubMed  Google Scholar 

  32. Zafar SN, Miller K, Felton J, Wise ES, Kligman M (2019) Postoperative bleeding after laparoscopic Roux en Y gastric bypass: predictors and consequences. Surg Endosc 33:272–280

    PubMed  Google Scholar 

  33. Zafar SN, Shockcor N, Prasad N, Wise ES, Jackson HT, Kligman M (2019) Laparoscopic sleeve gastrectomy has a lower risk of postoperative bleeding than laparoscopic Roux-en-Y gastric bypass. Surg Laparosc Endosc Percutan Tech 29:53–57

    PubMed  Google Scholar 

  34. Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (Published January 2017) User Guide for the 2015 Participant Use Data File (PUF)

  35. Department of Health and Human Services Office for Human Research Protections (2018) 45 CFR 46-Protection of Human Subjects, 2018 Requirements: 104 Exempt research. Department of Health and Human Services Office for Human Research Protections, Rockville

    Google Scholar 

  36. Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (October 2016) Standards Manual version 2.0. Resources for Optimal Care of the Metabolic and Bariatric Surgery Patient. 2016

  37. American Society for Metabolic and Bariatric Surgery (2018) Who is a Candidate for Bariatric Surgery?. American Society for Metabolic and Bariatric Surgery, Gainsville

    Google Scholar 

  38. Fryar CD, Kruszon-Moran D, Gu Q, Ogden CL (2018) Mean body weight, height, waist circumference, and body mass index among adults: United States, 1999–2000 through 2015–2016. Natl Health Stat Report 1:16

    Google Scholar 

  39. Aminian A, Chang J, Brethauer SA, Kim JJ, American Society for M, Bariatric Surgery Clinical Issues C (2018) ASMBS updated position statement on bariatric surgery in class I obesity (BMI 30-35 kg/m(2)). Surg Obes Relat Dis 14:1071–1087

    PubMed  Google Scholar 

  40. AHIMA e-HIM Work Group on EHR Data Content (2006) Data standard time: data content standardization and the HIM role. J AHIMA 77:26–32

    Google Scholar 

  41. Amster A, Jentzsch J, Pasupuleti H, Subramanian KG (2015) Completeness, accuracy, and computability of National Quality Forum-specified eMeasures. J Am Med Inform Assoc 22:409–416

    PubMed  Google Scholar 

  42. Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250:363–376

    PubMed  Google Scholar 

  43. Huser V, Kahn MG, Brown JS, Gouripeddi R (2018) Methods for examining data quality in healthcare integrated data repositories. Pac Symp Biocomput 23:628–633

    PubMed  Google Scholar 

  44. Ko C (2012) The critical importance of good data to improving quality. Patient Safety & Quality Healthcare

  45. Kupzyk KA, Cohen MZ (2015) Data validation and other strategies for data entry. West J Nurs Res 37:546–556

    PubMed  Google Scholar 

  46. Marc DT, Beattie J, Herasevich V, Gatewood L, Zhang R (2016) Assessing metadata quality of a federally sponsored health data repository. AMIA Annu Symp Proc 2016:864–873

    PubMed  Google Scholar 

  47. Nouraei SA, Hudovsky A, Virk JS, Saleh HA (2017) Impact of developing a multidisciplinary coded dataset standard on administrative data accuracy for septoplasty, septorhinoplasty and nasal trauma surgery. J Laryngol Otol 131:341–346

    CAS  PubMed  Google Scholar 

  48. Raval MV, Pawlik TM (2018) Practical guide to surgical data sets: national surgical quality improvement program (NSQIP) and Pediatric NSQIP. JAMA Surg 153:764–765

    PubMed  Google Scholar 

  49. Royal College of Surgeons of England (2018) Improving surgical data. Royal College of Surgeons of England, London

    Google Scholar 

  50. American College of Surgeons National Surgical Quality Improvement Program (2017) User guide for the 2016 ACS NSQIP procedure targeted participant use data file (PUF). Chicago, IL

  51. Research Data Assistance Center (ResDAC) (2019) Learn about CMS data. University of Minnesota School of Public Health, Minneapolis

    Google Scholar 

  52. Ferguson TB Jr, Hammill BG, Peterson ED, DeLong ER, Grover FL, Committee STSND (2002) A decade of change-risk profiles and outcomes for isolated coronary artery bypass grafting procedures, 1990–1999: a report from the STS National Database Committee and the Duke Clinical Research Institute. Society of Thoracic Surgeons. Ann Thorac Surg 73:480–489 (discussion 489–490)

    PubMed  Google Scholar 

  53. Welke KF, Ferguson TB Jr, Coombs LP, Dokholyan RS, Murray CJ, Schrader MA, Peterson ED (2004) Validity of the society of thoracic surgeons national adult cardiac surgery database. Ann Thorac Surg 77:1137–1139

    PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Monami Majumder and Evan Kessler for their assistance with the preparation of this manuscript.

Funding

This manuscript was in part funded by Surgical Outcomes and Research (UB SOAR) program at the Department of Surgery, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.

Author information

Authors and Affiliations

  1. Division of Health Services Policy and Practice, Department of Epidemiology and Environmental Health, School of Public Health and Health Professions, University at Buffalo, Buffalo, NY, USA

    K. Noyes

  2. Department of Surgery, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, NY, USA

    K. Noyes, A. A. Myneni, S. D. Schwaitzberg & A. B. Hoffman

Authors
  1. K. Noyes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. A. Myneni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. D. Schwaitzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. B. Hoffman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Noyes.

Ethics declarations

Disclosures

The earlier version of the study was presented at the 2019 SAGES Annual Meeting. Dr. Hoffman is a paid consultant for Ethicon US, LLC (not related to the study). Dr. Steven Schwaitzberg is a paid consultant for Nu View Surgical, Acuity Bio, Activ Surgical, Human Extensions, Levitra Magnetics and Arch Therapeutics (not related to the study). K. Noyes and A. A. Myneni declare that they have no conflict of interest or financial ties to disclose. The American College of Surgeons Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program and the centers participating in the ACS MBSAQIP are the source of the data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noyes, K., Myneni, A.A., Schwaitzberg, S.D. et al. Quality of MBSAQIP data: bad luck, or lack of QA plan?. Surg Endosc 34, 973–980 (2020). https://doi.org/10.1007/s00464-019-06884-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-019-06884-x

Keywords

Search

Navigation