World Journal of Surgery

, Volume 28, Issue 2, pp 193–200 | Cite as

Surgical mortality score: Risk management tool for auditing surgical performance

  • Vassilis G. Hadjianastassiou
  • Paris P. Tekkis
  • Jan D. Poloniecki
  • Manolis C. Gavalas
  • David R. Goldhill
Original Scientific Reports

Abstract

Existing methods of risk adjustment in surgical audit are complex and costly. The present study aimed to develop a simple risk stratification score for mortality and a robust audit tool using the existing resources of the hospital Patient Administration System (PAS) database. This was an observational study for all patients undergoing surgical procedures over a two-year period, at a London university hospital. Logistic regression analysis was used to determine predictive factors of in-hospital mortality, the study outcome. Odds ratios were used as weights in the derivation of a simple risk-stratification model—the Surgical Mortality Score (SMS). Observed-to-expected mortality risk ratios were calculated for application of the SMS model insurgical audit. There were 11,089 eligible cases, under five surgical specialties (maxillofacial, orthopedic, renal transplant/dialysis, general, and neurosurgery). Incomplete data were 3.7% of the total, with no evidence of systematic underreporting. The SMS model was well calibrated [Hosmer-Lemeshow C-statistic: development set (3.432, p = 0.33), validation set (6.359, p = 0.10) with a high discriminant ability (ROC areas: development set [0.837, S.E. = 0.013] validation set [0.816, S.E. = 0.016]). Subgroup analyses confirmed that the model can be used by the individual specialties for both elective and emergency cases. The SMS is an accurate risk-stratification model derived from existing database resources. It is simple to apply as a risk-management, screening tool to detect aberrations from expected surgical outcomes and to assist in surgical audit.

Résumé

Les techniques d’ajustement actuellement utilisées pour les scores de gravité en chirurgie sont complexes et coûteuses. Le but de cette étude a été de développer, à partir de la banque de données existante dans notre Hôpital, le système «Patient Administration System (PAS)», un score de stratification de risque de mortalité simple afin d’obtenir un outil d’audit robuste. D’après une période d’observation de deux ans pour tous les patients ayant eu une intervention chirurgicale dans un Hôpital universitaire de Londres, on a déterminé par analyse de régression logistique les facteurs prédictifs de mortalité hospitalière. Les rapports de côte ont été calculés et utilisés pour pondérer la réalisation d’un modèle simple de stratification de risques-le Surgical Mortality Score (SMS). On a aussi calculé les rapports mortalité observée/mortalité attendue appliqués au modèle de SMS lors d’un audit chirurgical. 11089 patients dans cinq spécialités chirurgicales (maxillo-faciale, orthopédie, transplantation et dialyse rénale, chirurgie générate et neurochirurgie) étaient éligibles. Dans 3.7% des cas, les données étaient incomplètes, toutefois sans preuve de sous-estimation de façon systématique. Le modèle SMS était bien calibré [statistique de Hosmer-Lemeshow C: développement (3.432, p = 0.33), validation (6.359, p = 0.10) avec une capacité de discrimination élevée (aires ROC: développement (0.837 ± [DS] 0.013), validation (0.816 ± 0.016). L’analyse de sous-groupe a confirmé que le modèle pouvait être utilisé par chaque spécialité, à la fois pour les interventions à froid et pour les urgences. Le modèle SMS est un modèle de stratification de risque précis, pouvant être dérivé des banques de données existantes. Il est simple à appliquer comme outil de dépistage pour détecter des déviations par rapport aux résultats attendus et ainsi, est d’une grande utilité pour les audits.

Resumen

Los métodos actuales de ajuste de riesgos en la auditoría quirúrgica son complejos y costosos. El presente estudio tuvo como objeto desarrollar una estratificación simple de la gradación o puntaje del riesgo de mortalidad y un instrumento robusto de auditoría utilizando los recursos existentes en la base de datos hospitalaria del Patient Administration Systems (PAS). Se realizó un estudio observacional de la totalidad de los pacientes sometidos a procedimientos quirúrgicos en un periodo de dos años en el London University Hospital. Se utilizó el análisis logístico de regresión para determinar los factores de predicción de mortalidad hospitalaria, que fue el estudio de resultados. Los odds ratios (razón de disparidad) fueron utilizados para la derivación del modelo simplificado de estratificación de riesgo-el puntaje de mortalidad quirúrgica (Surgical Mortality Score, SMS). La razón de la mortalidad observada frente al riesgo de mortalidad fue calculada mediante la aplicación del modelo SMS en la auditoría quirúrgica. Hubo 11.089 casos elegibles, ubicados en cinco especialidades quirúrgicas (maxilo-facial, ortopedia, trasplante renal/diálisis, cirugía general y neurocirugía). Se encontraron datos incompletos en 3.7% del total, sin evidencia de subregistro sistemático. El modelo SMS fue debidamente calibrado [Hosmer-Lemeshow-C-statistic; development set (3.432, p = 0.33), un set de validación (6.359, p = 0.10), con una alta habilidad de discriminación (ROC áreas: development set (0.837, S.E. = 0.013), set de validación (0816, S.E. = 0.016)]. Los análisis de subgrupo confirmaron que el modelo puede ser utilizado en las especialidades individuales, tanto en cirugía electiva como de urgencia. El SMS es un modelo seguro de estratificación de riesgo, derivado de recursos en las bases de datos existentes. Es fácil de aplicar como un instrumento de manejo de riesgo, de tamizaje para detectar aberraciones en los resultados quirúrgicos y de ayuda en la auditoría quirúrgica.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Boyd O, Grounds RM. Physiological scoring systems and audit. Lancet 1993;341:1573–1574PubMedCrossRefGoogle Scholar
  2. 2.
    Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes Ann Arbor, MI, Health Administration Press, 1997;169–242Google Scholar
  3. 3.
    Romano PS, Mark DH. Bias in the coding of hospital discharge data and its implications for quality assessment. Med. Care 1994;32:81–90PubMedCrossRefGoogle Scholar
  4. 4.
    Khuri SF, Daley J, Henderson W, et al. Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J. Am. Coll. Surg. 1997;185:315–327PubMedGoogle Scholar
  5. 5.
    Information Authority NHS. Data Quality Classifications March 2002. Available from: URL: http://www.nhsia.nhs.uk/dataquality/pages/classifications.aspGoogle Scholar
  6. 6.
    Information Authority NHS. Data Quality Audit Framework for Coded Clinical Data 2000. Available from: URL: http://www.nhsia.nhs.uk/dataquality/pages/data-quality/audit_frmwk.aspGoogle Scholar
  7. 7.
    National Confidential Enquiry into Perioperative Deaths. Classification of Operations (NCEPOD definitions). Available from: URL: http://www.ncepod.org.uk/TaNAppend.pdfGoogle Scholar
  8. 8.
    National Confidential Enquiry into Perioperative Deaths. Who operates When. 1996. Available from: URL: http://www.ncepod.org.uk/reports.htmGoogle Scholar
  9. 9.
    Tekkis PP. Median operation times 2001. Available from: URL: http://www.riskprediction.org.uk/operation_time.htmlGoogle Scholar
  10. 10.
    Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 1996;15:361–387PubMedCrossRefGoogle Scholar
  11. 11.
    Schafer JL. NORM: multiple imputation of incomplete multivariate data under a normal model, version 2; 1999. Software for Windows 95/98/NT. (computer program) Available from: URL: http://www.stat.psu.edu/~jls/misoftwa.html.Google Scholar
  12. 12.
    Picard RR, Berk KN. Data splitting. American Statistician 1990;44: 140–147CrossRefGoogle Scholar
  13. 13.
    Hosmer JA, Lemeshow S, editors. Applied Logistic Regression l40-145Google Scholar
  14. 14.
    Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operator characteristic (ROC) curve. Radiology 1982;143:29–36PubMedGoogle Scholar
  15. 15.
    Weijnen CF, Numans ME, de Vit NJ, et al. Testing for Helicobacter pylori in dyspeptic patients suspected of peptic ulcer disease in primary care: cross sectional study. BMJ 2001;323:71–75PubMedCrossRefGoogle Scholar
  16. 16.
    Department of Health. NHSE. Clinical indicators June 1999. Available from: URL: http://www.doh.gov.uk/indicat/techannx.htm (Annex C)Google Scholar
  17. 17.
    Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ 2001;323:224–228PubMedCrossRefGoogle Scholar
  18. 18.
    De Ritis G, Giovannini C, Picardo S, et al. Multivariate prediction of in-hospital mortality associated with surgical procedures. Minerva Anastesiol. 1995;61:173–181Google Scholar
  19. 19.
    Marshall G, Grover FL, Henderson WG, et al. Assessment of predictive models for binary outcomes: an empirical approach using operative death from cardiac surgery. Stat. Med. 1994;13:1501–1511PubMedCrossRefGoogle Scholar
  20. 20.
    Poloniecki J, Valencia O, Littlejohns P. Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery. BMJ 1998;316:1697–1700PubMedGoogle Scholar
  21. 21.
    Prytherch DR, Whiteley MS, Higgins B, et al. POSSUM and Portsmouth POSSUM for predicting mortality. Br. J. Surg. 1998;85:1217–1220PubMedCrossRefGoogle Scholar
  22. 22.
    Copeland GP. Surgical scoring, risk assessment and the surgeon. J. R. Coll. Surg. Edinb. 1992;37:145–148PubMedGoogle Scholar
  23. 23.
    Poloniecki JD, Roxburgh JC. Performance data and the mortuary register. Ann. R. Coll. Surg. Engl. 2000;82:401–404PubMedGoogle Scholar
  24. 24.
    Krumholz HM. Mathematical models and the assessment of performance in cardiology. Circulation 1999;99:2067–2069PubMedGoogle Scholar

Copyright information

© Société Internationale de Chirurgie 2004

Authors and Affiliations

  • Vassilis G. Hadjianastassiou
    • 1
  • Paris P. Tekkis
    • 2
  • Jan D. Poloniecki
    • 3
  • Manolis C. Gavalas
    • 4
  • David R. Goldhill
    • 5
  1. 1.Department of Vascular SurgeryUniversity Hospital LewishamLondonUK
  2. 2.Department of SurgerySt. Mark’s HospitalHarrowUK
  3. 3.Department of Public Health SciencesSt George’s Hospital Medical SchoolLondonUK
  4. 4.Accident and Emergency DepartmentUniversity College HospitalLondonUK
  5. 5.Department of AnaesthesiaRoyal National Orthopedic HospitalStanmoreUK

Personalised recommendations