Abstract
For more than three decades, multivariable risk factor analysis has been the main statistical technique for identifying and quantifying treatment outcome differences adjusted for patient characteristics, these differences being treated as associations with outcomes, and not causes. There is no guarantee that risk factor analysis is an effective strategy for discovery of a cause-and-effect mechanism. Multivariable logistic regression appears to be very suitable for epi-demiological surgical research, especially for dichoto-mous outcomes such as mortality, as disease occurrence has multiple risk factors, which can be mutually correlated. The mathematical model known as a multiple logistic regression assumes that the dependent variable (the outcome of interest, such as death in this case) is linearly and additively related to the independent variables (patient risk factors) on the logistic scale. The technique is useful primarily because it produces a direct estimate of the odds ratio of any risk factor, which reflects the dose—response relationship between these and the outcome. However, if this actual relationship is non-linear, then an adequate fit can usually be achieved by adding polynomial and product interaction terms to the model, although the meaning of the odds ratios in such an interaction model would be severely circumscribed. An essential step in risk stratification modeling (RSM) is to evaluate whether there is evidence of an interaction between the variables used in a model. This interaction would imply that the effect of one of the variables is not constant over levels of the other. An important drawback of logistic regression is that it can increase bias because of misclassification and measurement errors in confounding variables and differences between conditional and unconditional odds ratio estimates of treatment effects. In this chapter, we outline all the basic components that the surgeon needs to know about risk stratification and predictive modelling.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- AAA:
-
Abdominal aortic aneurysm
- ANN:
-
Artificial neural network
- APACHE:
-
Acute physiology and chronic health evaluation
- APS:
-
Acute physiology score
- ASA:
-
American Society of Anaesthesiology
- ECG:
-
Electrocardiogram
- ICU:
-
Intensive care unit
- MPM:
-
Mortality prediction model
- O:E:
-
Observed:expected
- POSSUM:
-
Physiology and operative severity score for the enumeration of mortality and morbidity
- ROC:
-
Receiver operating characteristic curve
- RSM:
-
Risk stratification modelling
- SAPS:
-
Simplified acute physiology score
- SMR:
-
Standardised mortality ratio
- SMS:
-
Surgical mortality score
- VBHOM:
-
Vascular biochemistry and haematology outcome models
References
Hadjianastassiou VG, Tekkis PP, Poloniecki JD et al (2004) Surgical mortality score: risk management tool for auditing surgical performance. World J Surg 28:193–200
Gunning K, Rowan K (1999) ABC of intensive care: outcome data and scoring systems. BMJ 319:241–244
Lemeshow S, Klar J, Teres D (1995) Outcome prediction for individual intensive care patients: useful, misused, or abused? Intensive Care Med 21:770–776
Marcin JP, Pollack MM, Patel KM et al (2000) Combining physician's subjective and physiology-based objective mortality risk predictions. Crit Care Med 28:2984–2990
Tekkis PP, Poloniecki JD, Thompson MR et al (2003) Operative mortality in colorectal cancer: prospective national study. BMJ 327:1196–1201
Cullen DJ, Chernow B (1994) Predicting outcome in critically ill patients. Crit Care Med 22:1345–1348
Hadjianastassiou VG, Franco L, Jerez JM et al (2006) Informed prognosis [corrected] after abdominal aortic aneu-rysm repair using predictive modeling techniques [corrected]. J Vasc Surg 43:467–473
Joint Commission Resources (1999) Florence nightingale: measuring health care outcomes. Joint Commission on Accreditation of Healthcare Organizations, Oakbrook, IL
Nightingale F, Jcaho (1999) Measuring Hospital Care Outcomes. Joint Commission on Accreditation of Healthcare
Mourouga P, Goldfrad C, Rowan KM (2000) Does it fit? Is it good? Assessment of scoring systems (severity scoring in the critically ill patient). Curr Opin Crit Care 6:176–180
Hosmer DW, Lemeshow S (2000) Applied logistic regression. Wiley, New York
Jones HJ, de Cossart L (1999) Risk scoring in surgical patients. Br J Surg 86:149–157
Vacanti CJ, VanHouten RJ, Hill RC (1970) A statistical analysis of the relationship of physical status to postoperative mortality in 68,388 cases. Anesth Analg 49:564–566
Schuster DP (1992) Predicting outcome after ICU admission. The art and science of assessing risk. Chest 102:1861–1870
Knaus WA, Zimmerman JE, Wagner DP et al (1981) APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 9:591–597
Knaus WA, Draper EA, Wagner DP et al (1985) APACHE II: a severity of disease classification system. Crit Care Med 13:818–829
Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–387
Wisner DH (1992) History and current status of scoring systems for critical care. Arch Surg 127:352–356
Knaus WA, Wagner DP, Draper EA et al (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636
Lemeshow S, Le Gall JR (1994) Modeling the severity of illness of ICU patients. A systems update. JAMA 272:1049–1055
Berger MM, Marazzi A, Freeman J et al (1992) Evaluation of the consistency of Acute Physiology and Chronic Health Evaluation (APACHE II) scoring in a surgical intensive care unit. Crit Care Med 20:1681–1687
Beck DH, Taylor BL, Millar B et al (1997) Prediction of outcome from intensive care: a prospective cohort study comparing Acute Physiology and Chronic Health Evaluation II and III prognostic systems in a United Kingdom intensive care unit. Crit Care Med 25:9–15
Barie PS, Hydo LJ, Fischer E (1995) Comparison of APACHE II and III scoring systems for mortality prediction in critical surgical illness. Arch Surg 130:77–82
Beck DH, Smith GB, Pappachan JV et al (2003) External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive Care Med 29:249–256
Rowan KM, Kerr JH, Major E et al (1993) Intensive Care Society's APACHE II study in Britain and Ireland-II: outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. BMJ 307:977–981
Rowan KM, Kerr JH, Major E et al (1993) Intensive Care Society's APACHE II study in Britain and Ireland-I: variations in case mix of adult admissions to general intensive care units and impact on outcome. BMJ 307:972–977
Pappachan J V, Millar B, Bennett ED et al (1999) Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system. Chest 115:802–810
Le Gall JR, Loirat P, Alperovitch A et al (1984) A simplified acute physiology score for ICU patients. Crit Care Med 12:975–977
Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963
Beck DH, Smith GB, Taylor BL (2002) The impact of low-risk intensive care unit admissions on mortality probabilities by SAPS II, APACHE II and APACHE III. Anaesthesia 57:21–26
Nuttall P. The passionate statistician (Florence Nightingale). Nurs Times. 1983 Sep 28-Oct 4;79(39):25–7
Lemeshow S, Teres D, Klar J et al (1993) Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486
Rowan KM, Kerr JH, Major E et al (1994) Intensive Care Society's acute physiology and chronic health evaluation (APACHE II) study in Britain and Ireland: a prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients. Crit Care Med 22:1392–1401
Livingston BM, MacKirdy FN, Howie JC et al (2000) Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med 28:1820–1827
Boyd O, Grounds RM (1993) Physiological scoring systems and audit. Lancet 341:1573–1574
Iezzoni L (1997) Risk adjustment for measuring health care outcomes. Health Administration Press, Chicago
Khuri SF, Daley J, Henderson W et al (1997) Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J Am Coll Surg 185:315–327
Copeland GP, Jones D, Walters M (1991) POSSUM: a scoring system for surgical audit. Br J Surg 78:355–360
Wijesinghe LD, Mahmood T, Scott DJ et al (1998) Comparison of POSSUM and the Portsmouth predictor equation for predicting death following vascular surgery. Br J Surg 85:209–212
Neary WD, Heather BP, Earnshaw JJ (2003) The physiological and operative severity Score for the enumeration of mortality and morbidity (POSSUM). Br J Surg 90:157–165
Whiteley MS, Prytherch DR, Higgins B et al (1996) An evaluation of the POSSUM surgical scoring system. Br J Surg 83:812–815
Prytherch DR, Whiteley MS, Higgins B et al (1998) POSSUM and Portsmouth POSSUM for predicting mortality. Physiological and operative severity score for the enumeration of mortality and morbidity. Br J Surg 85:1217–1220
Prytherch DR, Ridler BM, Beard JD et al (2001) A model for national outcome audit in vascular surgery. Eur J Vasc Endovasc Surg 21:477–483
Prytherch DR, Sutton GL, Boyle JR (2001) Portsmouth POSSUM models for abdominal aortic aneurysm surgery. Br J Surg 88:958–963
Tekkis PP, Prytherch DR, Kocher HM et al (2004) Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br J Surg 91:1174–1182
Tekkis PP, McCulloch P, Poloniecki JD et al (2004) Risk-adjusted prediction of operative mortality in oesophagogas-tric surgery with O-POSSUM. Br J Surg 91:288–295
Bann SD, Sarin S (2001) Comparative audit: the trouble with POSSUM. J R Soc Med 94:632–634
Lloyd H, Ahmed I, Taylor S et al (2005) Index for predicting mortality in elderly surgical patients. Br J Surg 92:487–492
Prytherch DR, Ridler BM, Ashley S (2005) Risk-adjusted predictive models of mortality after index arterial operations using a minimal data set. Br J Surg 92:714–718
Prytherch DR, Sirl JS, Weaver PC et al (2003) Towards a national clinical minimum data set for general surgery. Br J Surg 90:1300–1305
Tang T, Walsh SR, Prytherch DR et al (2007) VBHOM, a data economic model for predicting the outcome after open abdominal aortic aneurysm surgery. Br J Surg 94:717–721
Hadjianastassiou V (2007) VBHOM, a data economic model for predicting the outcome after open abdominal aortic aneu-rysm surgery. Br J Surg 94:1308
Hadjianastassiou VG, Tekkis PP, Goldhill DR et al (2005) Quantification of mortality risk after abdominal aortic aneu-rysm repair. Br J Surg 92:1092–1098
Hadjianastassiou VG, Tekkis PP, Athanasiou T et al (2007) External validity of a mortality prediction model in patients after open abdominal aortic aneurysm repair using multilevel methodology. Eur J Vasc Endovasc Surg 34:514–521
Hadjianastassiou VG, Tekkis PP, Athanasiou T et al (2007) Comparison of mortality prediction models after open abdominal aortic aneurysm repair. Eur J Vasc Endovasc Surg 33:536–543
Daley J, Forbes MG, Young GJ et al (1997) Validating risk-adjusted surgical outcomes: site visit assessment of process and structure. National VA Surgical Risk Study. J Am Coll Surg 185:341–351
Khuri SF, Daley J, Henderson W et al (1995) The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg 180:519–531
Khuri SF, Daley J, Henderson W et al (1998) The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg 228:491–507
Samy AK, Murray G, MacBain G (1994) Glasgow aneurysm score. Cardiovasc Surg 2:41–44
Samy AK, Murray G, MacBain G (1996) Prospective evaluation of the Glasgow Aneurysm Score. J R Coll Surg Edinb 41:105–107
Prance SE, Wilson YG, Cosgrove CM et al (1999) Ruptured abdominal aortic aneurysms: selecting patients for surgery. Eur J Vasc Endovasc Surg 17:129–132
Tambyraja AL, Fraser SC, Murie JA et al (2005) Validity of the Glasgow Aneurysm Score and the Hardman Index in predicting outcome after ruptured abdominal aortic aneurysm repair. Br J Surg 92:570–573
De Ritis G, Giovannini C, Picardo S et al (1995) Multivariate prediction of in-hospital mortality associated with surgical procedures. Minerva Anestesiol 61:173–181
Lee J (1986) An insight on the use of multiple logistic regression analysis to estimate association between risk factor and disease occurrence. Int J Epidemiol 15:22–29
Harrell FE Jr, Lee KL, Matchar DB et al (1985) Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep 69:1071–1077
Picard RR, Berk KN (1990) Data splitting. Am Stat 44:140–147
Hadorn DC, Draper D, Rogers WH et al (1992) Cross-validation performance of mortality prediction models. Stat Med 11:475–489
Efron B, Tibshirani R (1993) An introduction to the Bootstrap. Chapman & Hall, New York
Cox DR (1972) Regression models and life-tables. J R Stat Soc B 34:187–202
Bennett N (1976) Teaching styles and pupil progress. Open Books, London
Aitkin M, Anderson D, Hinde J (1981) Statistical modelling of data on teaching styles (with discussion). J Royal Statist Soc A 149:148–161
Spiegelhalter DJ, Aylin P, Best NG et al (2002) Commissioned analysis of surgical performance by using routine data: lessons from Bristol inquiry. J R Statist Soc A 165:1–31
Marshall EC, Spiegelhalter DJ (1998) Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. BMJ 316:1701–1704; discussion 1705
Kee F, Wilson RH, Harper C et al (1999) Influence of hospital and clinician workload on survival from colorectal cancer: cohort study. BMJ 318:1381–1385
McCulloch P, Ward J, Tekkis PP (2003) Mortality and morbidity in gastro-oesophageal cancer surgery: initial results of ASCOT multicentre prospective cohort study. BMJ 327:1192–1197
Beck DH, Smith GB, Pappachan JV (2002) The effects of two methods for customising the original SAPS II model for intensive care patients from South England. Anaesthesia 57:785–793
Moreno R, Miranda DR, Fidler V et al (1998) Evaluation of two outcome prediction models on an independent database. Crit Care Med 26:50–61
Cologne JB, Shibata Y (1995) Optimal case-control matching in practice. Epidemiology 6:271–275
Rosenbaum PR (1989) Optimal matching in observational studies. J Am Stat Assoc 84:1024–1032
D'Agostino RB Jr (1998) Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med 17:2265–2281
Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55
Rubin DB (1980) Bias reduction using mahalanobis-metric matching. Biometrics 36:293–298
Drew PJ, Monson JR (2000) Artificial neural networks. Surgery 127:3–11
Cross SS, Harrison RF, Kennedy RL (1995) Introduction to neural networks. Lancet 346:1075–1079
Ramesh AN, Kambhampati C, Monson JR et al (2004) Artificial intelligence in medicine. Ann R Coll Surg Engl 86:334–338
Werbos P (1974) Beyond regression: new tools for prediction and analysis in the behavioral sciences. Harvard University Press, Cambridge
Burke H (1994) Artificial neural networks for cancer research: outcome prediction. Sem Surg Oncol 10:73–79
Sawyer MD (2000) Invited commentary: artificial neural networks — an introduction. Surgery 127:1–2
Wei JT, Zhang Z, Barnhill SD et al (1998) Understanding arti-ficial neural networks and exploring their potential applications for the practicing urologist. Urology 52:161–172
De Laurentis M, Ravdin PM (1994) A technique for using neural network analysis to perform survival analysis of censored data. Cancer Lett 77:127–138
Baxt WG (1995) Application of artificial neural networks to clinical medicine. Lancet 346:1135–1138
Dybowski R, Gant V (1995) Artificial neural networks in pathology and medical laboratories. Lancet 346:1203–1207
Dybowski R, Weller P, Chang R et al (1996) Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm. Lancet 347:1146–1150
Wong LS, Young JD (1999) A comparison of ICU mortality prediction using the APACHE II scoring system and artifi-cial neural networks. Anaesthesia 54:1048–1054
Bottaci L, Drew PJ, Hartley JE et al (1997) Artificial neural networks applied to outcome prediction for colorectal cancer patients in separate institutions. Lancet 350:469–472
Wyatt J (1995) Nervous about artificial neural networks? Lancet 346:1175–1177
Moreno R, Apolone G, Miranda DR (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47
Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226
Wasson JH, Sox HC, Neff RK et al (1985) Clinical prediction rules. Applications and methodological standards. N Engl J Med 313:793–799
Charlson ME, Ales KL, Simon R et al (1987) Why predictive indexes perform less well in validation studies. Is it magic or methods? Arch Intern Med 147:2155–2161
Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36
Harrell FE Jr, Califf RM, Pryor DB et al (1982) Evaluating the yield of medical tests. JAMA 247:2543–2546
Weijnen CF, Numans ME, de Wit NJ et al (2001) Testing for Helicobacter pylori in dyspeptic patients suspected of peptic ulcer disease in primary care: cross sectional study. BMJ 323:71–75
Schafer JH, Maurer A, Jochimsen F et al (1990) Outcome prediction models on admission in a medical intensive care unit: do they predict individual outcome? Crit Care Med 18:1111–1118
Apolone G, Bertolini G, D'Amico R et al (1996) The performance of SAPS II in a cohort of patients admitted to 99 Italian ICUs: results from GiViTI. Gruppo Italiano per la Valutazione degli interventi in Terapia Intensiva. Intensive Care Med 22:1368–1378
Poses RM, Cebul RD, Collins M et al (1986) The importance of disease prevalence in transporting clinical prediction rules. The case of streptococcal pharyngitis. Ann Intern Med 105:586–591
Ivanov J, Tu JV, Naylor CD (1999) Ready-made, recalibrated, or remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Circulation 99:2098–2104
Lemeshow S, Klar J, Teres D et al (1994) Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study. Crit Care Med 22:1351–1358
Zhu BP, Lemeshow S, Hosmer DW et al (1996) Factors affecting the performance of the models in the Mortality Probability Model II system and strategies of customization: a simulation study. Crit Care Med 24:57–63
Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473
Teres D, Lemeshow S (1999) When to customize a severity model. Intensive Care Med 25:140–142
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hadjianastassiou, V.G., Athanasiou, T., Hands, L.J. (2010). Risk Stratification and Prediction Modelling in Surgery. In: Athanasiou, T., Debas, H., Darzi, A. (eds) Key Topics in Surgical Research and Methodology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71915-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-540-71915-1_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71914-4
Online ISBN: 978-3-540-71915-1
eBook Packages: MedicineMedicine (R0)