Skip to main content

The interrater reliability of SAPS II and SAPS 3

Intensive Care Medicine volume 36, pages 850–853 (2010)Cite this article

An Erratum to this article was published on 06 May 2010

Abstract

Purpose

In this study we evaluated and compared the interrater reliability of SAPS II and SAPS 3 in order to measure the consistency of performance among different raters.

Method

Ten junior doctors working at two general ICUs were trained in the use of SAPS II and SAPS 3 using a 2.5-h training program. After training they scored 24 cases in both systems. Scores were analyzed using intraclass correlation coefficient (ICC) statistics. In order to identify variables with low reliability, subscores were analyzed using the ICC, and single-variables were compared to a template score using weighted kappa statistics.

Results

The ICC (95% CI) of the scores was 0.84 (0.74, 0.91) in SAPS II and 0.80 (0.68, 0.89) in SAPS 3, which is considered adequate for both systems. Mean mortality predictions among the raters had a range of 0.12 in SAPS II and 0.19 in SAPS 3. Administrative data including age had high reliability, whereas variables based on diagnostic information had only moderate reliability. Laboratory data had consistently higher reliability than variables based on the interpretation of charts.

Conclusion

Both SAPS II and SAPS 3 have adequate interrater reliability, but the standardized mortality ratios are still likely to be influenced by the rater's scoring practice.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Notes

  1. http://faculty.vassar.edu/lowry/VassarStats.html accessed in June 2009.

References

  1. Le Gall JR, Lemeshow S, Saulnier F (1993) A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963

    Article  PubMed  Google Scholar 

  2. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators (2005) SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345–1355

    Article  PubMed  Google Scholar 

  3. Strand K, Søreide E, Aardal S, Flaatten H (2009) A comparison of SAPS II and SAPS 3 in a Norwegian intensive care unit population. Acta Anaesthesiol Scand 53:595–600

    Article  CAS  PubMed  Google Scholar 

  4. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators (2005) SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 31:1336–1344

    Article  PubMed  Google Scholar 

  5. Capuzzo M, Valpondi V, Sgarbi A, Bortolazzi S, Pavoni V, Gilli G, Candini G, Gritti G, Alvisi R (1998) Reliability of the general severity scoring systems, APACHE II and SAPS II. Clin Intensive Care 9:4–10

    Article  Google Scholar 

  6. Polderman KG, Jorna EM, Girbes AR (2001) Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Med 27:1365–1369

    Article  CAS  PubMed  Google Scholar 

  7. Kho ME, McDonald E, Stratford PW, Cook DJ (2007) Interrater reliability of APACHE II scores for medical-surgical intensive care patients: a prospective blinded study. Am J Crit Care 16:378–383

    PubMed  Google Scholar 

  8. Chen LM, Martin CM, Morrison TL, Sibbald WJ (1999) Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. Crit Care Med 27:1999–2004

    Article  CAS  PubMed  Google Scholar 

  9. Wenner JB, Norena M, Khan N, Palepu A, Ayas NT, Wong H, Dodek PM (2009) Reliability of intensive care unit admitting and comorbid diagnoses, race, elements of acute physiology and chronic health evaluation II score, and predicted probability of mortality in an electronic intensive care unit database. J Crit Care 24:401–407

    Article  PubMed  Google Scholar 

  10. Walter SD, Eliasziw M, Donner A (1998) Sample size and optimal designs for reliability studies. Stat Med 17:101–110

    Article  CAS  PubMed  Google Scholar 

  11. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428

    Article  CAS  PubMed  Google Scholar 

  12. Altman D (1991) Practical statistics for medical research. Chapman & Hall, London

    Google Scholar 

  13. Sim J, Wright CC (2005) The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 85:257–268

    PubMed  Google Scholar 

  14. Suistomaa M, Kari A, Ruokonen E, Takala J (2000) Sampling rate causes bias in APACHE II and SAPS II scores. Intensive Care Med 26:1773–1778

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Rolf Moe-Nilssen, University of Bergen, for contributing to the design of the study. Johan Torgersen, Haukeland University Hospital, assisted in the collection of data.

Author information

Affiliations

  1. Health Services Research Centre, Akershus University Hospital, Lørenskog, Norway

    K. Strand

  2. Department of Anesthesia and Intensive Care, Stavanger University Hospital, PB 8100, 4068, Stavanger, Norway

    K. Strand

  3. Section for Physiotherapy Science, Department of Public Health and Primary Health Care, University of Bergen, Bergen, Norway

    L. I. Strand

  4. Department of Anesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway

    H. Flaatten

  5. Department of Surgical Sciences, University of Bergen, Bergen, Norway

    H. Flaatten

Authors
  1. K. Strand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. I. Strand
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Flaatten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Strand.

Additional information

An erratum to this article can be found at http://dx.doi.org/10.1007/s00134-010-1896-3

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Strand, K., Strand, L.I. & Flaatten, H. The interrater reliability of SAPS II and SAPS 3. Intensive Care Med 36, 850–853 (2010). https://doi.org/10.1007/s00134-010-1772-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00134-010-1772-1

Keywords

  • Severity of illness index
  • Intensive care
  • Quality assurance
  • Health care
  • Critical care