Journal of Clinical Monitoring and Computing

, Volume 29, Issue 1, pp 127–139 | Cite as

Improvements in the application and reporting of advanced Bland–Altman methods of comparison

  • Erik Olofsen
  • Albert Dahan
  • Gerard Borsboom
  • Gordon Drummond
Original Research


Bland and Altman have developed a measure called “limits of agreement” to assess correspondence of two methods of clinical measurement. In many circumstances, comparisons are made using several paired measurements in each individual subject. If such measurements are considered as statistically independent pairs, rather than as sets of measurements from separate individuals, limits of agreement will be too narrow. In addition, the confidence intervals for these limits will also be too narrow. Suitable software to compute valid limits of agreement and their confidence intervals is not readily available. Therefore, we set out to provide a freely available implementation accompanied by a formal description of the more advanced Bland–Altman comparison methods. We validate the implementation using simulated data, and demonstrate the effects caused by failing to take the presence of multiple paired measurements per individual properly into account. We propose a standard format of reporting that would improve analysis and interpretation of comparison studies.


Bland–Altman Limits of agreement Confidence intervals Software 



The statistical properties of the simulated data for the example application were inspired by a real data set kindly provided by Prof. L.A.H. Critchley.


  1. 1.
    Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. Stat. 1983;32:307–17.Google Scholar
  2. 2.
    Biancofiore G, Critchley LAH, Lee A, Yang X, Bindi LM, Esposito M, Bisà M, Meacci L, Mozzo R, Filipponi F. Evaluation of a new software version of the Flotrac/Vigileo (version 3.02) and a comparison with previous data in cirrhotic patients undergoing liver transplant surgery. Anesth Analg. 2011;113:515–22.PubMedGoogle Scholar
  3. 3.
    Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;i:307–10.Google Scholar
  4. 4.
    Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60.PubMedCrossRefGoogle Scholar
  5. 5.
    Bland JM, Altman DG. Applying the right statistics: analyses of measurement studies. Ultrasound Obstet Gynecol. 2003;22:85–93.PubMedCrossRefGoogle Scholar
  6. 6.
    Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82.PubMedCrossRefGoogle Scholar
  7. 7.
    Bland JM, Altman DG. Agreed statistics: measurement method comparison. Anesthesiology. 2012;116:182–5.PubMedCrossRefGoogle Scholar
  8. 8.
    Burdick RK, Graybill FA. Confidence intervals on linear combinations of variance components in the unbalanced one-way classification. Technometrics. 1984;26:131–6.CrossRefGoogle Scholar
  9. 9.
    Columb MO. Clinical measurement and assessing agreement. Curr Anaesth Crit Care. 2008;19:328–9.CrossRefGoogle Scholar
  10. 10.
    Donner A, Zou GY. Closed-form confidence intervals for functions of the normal mean and standard deviation. Stat Methods Med Res. 2012;21:347–59.PubMedCrossRefGoogle Scholar
  11. 11.
    Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman and Hall; 1993.CrossRefGoogle Scholar
  12. 12.
    Hamilton C, Lewis S. The importance of using the correct bounds on the Bland–Altman limits of agreement when multiple measurements are recorded per patient. J Clin Monit Comput. 2010;24:173–5.PubMedCrossRefGoogle Scholar
  13. 13.
    Hamilton C, Stamey J. Using Bland–Altman to assess agreement between two medical devices—don’t forget the confidence intervals!. J Clin Monit Comput. 2007;21:331–3.PubMedCrossRefGoogle Scholar
  14. 14.
    Mantha S, Roizen MF, Fleisher LA, Thisted R, Foss J. Comparing methods of clinical measurement: reporting standards for Bland and Altman analysis. Anesth Analg. 2000;90:593–602.PubMedCrossRefGoogle Scholar
  15. 15.
    Mozilla Foundation: Javascript. Accessed April 2013.
  16. 16.
    Myles PS, Cui J. Using the Bland–Altman method to measure agreement with repeated measures. Br J Anaesth. 2007;99:309–11.PubMedCrossRefGoogle Scholar
  17. 17.
    Oldham PD. A note on the analysis of repeated measurements of the same subjects. J Chronic Dis. 1962;15:969–77.PubMedCrossRefGoogle Scholar
  18. 18.
    Schnur D, et al. Flot: attractive JavaScript plotting for jQuery. Accessed April 2013.
  19. 19.
    Sjöstrand F, Rodhe P, Berglund E, Lundström N, Svensen C. The use of a noninvasive hemoglobin monitor for volume kinetic analysis in an emergency room setting. Anesth Analg. 2013;116:337–42.PubMedCrossRefGoogle Scholar
  20. 20.
    The jQuery Foundation: jQuery. Accessed April 2013.
  21. 21.
    Thomas JD, Hultquist RA. Interval estimation for the unbalanced case of the one-way random effects model. Ann Stat. 1978;6:582–7.CrossRefGoogle Scholar
  22. 22.
    Uemura K, Kawada T, Inagaki M, Sugimachi M. A minimally invasive monitoring system of cardiac output using aortic flow velocity and peripheral arterial pressure profile. Anesth Analg. 2013;116:1006–17.PubMedCrossRefGoogle Scholar
  23. 23.
    UncertWeb: jStat: a JavaScript statistical library. Accessed April 2013.
  24. 24.
    Zou GY. Confidence interval estimation for the Bland–Altman limits of agreement with multiple observations per individual. Stat Methods Med Res. 2013;22:630–42.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Erik Olofsen
    • 1
  • Albert Dahan
    • 1
  • Gerard Borsboom
    • 2
  • Gordon Drummond
    • 3
  1. 1.Department of Anesthesiology P5QLeiden University Medical CenterLeidenThe Netherlands
  2. 2.Department of Public HealthErasmus Medical CenterRotterdamThe Netherlands
  3. 3.Department of Anaesthesia and Pain Medicine, Royal InfirmaryUniversity of EdinburghEdinburghScotland, UK

Personalised recommendations