Skip to main content
SpringerLink
Log in

Electronic health record data is unable to effectively characterize measurement error from pulse oximetry: a simulation study

  • Brief Report
  • Published:
Journal of Clinical Monitoring and Computing Aims and scope Submit manuscript

Abstract

Large data sets from electronic health records (EHR) have been used in journal articles to demonstrate race-based imprecision in pulse oximetry (SpO2) measurements. These articles do not appear to recognize the impact of the variability of the SpO2 values with respect to time (“deviation time”). This manuscript seeks to demonstrate that due to this variability, EHR data should not be used to quantify SpO2 error. Using the MIMIC-IV Waveform dataset, SpO2 values are sampled from 198 patients admitted to an intensive care unit and used as reference samples. The error derived from the EHR data is simulated using a set of deviation times. The laboratory oxygen saturation measurements are also simulated such that the performance of three simulated pulse oximeter devices will produce an average root mean squared (ARMS) error of 2%. An analysis is then undertaken to reproduce a medical device submission to a regulatory body by quantifying the mean error, the standard deviation of the error, and the ARMS error. Bland-Altman plots were also generated with their Limits of Agreements. Each analysis was repeated to evaluate whether the measurement errors were affected by increasing the deviation time. All error values increased linearly with respect to the logarithm of the time deviation. At 10 min, the ARMS error increased from a baseline of 2% to over 4%. EHR data cannot be reliably used to quantify SpO2 error. Caution should be used in interpreting prior manuscripts that rely on EHR data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The MIMIC-IV waveform dataset is presently available online.

The source code used to compile and process the data is available online:

https://github.com/rasman/pulseox_ehr

Notes

  1. https://github.com/rasman/pulseox_ehr

References

  1. Bickler PE, Feiner JR, Severinghaus JW. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology. 2005;102:715–9. https://doi.org/10.1097/00000542-200504000-00004.

    Article  PubMed  Google Scholar 

  2. Barker SJ, Wilson WC. Racial effects on Masimo pulse oximetry: a laboratory study. J Clin Monit Comput. 2022. https://doi.org/10.1007/s10877-022-00927-w.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Fawzy A, Wu TD, Wang K, Robinson ML, Farha J, Bradke A, Golden SH, Xu Y, Garibaldi BT. Racial and ethnic discrepancy in pulse oximetry and delayed identification of treatment eligibility among patients with COVID-19. JAMA Intern Med. 2022;182:730–8. https://doi.org/10.1001/jamainternmed.2022.1906.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Sjoding MW, Dickson RP, Iwashyna TJ, Gay SE, Valley TS. Racial Bias in Pulse Oximetry Measurement. N Engl J Med. 2020;383:2477–8. https://doi.org/10.1056/NEJMc2029240.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Valbuena VSM, Seelye S, Sjoding MW, Valley TS, Dickson RP, Gay SE, Claar D, Prescott HC, Iwashyna TJ. Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the Veterans Health Administration 2013-19: multicenter, retrospective cohort study. BMJ. 2022;378:e069775. https://doi.org/10.1136/bmj-2021-069775.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Al-Halawani R, Charlton PH, Qassem M, Kyriacou PA. A review of the effect of skin pigmentation on pulse oximeter accuracy. Physiol Meas. 2023. https://doi.org/10.1088/1361-6579/acd51a.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Shi C, Goodall M, Dumville J, Hill J, Norman G, Hamer O, Clegg A, Watkins CL, Georgiou G, Hodkinson A, Lightbody CE, Dark P, Cullum N. The accuracy of pulse oximetry in measuring oxygen saturation by levels of skin pigmentation: a systematic review and meta-analysis. BMC Med. 2022;20:267. https://doi.org/10.1186/s12916-022-02452-8.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Sarraf E, Mathews DM, Tsai MH, Goodwin AJ. Optimizing the use of point of care testing devices for screening patients. J Clin Monit Comput. 2020;34:411–9. https://doi.org/10.1007/s10877-019-00332-w.

    Article  PubMed  Google Scholar 

  9. Rezek IA, Roberts SJ. Stochastic complexity measures for physiological signal analysis. IEEE Trans Biomed Eng. 1998;45:1186–91. https://doi.org/10.1109/10.709563.

    Article  CAS  PubMed  Google Scholar 

  10. Clarke M, Gokalp H, Fursse J, Jones RW. Dynamic threshold analysis of Daily Oxygen Saturation for Improved Management of COPD patients. IEEE J Biomed Health Inform. 2016;20:1352–60. https://doi.org/10.1109/jbhi.2015.2464275.

    Article  PubMed  Google Scholar 

  11. Moody B, Hao S, Gow B, Pollard T, Zong W, Mark R. (2022) MIMIC-IV Waveform Database (version 0.1.0) https://doi.org/10.13026/a2mw-f949.

  12. Goldberger A, Amaral L, Glass L, Hausdorff J, Ivanov P, Mark R, Mietus J, Moody G, Peng C, Stanley H. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation [Online]. 2000;101:e215–20. https://doi.org/10.1161/01.cir.101.23.e215.

    Article  CAS  PubMed  Google Scholar 

  13. (2013) Pulse Oximeters - Premarket Notification Submissions [510(k)s]: Guidance for Industry and Food and Drug Administration Staff.FDA-2007-D-0205. https://www.fda.gov/media/72470/download.

  14. Gehring H, Duembgen L, Peterlein M, Hagelberg S, Dibbelt L. Hemoximetry as the gold standard? Error assessment based on differences among identical blood gas analyzer devices of five manufacturers. Anesth Analg. 2007;105:24–s30. https://doi.org/10.1213/01.ane.0000268713.58174.cc.

    Article  Google Scholar 

  15. Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82. https://doi.org/10.1080/10543400701329422.

    Article  MathSciNet  PubMed  Google Scholar 

  16. Goodwin AJ, Eytan D, Dixon W, Goodfellow SD, Doherty Z, Greer RW, McEwan A, Tracy M, Laussen PC, Assadi A, Mazwi M. Timing errors and temporal uncertainty in clinical databases-A narrative review. Front Digit Health. 2022;4:932599. https://doi.org/10.3389/fdgth.2022.932599.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Weisstein EW. (2023) Expectation Value. MathWorld–A Wolfram Web Resource https://mathworld.wolfram.com/ExpectationValue.html. Aug 22 2023.

Download references

Acknowledgements

The author would like to thank Ms. Emily Raffensberger for her assistance in proofreading the manuscript.

Funding

This work was supported by departmental funds.

Author information

Authors and Affiliations

  1. Department of Anesthesiology and Perioperative Medicine, Penn State College of Medicine, 500 University Drive Mail Code, H187, Hershey, PA, 17033, USA

    Elie Sarraf

Authors
  1. Elie Sarraf
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ES was the sole participant in the study conception, design, analysis and drafting of the manuscript.

Corresponding author

Correspondence to Elie Sarraf.

Ethics declarations

Competing interests

Elie Sarraf was an employee of Masimo Corp from July 2013 to January 2015.

Ethical approval

Not applicable (data publicly available).

Informed consent

Not applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (XLSX 23.3 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarraf, E. Electronic health record data is unable to effectively characterize measurement error from pulse oximetry: a simulation study. J Clin Monit Comput (2024). https://doi.org/10.1007/s10877-024-01131-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10877-024-01131-8

Keywords

Search

Navigation