Abstract
Background
Our goal was to evaluate the performance of a new wearable arm located pulse oximeter.
Methods
Twelve volunteers were monitored with three pulse oximeters and underwent desaturation to 70% SaO2.
We compared the accuracy of SpO2 reading from the SmartCardia system with SpO2 using two well established devices (Masimo and Nellcor) as reference.
Oximetry was performed at different level of oxygen saturation varying from 70 to 100%. Bias, ARMS and precision were evaluated using Bland-Altman plots.
Results
The mean (SD) differences between SaO2 compared to SpO2 and the devices were as follows: SaO2 versus Masimo 2,12 ± 1,01% (95% CI 1,45 to 2,79), SaO2 versus Nellcor 0,78 ± 0,58% (95% CI − 0,29 to 1,65) and SaO2 versus SmartCardia 0,42 ± 0,24% (95% CI − 0,64 to 1,46). The bias between SmartCardia, Masimo, Nellcor devices and SaO2 was 0.16 (95% CI 0.05 to 0.33) and LoA (level of agreements) 2.96 (95% CI − 2,68 to 2,89) for SmartCardia, 2,02 (95% CI 1,49 to 2,54) and LoA − 6 to 11 for Masimo, and 0,76 (95% CI 0,5 to − 1) and LoA − 3,5 to 5,0 for Nellcor. ARMS for the 70–100% SaO2 range was 1,4 for SmartCardia, 5,0 for Masimo and 2,31 for Nellcor.
Conclusions
The new wireless SmartCardia SpO2 measurement system demonstrated in-line results, bias, ARMS and precision in healthy volunteers, when compared with the gold standard SaO2 and with two well established systems, Masimo and Nellcor.
Trial registration
The present trial was prospectively registered at UCSF record (registration number:10–00437), on March 8, 2021.
Similar content being viewed by others
Introduction
Pulse oximetry is a procedure for measuring the level of oxygen in the blood expressed as oxygen saturation [1,2,3]. This parameter is widely used in the evaluation of various medical conditions that affect the function of the heart and lungs and is a standard monitoring modality during anesthesia delivery [4,5,6].
Conventional oximeters use either the reflective or the transmitive method to measure SpO2. The reflective method is known to be less accurate. For this reason, most commercially available patches do not provide SpO2 measurements in addition to cardiac monitoring. The transmissive method is the more commonly used of the two. Transmitive technology transmits red and infrared light through the finger to a photo detector. The pulse oximeter sensor reads the transmitted light beams to estimate oxygen saturation of the blood and pulse rate. This method positions the transmitter and receiver in the same plane and the sensors can be placed on areas of the anatomy other than the finger. Conventional pulse oximeters have various limitations including false alarm and failed measurements [7,8,9,10,11] which may be related to positioning of the device located on the finger. A new generation of reflective pulse oximetry device (SmartCardia oximeter) has been designed on a wireless basis [12, 13]. The present trial was designed to validate this technology and to demonstrate the potential of this device using the reflective method. We compared this new device to the Nellcor N-600 (Covidien, Boulder, CO, USA) and the Masimo Radical device (Masimo Corp., Irvine, CA, USA), both units considered the two most used and accurate FDA approved non-invasive devices for SpO2 measurements. SaO2 was used as the gold standard.
Methods
Ethical considerations
Even if the device was developed in Switzerland the trial was conducted in a reference laboratory accepted by the FDA, the Hypoxia Research Laboratory at UCSF. The study was reviewed and approved by the UCSF Ethical Committee on Human Research. The Approval number is 10–00437, expiring on March 8th, 2021. The approval letter is on file at UCSF. The laboratory conforms to Good Clinical Practice Standards for the involvement of human subjects and handling of test data. Written informed consent was obtained from each participant.
System description
The SmartCardia 7 L device is a wireless patch with a low-cost disposable component and a re-usable electronic unit (Fig. 1). The patch acquires ECG and measures heart rate (HR) and SpO2. The data are transmitted by Bluetooth to a mobile phone or router. The measured signals and parameters are also stored on the device. The device for SpO2 recording is placed on the left arm of the subject (Fig. 1). The size of the patch is 55 × 130 mm. The patch-based device offers up to 14-days monitoring and data storage and 7-days real-time connectivity with the cloud through a smartphone. The ability to receive, store, and interpret a broad range of signals offers the opportunity to go far beyond monitoring individual parameters. If the patient’s vital measurements reach a pathological value, the system gives an alert on the cloud and the physician can see the real-time parameters and ECG signals.
Protocol
The protocol required brief stable arterial oxygen desaturation in healthy volunteers and sampling arterial blood when a stable level of hypoxia has been attained. The blood sample was analyzed for oxygen saturation with a gold standard bench CO-oximeter, currently a Radiometer ABL-90 multi-wavelength oximeter (Hemoximeter, Radiometer, Copenhagen (Denmark), serial 1393-090R0359N0002). This instrument contains factory certified calibration standards and quality control algorithms. Twenty to twenty-five arterial blood samples from each subject can be analyzed following a protocol aligned with current ISO and FDA guidance documents for pulse oximeter testing.
A radial arterial cannula was placed in either the left or right wrist of each subject for arterial blood sampling and blood pressure monitoring.
Our approach to obtaining stable, safe, and controlled hypoxia was breath-by-breath respiratory gas analysis. A computer program permits the inspired gas mixture to be adjusted to achieve a level of lung alveolar gas that will achieve the desired degree of hypoxia. This was obtained with the use of a nonerebreathing circuit with CO2 removal. Typically, saturation is determined once with air breathing and then at each of the following levels, e.g., 93, 90, 87, 85, 82, 80, 77, 75 and 70% saturation for about 30–60 s at each level. An arterial blood sample is obtained from the catheter at the end of each hypoxic plateau. The operator changes the inspired oxygen concentration at the end of each blood sampling to attain the next desired steady-state hypoxic condition. A run takes 10–15 min, and each run is terminated by a breath of 100% O2 followed by room air. Two runs together enable obtaining a total of 20–25 blood samples, two samples at each different plateau. Saturation of each arterial blood sample is determined by direct oximetry using the Radiometer ABL-90 multi- wavelength oximeter.
Sites for affixing pulse oximeter probes was the fingers for the Masimo device, the ears for the Nellcor, and the arm for the SmartCardia. In order to avoid inaccurate readings due to head or fingers movements with these 2 devices, the arm of the subjects was fixed during the measurements period. In addition, the participants were asked not to move their heads, as much as possible for the duration of the measurements.
Statistics
Pulse oximeter data was taken as 5 s averages corresponding to the point of SaO2 analysis. Individual data points may be missed or excluded for dropped signals or failure of the oximeter signal to achieve an appropriate plateau. Agreement in SpO2 and arterial SaO2 were analyzed using Bland Altman analysis [14, 15]. Bland Altman curve gives a graphical representation of the agreement between SpO2 and arterial SaO2 value. The average of the SpO2 and SaO2 is plotted on the x-axis while the difference between the two values is plotted on the y-axis. The more the graph points toward a zero difference with narrow dispersion of the ‘body’, the better the agreement.
Tables of mean, standard deviation, standard error, minimum, maximum, 95% CI, count and root mean square are provided for each oximeter’s bias, and all oximeters combined in the following ranges of SaO2: 70–100%, 60–70%, 70–80%, 80–90%, and 90–100%. LoA was obtained between methods of measurement with multiple observations per individual [16]. Root mean square error (ARMS) was calculated as it represents the best way to compare SaO2 measurements. It includes both values of SaO2 and its stability. The following formula was used for calculation:
Data were managed on MS excel spreadsheet and analysed using stata 9.0 software (Stata Corp, College Station, TX). On Bland Altman curves linear regression is shown for all subjects combined and the equation with R2 is shown on the plot. Mean bias is displayed as a solid horizontal line, and the upper and lower limits of agreement (mean bias ±1.96•SD*) are shown by dashed horizontal lines. For “pooled” plots, different markers are used for each pulse oximeter. Continuous variables were compared using ‘t’ test. A p value < 0.05 was considered as significant.
Results
The current study was performed on 4/13/2021 and 4/14/2021 at UCSF. Twelve volunteers with a mean age of 28 (range 21–29) (3 women and 9 men) were included in the trial (Table 1). All subjects enrolled in the study had normal Hb level (Hb ≥ 10 g/l) and all were healthy and non-smoking individuals.
A total of 298 samples were obtained at the saturation plateaus during the study. Data were collected for each type of oximeter and probes. Mean (SD) differences measurements between SaO2 and SpO2 and between the devices tested were as follow: SaO2 versus Masimo 2,12 ± 1,01% (95% confidence interval CI, 1,45 – 2,79), SaO2 versus Nellcor 0,78 ± 0,58% (CI, − 0,29 - 1,65) and SaO2 versus SmartCardia 0,42 ± 0,24% (95% CI, − 0,64 - 1,46). A statistically significant difference in precision was observed between the SmartCardia and the Masimo device (p < 0.001), between the SmartCardia and the Nellcor device (p = 0.03) and between the Nellcor and the Masimo (p < 0.001) (Table 1 and Fig. 2).
Bias, precision and ARMS are presented in Table 2. Plots assessing the agreement between arterial SaO2 and SpO2 detected by the SmartCardia, the Masimo and the Nellcor pulse oximeters are shown in Fig. 2. The bias between SmartCardia, Masimo, Nellcor and SaO2 was as follows: 0,16 (95% CI 0.05 to 0,33) and the LoA were 2,96 (95% CI − 2,68 to 2,89) for SmartCardia, 2,02 (95% CI 1,49 to 2,54) and the LoA were − 6 to 11 for Masimo, 0,76 (95% CI 0,5 to − 1) and the LoA were − 3,5 to 5,0 for Nellcor. This bias is significantly lower for SmartCardia compared to Masimo (p < 0.001) and Nellcor (p < 0.05), and for Nellcor compared with Masimo (p < 0.05). ARMS for the 70–100% SaO2 range was 1,4 for the SmartCardia device, 5,0 for the Masimo device and 2,31 for the Nellcor device (Table 2 and Fig. 3).
Bland-Altman curve showing agreement between a: SmartCardia, b: Masimo and c: Nellcor, SpO2 and arterial SpO2. Data is plotted as Hemoximeter data (SaO2) vs. pulse oximeter bias (SpO2 – SaO2). A different marker is used for each study subject. Linear regression is shown for all subjects combined, and the equation with R2 is shown on the plot. Mean bias is displayed as a solid horizontal line, and the upper and lower LoA (mean bias ±1.96•SD*) are shown by dashed horizontal lines
Discussion
Currently, arterial oxygenation is primarily measured using pulse oximeters that provide immediate and continuous non-invasive surveillance for SpO2. Several devices are available in clinics with some limitations, including false alarms and failed measurements when used in patients with low perfusion or during patient motion. The positioning of the fingertips makes measurements at times difficult and impractical for patients. Some trials showed varying sensitivity and accuracy in patients with hypotension and hypoxemia [14, 16], primarily due to the location of the device. We compared the performance of two well-established pulse oximeters to a next-generation pulse oximeter, the SmartCardia device. This wireless system is located on the patient’s arm and the values are transmitted electronically and stored in the cloud, both representing major benefits over other devices. Furthermore, today, as a result of the COVID-19 pandemic, the measurement of SpO2, even outside the hospital, has become increasingly important. We have shown that despite the use of the usually less accurate reflective method, the SmartCardia device provides very precise and precise results when compared to well-established devices in clinical use. Limitations due to the presence of wired connections are avoided, and patient hand movement is not limited by the presence of a finger mounted sensor. Furthermore, the oximeter can be worn during walking with permanent transmission of data to a control station or a phone providing reliable electronic wifi transmission. This allows access to permanent and uninterrupted information regarding patient oxygenation levels. To confirm the quality of the device, which is currently awaiting FDA approval, its performance was compared with the Masimo and Nellcor pulse oximeter devices. Both use the transductive method. In our study, the results are encouraging and equivalent or even slightly better based on bias, ARMS and precision measurements. We also showed that this system provides good SpO2 assessments over the range of SaO2 tested between 70 to 100% oxygen saturation.
Our trial involved young healthy volunteer subjects in a reference laboratory setting. This allowed the use of a more rigorous protocol while maintaining control over variables.
As mentioned before, this system is placed on the subject’s arm which is an advantage due to fewer disturbances related to the hand’s movements. Devices placed on the ear are also more prone to disturbances related to movements of the patient’s head. We found significant performance differences among the 3 devices tested. We showed that the Masimo device tended towards 2 to 3% higher SpO2 values than actual SaO2. This may be clinically relevant if confirmed in ICU patients. As our trial was not performed in a clinical setting, we cannot conclude that one oximeter would be better than the others in daily practice. Further trials in real life situations, such as ICU environments, hypotensive or hypothermic conditions, etc. are needed to further draw definitive conclusions. It is our experience that even well simulated clinical scenarios often lead to ambiguous results. We observed a significant difference in the measurement of SpO2 between the Nellcor and the Masimo in favor of the Nellcor system. However, our data were collected in healthy volunteers. Furthermore, previous trials comparing the Masimo, the Nellcor and the Philips Intellivue have not shown significant differences between these three pulse oximeters when tested in ICU patients [16, 17]. Conversely, other trials conclude to a higher sensitivity and specificity of the Masimo device when compared to the Nellcor system [18]. These divergent results clearly show the need for further studies utilizing real life environments and patient’s conditions to further corroborate our results.
Study limitation
The major limitation is that we performed this trial in a laboratory. This may not fully replicate the characteristics of complex clinical settings exhibited by patient populations in the ICU or operating theater. In addition, the trial was conducted with a small number of patients. These results need confirmation in a larger cohort of subjects in a clinical setting.
Conclusion
The new wireless SmartCardia SpO2 measurement system demonstrated in-line results, bias, ARMS and precision in healthy volunteers, when compared with the gold standard SaO2 and with two well established systems, Masimo and Nellcor.
Availability of data and materials
Data from the trail are stored on the UCSF files and can be consulted at any time. The datasets used and/or analysed during the current study available from the corresponding author or from Francisco Rincon on reasonable request.’
Abbreviations
- Hb:
-
Hemoglobin blood level
- UCSF:
-
University of California, San Francisco
- CI:
-
Confidence interval
- ICU:
-
Intensive care unit
- SpO2:
-
Saturation measured by the tested oximetry device
- SaO2:
-
Saturation measured by the Hemoximeter
- LoA:
-
Limits of agreements
- ARMS:
-
Root mean square error
References
Jubran A. Pulse oximetry. Intensive Care Med. 2004;30:2017.
Neff TA. Routine oximetry. A fifth vital sign? Chest. 1988;94:227.
Chan ED, Chan MM, Chan MM. Pulse oximetry: understanding its basic principles facilitates appreciation of its limitations. Respir Med. 2013;107:789.
Eichhorn JH, Cooper JB, Cullen DJ, et al. Standards for patient monitoring during anesthesia at Harvard Medical School. JAMA. 1986;256:1017–20.
Runciman W. International Task Force on Anaesthesia Safety. Eur J Anaesthesiol Suppl. 1993;10:12–5.
World Federation of Societies of Anaesthesiologists. International Standard for a Safe Practice of Anaesthesia. 2008. Available at: practice-of-anaesthesia.html. Accessed on 8 Sept 2008.
Lawson D, Norley I, Korbon G, Loeb R, Ellis J. Blood flow limits and pulse oximeter signal detection. Anesthesiology. 1987;67:599–603.
Severinghaus JW, Spellman MJ Jr. Pulse oximeter failure thresholds in hypotension and vasoconstriction. Anesthesiology. 1990;73:532–7.
Wilson S. Conscious sedation and pulse oximetry: false alarms? Pediatr Dent. 1990;12:228–32.
Plummer JL, Zakaria AZ, Ilsley AH, Fronsko RR, Owen H. Evaluation of the influence of movement on saturation readings from pulse oximeters. Anaesthesia. 1995;50:423–6.
Wiklund L, Hök B, Ståhl K, Jordeby-Jönsson A. Post anesthesia monitoring revisited: incidence of true and false alarms from different monitoring devices. J Clin Anesth. 1994;6:182–8.
Murali S, Rincon F, Cassina T, Cook S, Goy JJ. Heart Rate and Oxygen Saturation Monitoring with a New Wearable Wireless Device in the Intensive Care Unit: Pilot Comparison Trial. JMIR Biomed Eng. 2020;5(1):e18158.
Murali S, Brugger N, Rincon F, Mashru M, Cook S, Goy JJ. Cardiac ambulatory monitoring: new wireless device validated against conventional Holter monitoring in a case series. Front Cardiovasc Med. 2020;7:587945. https://doi.org/10.3389/fcvm.
Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60.
Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82.
Ahrens T, Ott K. Comparison of three new generation pulse oximeters in a medical intensive care unit. Crit. Care Med. 2006;34:A 85.
Shah N, Ragaswamy H, Govindugari K, Estanol L. Performance of the three new-generation pulse oximeters during motion and low perfusion in volunteers. J Clin Anesth. 2012;24:385–91.
Bipin J, Rakesh L, Kabra S. Comparison of two new generation pulse oximeters with arterial oxygen saturation in critically ill children. Indian J Pediatr. 2014;81:1297–301.
Acknowledgements
not applicable.
Funding
The only funding body was SmartCardia. SmartCardia didn’t play any role in the design of the study, as the study was designed taking into account only FDA requirements for SpO2 validation. SmartCardia collected data from its device only, and the lab where the study took place collected the gold standard data only. After the study, SmartCardia shared all data with the lab in a completely blind manner (SmartCardia never had access to the gold standard data), and the lab performed the analysis and interpretation of the data.
Author information
Authors and Affiliations
Contributions
All authors have read and approved the manuscript. Francisco Rincon and Srini Murali collected the data. Redaction of the manuscript was done by Jean-Jacques Goy and statistics performed by Julien Pidoux and S. Murali.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The study was reviewed and approved by the UCSF Ethical Committee on Human Research. The approval letter is on file at UCSF. The laboratory conforms to Good Clinical Practice Standards for the involvement of human subjects and handling of test data. Written informed consent was obtained from each participant.
Consent for publication
Not applicable.
Competing interests
Francisco Rincon and Srini Murali are developers of the SmartCardia oximeter and member of the board of SmartCardia. The other authors have no competing interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Rincon, F., Pidoux, J., Murali, S. et al. Performance of the new SmartCardia wireless, wearable oximeter: a comparison with arterial SaO2 in healthy volunteers. BMC Anesthesiol 22, 77 (2022). https://doi.org/10.1186/s12871-022-01604-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12871-022-01604-w