Advertisement

Annals of Biomedical Engineering

, Volume 42, Issue 4, pp 885–898 | Cite as

Respiratory Rate Estimation from the Built-in Cameras of Smartphones and Tablets

  • Yunyoung Nam
  • Jinseok Lee
  • Ki H. Chon
Article

Abstract

This paper presents a method for respiratory rate estimation using the camera of a smartphone, an MP3 player or a tablet. The iPhone 4S, iPad 2, iPod 5, and Galaxy S3 were used to estimate respiratory rates from the pulse signal derived from a finger placed on the camera lens of these devices. Prior to estimation of respiratory rates, we systematically investigated the optimal signal quality of these 4 devices by dividing the video camera’s resolution into 12 different pixel regions. We also investigated the optimal signal quality among the red, green and blue color bands for each of these 12 pixel regions for all four devices. It was found that the green color band provided the best signal quality for all 4 devices and that the left half VGA pixel region was found to be the best choice only for iPhone 4S. For the other three devices, smaller 50 × 50 pixel regions were found to provide better or equally good signal quality than the larger pixel regions. Using the green signal and the optimal pixel regions derived from the four devices, we then investigated the suitability of the smartphones, the iPod 5 and the tablet for respiratory rate estimation using three different computational methods: the autoregressive (AR) model, variable-frequency complex demodulation (VFCDM), and continuous wavelet transform (CWT) approaches. Specifically, these time-varying spectral techniques were used to identify the frequency and amplitude modulations as they contain respiratory rate information. To evaluate the performance of the three computational methods and the pixel regions for the optimal signal quality, data were collected from 10 healthy subjects. It was found that the VFCDM method provided good estimates of breathing rates that were in the normal range (12–24 breaths/min). Both CWT and VFCDM methods provided reasonably good estimates for breathing rates that were higher than 26 breaths/min but their accuracy degraded concomitantly with increased respiratory rates. Overall, the VFCDM method provided the best results for accuracy (smaller median error), consistency (smaller interquartile range of the median value), and computational efficiency (less than 0.5 s on 1 min of data using a MATLAB implementation) to extract breathing rates that varied from 12 to 36 breaths/min. The AR method provided the least accurate respiratory rate estimation among the three methods. This work illustrates that both heart rates and normal breathing rates can be accurately derived from a video signal obtained from smartphones, an MP3 player and tablets with or without a flashlight.

Keywords

Respiratory rate estimation Autoregressive model Continuous wavelet transform Variable frequency complex demodulation method Smartphone Tablet 

Notes

Acknowledgments

This work was supported in part by the US Army Medical Research and Materiel Command (USAMRMC) under Grant No. W81XWH-12-1-0541.

References

  1. 1.
    Allison, R., E. Holmes, and J. Nyboer. Volumetric dynamics of respiration as measured by electrical impedance plethysmography. J. Appl. Physiol. 19:166–173, 1964.PubMedGoogle Scholar
  2. 2.
    Chon, K. H., S. Dash, and K. Ju. Estimation of respiratory rate from photoplethysmogram data using time–frequency spectral estimation. Biomed. Eng. IEEE Trans. 56:2054–2063, 2009.CrossRefGoogle Scholar
  3. 3.
    Fieselmann, J. F., M. S. Hendryx, C. M. Helms, and D. S. Wakefield. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J. Gen. Intern. Med. 8:354–360, 1993.PubMedCrossRefGoogle Scholar
  4. 4.
    Grimaldi, D., Y. Kurylyak, F. Lamonaca, and A. Nastro. Photoplethysmography detection by smartphone’s videocamera. In: The 6th IEEE International Conference on Intelligent Data Acquisition and Advance Computing Systems: Technology and Applications, 2011.Google Scholar
  5. 5.
    Hasselgren, M., M. Arne, A. Lindahl, S. Janson, and B. Lundbäck. Estimated prevalences of respiratory symptoms, asthma and chronic obstructive pulmonary disease related to detection rate in primary health care. Scand. J. Prim. Health Care 19:54–57, 2001.PubMedCrossRefGoogle Scholar
  6. 6.
    Hirsch, J., and B. Bishop. Respiratory sinus arrhythmia in humans: how breathing pattern modulates heart rate. Am. J. Physiol. Heart Circ. Physiol. 241:H620–H629, 1981.Google Scholar
  7. 7.
    Lee, J., and K. Chon. Respiratory rate extraction via an autoregressive model using the optimal parameter search criterion. Ann. Biomed. Eng. 38:3218–3225, 2010.PubMedCrossRefGoogle Scholar
  8. 8.
    Lee, J., and K. H. Chon. An autoregressive model-based particle filtering algorithms for extraction of respiratory rates as high as 90 breaths per minute from pulse oximeter. Biomed. Eng. IEEE Trans. 57:2158–2167, 2010.CrossRefGoogle Scholar
  9. 9.
    Lee, J., and K. H. Chon. Time-varying autoregressive model-based multiple modes particle filtering algorithm for respiratory rate extraction from pulse oximeter. Biomed. Eng. IEEE Trans. 58:790–794, 2011.CrossRefGoogle Scholar
  10. 10.
    Lee, J., J. P. Florian, and K. H. Chon. Respiratory rate extraction from pulse oximeter and electrocardiographic recordings. Physiol. Meas. 32:1763, 2011.PubMedCrossRefGoogle Scholar
  11. 11.
    Leonard, P., T. Beattie, P. Addison, and J. Watson. Standard pulse oximeters can be used to monitor respiratory rate. Emerg. Med. J. 20:524–525, 2003.PubMedCentralPubMedCrossRefGoogle Scholar
  12. 12.
    Leonard, P. A., D. Clifton, P. S. Addison, J. N. Watson, and T. Beattie. An automated algorithm for determining respiratory rate by photoplethysmogram in children. Acta Paediatr. 95:1124–1128, 2006.PubMedCrossRefGoogle Scholar
  13. 13.
    Leonard, P., N. R. Grubb, P. S. Addison, D. Clifton, and J. N. Watson. An algorithm for the detection of individual breaths from the pulse oximeter waveform. J. Clin. Monit. Comput. 18:309–312, 2004.PubMedCrossRefGoogle Scholar
  14. 14.
    Liu, H., Y. Wang, and L. Wang. A review of non-contact, low-cost physiological information measurement based on photoplethysmographic imaging. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012. doi: 10.1109/EMBC.2012.6346371.Google Scholar
  15. 15.
    Lu, S., K. H. Ju, and K. H. Chon. A new algorithm for linear and nonlinear ARMA model parameter estimation using affine geometry. Biomed. Eng. IEEE Trans. 48:1116–1124, 2001.CrossRefGoogle Scholar
  16. 16.
    Mason, K. P., P. E. Burrows, M. M. Dorsey, D. Zurakowski, and B. Krauss. Accuracy of capnography with a 30 foot nasal cannula for monitoring respiratory rate and end-tidal CO2 in children. J. Clin. Monit. Comput. 16:259–262, 2000.PubMedCrossRefGoogle Scholar
  17. 17.
    McManus, D. D., J. Lee, O. Maitas, N. Esa, R. Pidikiti, A. Carlucci, J. Harrington, E. Mick, and K. H. Chon. A novel application for the detection of an irregular pulse using an iPhone 4S in patients with atrial fibrillation. Heart Rhythm 10:315–319, 2013.PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Rantonen, T., J. Jalonen, J. Grönlund, K. Antila, D. Southall, and I. Välimäki. Increased amplitude modulation of continuous respiration precedes sudden infant death syndrome:–Detection by spectral estimation of respirogram. Early Hum. Dev. 53:53–63, 1998.PubMedCrossRefGoogle Scholar
  19. 19.
    Scully, C. G., J. Lee, J. Meyer, A. M. Gorbach, D. Granquist-Fraser, Y. Mendelson, and K. H. Chon. Physiological parameter monitoring from optical recordings with a mobile phone. Biomed. Eng. IEEE Trans. 59:303–306, 2012.CrossRefGoogle Scholar
  20. 20.
    South, M. Measurement of respiratory rate and timing using a nasal thermocouple. J. Clin. Monit. 11:159–164, 1995.PubMedCrossRefGoogle Scholar
  21. 21.
    Subbe, C., R. Davies, E. Williams, P. Rutherford, and L. Gemmell. Effect of introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions*. Anaesthesia 58:797–802, 2003.PubMedCrossRefGoogle Scholar
  22. 22.
    Torrence, C., and G. P. Compo. A practical guide to wavelet analysis. Bull. Am. Meteorol. Soc. 79:61–78, 1998.CrossRefGoogle Scholar
  23. 23.
    Wang, H., K. Siu, K. Ju, and K. H. Chon. A high resolution approach to estimating time-frequency spectra and their amplitudes. Ann. Biomed. Eng. 34:326–338, 2006.PubMedCrossRefGoogle Scholar
  24. 24.
    Younes, M. Role of respiratory control mechanisms in the pathogenesis of obstructive sleep disorders. J. Appl. Physiol. 105:1389–1405, 2008.PubMedCrossRefGoogle Scholar

Copyright information

© Biomedical Engineering Society 2013

Authors and Affiliations

  1. 1.Department of Biomedical EngineeringWorcester Polytechnic InstituteWorcesterUSA
  2. 2.Department of Biomedical EngineeringWonkwang University School of MedicineIksanRepublic of Korea

Personalised recommendations