Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria

Abstract

We propose, in this study, an original method that was developed to remotely measure the instantaneous pulse rate using photoplethysmographic signals that were recorded from a low-cost webcam. The method is based on a prior selection of pixels of interest using a custom segmentation that used the face lightness distribution to define different sub-regions. The most relevant sub-regions are automatically selected and combined by evaluating their respective signal to noise ratio. Performances of the proposed technique were evaluated using an approved contact sensor on a set of seven healthy subjects. Different experiments while reading, with motion or while performing common tasks on a computer were conducted in the laboratory. The proposed segmentation technique was compared with other benchmark methods that were already introduced in the scientific literature. The results exhibit high degrees of correlation and low pulse rate absolute errors, demonstrating that the segmentation we propose in this study outperform available region-of-interest selection methods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Humphreys, K., Ward, T., & Markham, C. (2007). Noncontact simultaneous dual wavelength photoplethysmography: A further step toward noncontact pulse oximetry. Review of Scientific Instruments, 78, 044304.

    Article  Google Scholar 

  2. 2.

    Kranjec, J., Beguš, S., Geršak, G., & Drnovšek, J. (2014). Non-contact heart rate and heart rate variability measurements: A review. Biomedical Signal Processing and Control, 13, 102–112.

    Article  Google Scholar 

  3. 3.

    Poh, M. Z., McDuff, D., & Picard, R. W. (2011). Advancements in noncontact, multiparameter physiological measurements using a Webcam. IEEE Transactions on Biomedical Engineering, 58, 7–11.

    Article  Google Scholar 

  4. 4.

    McDuff, D., Gontarek, S., & Picard, R. W. (2014). Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering, 61, 2948–2954.

    Article  Google Scholar 

  5. 5.

    Tarassenko, L., Villarroel, M., Guazzi, A., Jorge, J., Clifton, D. A., & Pugh, C. (2014). Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiological Measurement, 35, 807–831.

    Article  Google Scholar 

  6. 6.

    Monkaresi, H., Calvo, R. A., & Hong, Y. (2014). A machine learning approach to improve contactless heart rate monitoring using a Webcam. IEEE Journal of Biomedical and Health Informatics, 18, 1153–1160.

    Article  Google Scholar 

  7. 7.

    Kong, L., Zhao, Y., Dong, L., Jian, Y., Jin, X., Li, B., et al. (2013). Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. Optics Express, 21, 17464–17471.

    Article  Google Scholar 

  8. 8.

    Shao, D., Yang, Y., Liu, C., Tsow, F., Yu, H., & Tao, N. (2014). Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time. IEEE Transactions on Biomedical Engineering, 61, 2760–2767.

    Article  Google Scholar 

  9. 9.

    Task Force of the European Society of Cardiology, North American Society of Pacing and Electrophysiology. (1996). Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043–1065.

    Article  Google Scholar 

  10. 10.

    Aarts, L. A. M., Jeanne, V., Cleary, J. P., Lieber, C., Nelson, J. S., Oetomo, S. B., et al. (2013). Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—A pilot study. Early Human Development, 89, 943–948.

    Article  Google Scholar 

  11. 11.

    Sun, Y., Hu, S., Azorin-Peris, V., Kalawsky, R., & Greenwald, S. (2013). Noncontact imaging photoplethysmography to effectively access pulse rate variability. Journal of Biomedial Optics, 18, 061205.

    Article  Google Scholar 

  12. 12.

    Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6, 156–166.

    Article  Google Scholar 

  13. 13.

    Hernandez, J., Morris, R., & Picard, R. (2011). Call Center Stress Recognition with Person-Specific Models. In S. Dmello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective Computing and Intelligent Interaction (Vol. 6974, pp. 538–547). Berlin: Springer.

    Google Scholar 

  14. 14.

    Hoover, A., Singh, A., Fishel-Brown, S., & Muth, E. (2012). Real-time detection of workload changes using heart rate variability. Biomedical Signal Processing and Control, 7, 333–341.

    Article  Google Scholar 

  15. 15.

    Melillo, P., Bracale, M., & Pecchia, L. (2011). Nonlinear heart rate variability features for real-life stress detection. Case study: students under stress due to university examination. BioMedical Engineering OnLine, 10, 96.

    Article  Google Scholar 

  16. 16.

    Yoshino, K., & Matsuoka, K. (2012). Personal adaptive method to assess mental tension during daily life using heart rate variability. Methods of Information in Medicine, 51, 39–44.

    Article  Google Scholar 

  17. 17.

    Pavlidis, I., Dowdall, J., Sun, N., Puri, C., Fei, J., & Garbey, M. (2007). Interacting with human physiology. Computer Vision and Image Understanding, 108, 150–170.

    Article  Google Scholar 

  18. 18.

    McDuff, D., Gontarek, S., & Picard, R. (2014). Remote measurement of cognitive stress via heart rate variability. Presented in the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Chicago, USA (pp. 2957–2960).

  19. 19.

    Bousefsaf, F., Maaoui, C., & Pruski, A. (2014). Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam. Computers in Biology and Medicine, 53, 154–163.

    Article  Google Scholar 

  20. 20.

    Stricker, R., Muller, S., & Gross, H. M. (2014). Non-contact video-based pulse rate measurement on a mobile service robot. In IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK (pp. 1056–1062).

  21. 21.

    Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA (pp. I-511–I-518).

  22. 22.

    Holton, B. D., Mannapperuma, K., Lesniewski, P. J., & Thomas, J. C. (2013). Signal recovery in imaging photoplethysmography. Physiological Measurement, 34, 1499–1511.

    Article  Google Scholar 

  23. 23.

    Sun, Y., Hu, S., Azorin-Peris, V., Greenwald, S., Chambers, J., & Zhu, Y. (2011). Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise. Journal of Biomedial Optics, 16, 077010.

    Article  Google Scholar 

  24. 24.

    Takano, C., & Ohta, Y. (2007). Heart rate measurement based on a time-lapse image. Medical Engineering & Physics, 29, 853–857.

    Article  Google Scholar 

  25. 25.

    Feng, L., Po, L. M., Xu, X., Li, Y., & Ma, R. (2014). Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Transactions on Circuits and Systems for Video Technology, 25, 879–891.

    Article  Google Scholar 

  26. 26.

    Lewandowska, M., Rumiński, J., Kocejko, T., & Nowak, J. (2011). Measuring pulse rate with a Webcam—A non-contact method for evaluating cardiac activity. Presented at the Federated Conference on Computer Science and Information Systems, Szczecin, Poland (pp. 405–410).

  27. 27.

    Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008). Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding, 110, 346–359.

    Article  Google Scholar 

  28. 28.

    Shi, J., & Tomasi, C. (1994). Good features to track. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, USA (pp. 593–600).

  29. 29.

    Li, X., Chen, J., Zhao, G., & Pietikainen, M. (2014). Remote heart rate measurement from face videos under realistic situations. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, USA (pp. 4264–4271).

  30. 30.

    Bousefsaf, F., Maaoui, C., & Pruski, A. (2013). Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate. Biomedical Signal Processing and Control, 8, 568–574.

    Article  Google Scholar 

  31. 31.

    Mahmoud, T. M. (2008). A new fast skin color detection technique. World Academy of Science, Engineering and Technology, 43, 501–505.

    Google Scholar 

  32. 32.

    Bal, U. (2014). Non-contact estimation of heart rate and oxygen saturation using ambient light. Biomedical Optics Express, 6, 86–97.

    Article  Google Scholar 

  33. 33.

    Sahindrakar, P., Haan, G., & Kirenko, I. (2011). Improving motion robustness of contact-less monitoring of heart rate using video analysis. Graduation Project, Eindhoven University of Technology, Eindhoven, Netherland. http://alexandria.tue.nl/extra1/afstversl/wsk-i/sahindrakar2011.pdf.

  34. 34.

    Verkruysse, W., Svaasand, L. O., & Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics Express, 16, 21434–21445.

    Article  Google Scholar 

  35. 35.

    Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M., Incremental face alignment in the wild. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, USA (pp. 1859–1866).

  36. 36.

    Poynton, C. (2003). The CIE system of colorimetry. In J. Schanda (Ed.), Digital Video and HDTV: Algorithms and interfaces (pp. 211–231). San Francisco: Morgan Kaufmann Publishers Inc.

    Google Scholar 

  37. 37.

    Tarvainen, M. P., Ranta-Aho, P. O., & Karjalainen, P. A. (2002). An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 49, 172–175.

    Article  Google Scholar 

  38. 38.

    Fitzpatrick, T. B. (1975). Sun and skin (Soleil et peau). J. Med. Esthétique, 2, 33–34.

    Google Scholar 

  39. 39.

    Smith, A. R. (1978). Color gamut transform pairs. Presented at the 5th annual conference on Computer graphics and interactive techniques (pp. 12–19). New York: ACM Press.

  40. 40.

    Hajiarbabi, M., & Agah, A. (2014). Face detection in color images using skin segmentation. Journal of Intelligent and Robotic Systems, 8, 41–51.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Frédéric Bousefsaf.

Ethics declarations

Conflict of interest

Frederic Bousefsaf, Choubeila Maaoui and Alain Pruski declare that they have no conflict of interest.

Ethical Approval

All procedures performed in this study were in accordance with the 1964 Helsinki declaration and its later amendments. Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bousefsaf, F., Maaoui, C. & Pruski, A. Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria. J. Med. Biol. Eng. 37, 374–385 (2017). https://doi.org/10.1007/s40846-017-0229-1

Download citation

Keywords

  • Remote photoplethysmography
  • Pulse rate
  • Webcam
  • Region of interest