Advertisement

Frontiers of Computer Science

, Volume 12, Issue 5, pp 858–872 | Cite as

Remote heart rate measurement using low-cost RGB face video: a technical literature review

  • Philipp V. Rouast
  • Marc T. P. Adam
  • Raymond Chiong
  • David Cornforth
  • Ewa Lux
Review Article

Abstract

Remote photoplethysmography (rPPG) allows remote measurement of the heart rate using low-cost RGB imaging equipment. In this study, we review the development of the field of rPPG since its emergence in 2008. We also classify existing rPPG approaches and derive a framework that provides an overview of modular steps. Based on this framework, practitioners can use our classification to design algorithms for an rPPG approach that suits their specific needs. Researchers can use the reviewed and classified algorithms as a starting point to improve particular features of an rPPG algorithm.

Keywords

affective computing heart rate measurement remote non-contact camera-based photoplethysmography 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

This work was supported by a fellowship within the FITweltweit programme of the German Academic Exchange Service (DAAD).

Supplementary material

11704_2016_6243_MOESM1_ESM.ppt (482 kb)
Supplementary material, approximately 482 KB.

References

  1. 1.
    Zhang Z L, Pi Z Y, Liu B Y. Troika: a general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 522–531CrossRefGoogle Scholar
  2. 2.
    Adam M T P, Krämer J, Weinhardt C. Excitement up! Price down! Measuring emotions in dutch auctions. International Journal of Electronic Commerce, 2012, 13(2): 7–39CrossRefGoogle Scholar
  3. 3.
    Adam M T P, Krämer J, Müller MB. Auction fever! How time pressure and social competition affect bidders’ arousal and bids in retail auctions. Journal of Retailing, 2015, 91(3): 468–485CrossRefGoogle Scholar
  4. 4.
    Astor P J, Adam M T P, Jerčić P, Schaaff K, Weinhardt C. Integrating biosignals into information systems: a neurois tool for improving emotion regulation. Journal of Management Information Systems, 2013, 30(3): 247–278CrossRefGoogle Scholar
  5. 5.
    Riedl R. On the biology of technostress: literature review and research agenda. ACM SIGMIS Database, 2013, 44(1): 18–55CrossRefGoogle Scholar
  6. 6.
    Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Optics Express, 2008, 16(26): 21434–21445CrossRefGoogle Scholar
  7. 7.
    McDuff D J, Estepp J R, Piasecki A M, Blackford E B. A survey of remote optical photoplethysmographic imaging methods. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015, 6398–6404Google Scholar
  8. 8.
    Liu H, Wang Y D, Wang L. A review of non-contact, low-cost physiological information measurement based on photoplethysmographic imaging. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2088–2091Google Scholar
  9. 9.
    Kranjec J, Beguš S, Geršak G, Drnovšek J. Non-contact heart rate and heart rate variability measurements: a review. Biomedical Signal Processing and Control, 2014, 13(1): 102–112CrossRefGoogle Scholar
  10. 10.
    Poh M-Z, McDuff D J, Picard R W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express, 2010, 18(10): 10762–10774CrossRefGoogle Scholar
  11. 11.
    Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement, 2007, 28(3): R1–R39CrossRefGoogle Scholar
  12. 12.
    Lindberg L-G, Öberg P A. Optical properties of blood in motion. Optical Engineering, 1993, 32(2): 253–257CrossRefGoogle Scholar
  13. 13.
    Balakrishnan G, Durand F, Guttag J. Detecting pulse from head motions in video. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3430–3437Google Scholar
  14. 14.
    Starr I, Rawson A J, Schroeder H A, Joseph N R. Studies on the estimation of cardiac ouptut in man, and of abnormalities in cardiac function, from the heart’s recoil and the blood’s impacts; the ballistocardiogram. American Journal of Physiology, 1939, 127(1): 1–28Google Scholar
  15. 15.
    Hertzman A B, Spealman C R. Observations on the finger volume pulse recorded photoelectrically. American Journal of Physiology, 1937, 119(2): 334–335Google Scholar
  16. 16.
    Poh M-Z, McDuff D J, Picard RW. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 2011, 58(1): 7–11CrossRefGoogle Scholar
  17. 17.
    Lewandowska M, Ruminski J, Kocejko T. Measuring pulse rate with a webcam: a non-contact method for evaluating cardiac activity. In: Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS). 2011, 405–410Google Scholar
  18. 18.
    Kwon S, Kim H, Park K S. Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2174–2177Google Scholar
  19. 19.
    Lee K-Z, Hung P-C, Tsai L-W. Contact-free heart rate measurement using a camera. In: Proceedings of the 9th Conference on Computer and Robot Vision. 2012, 147–152Google Scholar
  20. 20.
    Wei L, Tian Y H, Wang Y W, Ebrahimi T, Huang T. Automatic webcam-based human heart rate measurements using laplacian eigenmap. In: Proceedings of Asian Conference on Computer Vision. 2013, 281–292Google Scholar
  21. 21.
    Wu H-Y, Rubinstein M, Shih E, Guttag J V, Durand F, Freeman W T. Eulerian video magnification for revealing subtle changes in the world. ACM Transactions on Graphics, 2012, 31(4): 1–8CrossRefGoogle Scholar
  22. 22.
    Wu H-Y. Eulerian video processing and medical applications. Dissertation for the Master Degree. Cambridge, MA: Massachusetts Institute of Technology, 2012Google Scholar
  23. 23.
    Shan L, Yu M H. Video-based heart rate measurement using head motion tracking and ica. In: Proceedings of the 6th International Congress on Image and Signal Processing. 2013, 160–164Google Scholar
  24. 24.
    Irani R, Nasrollahi K, Moeslund T B. Improved pulse detection from head motions using dct. In: Proceedings of the 9th International Conference on Computer Vision Theory and Applications. 2014, 118–124Google Scholar
  25. 25.
    De Haan G, Jeanne V. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 2013, 60(10): 2878–2886CrossRefGoogle Scholar
  26. 26.
    De Haan G, Van Leest A. Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiological Measurement, 2014, 35(9): 1913–1926CrossRefGoogle Scholar
  27. 27.
    Lempe G, Zaunseder S, Wirthgen T, Zipser S, Malberg H. ROI selection for remote photoplethysmography. In: Meinzer H-P, Deserno MT, Handels H, et al, eds. Bildverarbeitung für die Medizin 2013. Berlin: Springer, 2013, 99–103CrossRefGoogle Scholar
  28. 28.
    Datcu D, Cidota M, Lukosch S, Rothkrantz L. Noncontact automatic heart rate analysis in visible spectrum by specific face regions. In: Proceedings of the 14th International Conference on Computer Systems and Technologies. 2013, 120–127Google Scholar
  29. 29.
    Li X B, Chen J, Zhao G Y, Pietikäinen M. Remote heart rate measurement from face videos under realistic situations. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014, 4264–4271Google Scholar
  30. 30.
    McDuff D, Gontarek S, Picard RW. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(12): 2948–2954CrossRefGoogle Scholar
  31. 31.
    Kumar M, Veeraraghavan A, Sabharwal A. DistancePPG: robust noncontact vital signs monitoring using a camera. Biomedical Optics Express, 2015, 6(5): 1565–1588CrossRefGoogle Scholar
  32. 32.
    Tasli H E, Gudi A, Den Uyl M. Remote PPG based vital sign measurement using adaptive facial regions. In: Proceedings of IEEE International Conference on Image Processing. 2014, 1410–1414Google Scholar
  33. 33.
    Feng L T, Po L-M, Xu X Y, Li Y M. Motion artifacts suppression for remote imaging photoplethysmography. In: Proceedings of the 19th International Conference on Digital Signal Processing. 2014, 18–23Google Scholar
  34. 34.
    Feng L T, Po L-M, Xu X Y, Li Y M, Ma R Y. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(5): 879–891CrossRefGoogle Scholar
  35. 35.
    Wang W J, Stuijk S, De Haan G. Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 415–425CrossRefGoogle Scholar
  36. 36.
    McDuff D, Gontarek S, Picard R W. Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(10): 2593–2601CrossRefGoogle Scholar
  37. 37.
    Chwyl B, Chung A G, Deglint J, Wong A, Clausi D A. Remote heart rate measurement through broadband video via stochastic bayesian estimation. Vision Letters, 2015, 1(1): 5CrossRefGoogle Scholar
  38. 38.
    Monkaresi H, Calvo R A, Yan H. A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE Journal of Biomedical and Health Informatics, 2014, 18(4): 1153–1160CrossRefGoogle Scholar
  39. 39.
    Hsu Y, Lin Y L, Hsu W. Learning-based heart rate detection from remote photoplethysmography features. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2014, 4433–4437Google Scholar
  40. 40.
    Tran D N, Lee H, Kim C. A robust real time system for remote heart rate measurement via a camera. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2015, 1–6Google Scholar
  41. 41.
    Li M-C, Lin Y-H. A real-time non-contact pulse rate detector based on smartphone. In: Proceedings of International Symposium on Next-Generation Electronics. 2015, 1–3Google Scholar
  42. 42.
    Yu Y-P, Kwan B-H, Lim C-L, Wong S-L, Raveendran P. Video-based heart rate measurement using short-time fourier transform. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems. 2013, 704–707Google Scholar
  43. 43.
    Holton B D, Mannapperuma K, Lesniewski P J, Thomas J C. Signal recovery in imaging photoplethysmography. Physiological Measurement, 2013, 34(11): 1499–1511CrossRefGoogle Scholar
  44. 44.
    Xu S C, Sun L Y, Rohde G K. Robust efficient estimation of heart rate pulse from video. Biomedical Optics Express, 2014, 5(4): 1124–1135CrossRefGoogle Scholar
  45. 45.
    Feng L T, Po L-M, Xu X Y, Li Y M, Cheung C-H, Cheung K-W, Yuan F. Dynamic ROI based on K-means for remote photoplethysmography. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2015, 1310–1314Google Scholar
  46. 46.
    Han B, Ivanov K, Wang L, Yan Y. Exploration of the optimal skincamera distance for facial photoplethysmographic imaging measurement using cameras of different types. In: Proceedings of the 5th EAI International Conference onWireless Mobile Communication and Healthcare. 2015, 186–189Google Scholar
  47. 47.
    Zhang K H, Zhang L, Yang M-H. Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision. 2012, 864–877Google Scholar
  48. 48.
    Fernández A, Carúz J L, Usamentiaga R, Alvarez E, Casado R. Unobtrusive health monitoring system using video-based physiological information and activity measurements. In: Proceedings of International Conference on Computer, Information and Telecommunication Systems. 2012, 1–5Google Scholar
  49. 49.
    Danelljan M, Häger G, Felsberg M. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014, 1–10Google Scholar
  50. 50.
    Hoffmann K-P. Biosignale erfassen und verarbeiten. In: Kramme R, ed. Medizintechnik. Berlin: Springer, 2011, 667–688CrossRefGoogle Scholar
  51. 51.
    Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2001, 511–518Google Scholar
  52. 52.
    Cootes T F, Edwards G J, Taylor C J. Active appearance models. In: Proceedings of European Conference on Computer Vision. 1998, 484–498Google Scholar
  53. 53.
    Asthana A, Zafeiriou S, Cheng S Y, Pantic M. Robust discriminative response map fitting with constrained local models. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3444–3451Google Scholar
  54. 54.
    Saragih J M, Lucey S, Cohn J F. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 2011, 91(2): 200–215MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Martinez B, Valstar M F, Binefa X, Pantic M. Local evidence aggregation for regression-based facial point detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(5): 1149–1163CrossRefGoogle Scholar
  56. 56.
    Shi J B, Tomasi C. Good features to track. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1994, 593–600Google Scholar
  57. 57.
    Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence. 1981, 674–679Google Scholar
  58. 58.
    Bay H, Tuytelaars T, Van Gool L. SURF: speeded up robust features. In: Proceedings of European Conference on Computer Vision. 2006, 404–417Google Scholar
  59. 59.
    Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564–577CrossRefGoogle Scholar
  60. 60.
    Henriques J F, Caseiro R, Martins P, Batista J. Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of European Conference on Computer Vision. 2012, 702–715Google Scholar
  61. 61.
    Farnebäck G. Two-frame motion estimation based on polynomial expansion. In: Proceedings of Scandinavian Conference on Image Analysis. 2003, 363–370Google Scholar
  62. 62.
    Tarvainen M P, Ranta-Aho P O, Karjalainen P A. An advanced detrending method with application to hrv analysis. IEEE Transactions on Biomedical Engineering, 2002, 49(2): 172–175CrossRefGoogle Scholar
  63. 63.
    Rouast P V, Adam M T P, Cornforth D J, Lux E, Weinhardt C. Using contactless heart rate measurements for real-time assessment of affective states. In: Davis F D, Riedl R, Vom Brocke J, et al, eds. Information Systems and Neuroscience. Springer International Publishing, 2016, 157–163Google Scholar
  64. 64.
    Monkaresi H, Hussain M S, Calvo R A. Using remote heart rate measurement for affect detection. In: Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference. 2014, 118–123Google Scholar
  65. 65.
    Zhao F, Li M, Qian Y, Tsien J Z. Remote measurements of heart and respiration rates for telemedicine. PLoS ONE, 2013, 8(10): e71384CrossRefGoogle Scholar
  66. 66.
    McDuff D, Gontarek S, Picard R. Remote measurement of cognitive stress via heart rate variability. In: Proceedings of the 36th IEEE Annual International Conference of Engineering in Medicine and Biology Society. 2014, 2957–2960Google Scholar
  67. 67.
    Rahman M A, Barai A, Islam M A, Hashem M M A. Development of a device for remote monitoring of heart rate and body temperature. In: Proceedings of the 15th International Conference on Computer and Information Technology. 2012, 411–416Google Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Philipp V. Rouast
    • 1
  • Marc T. P. Adam
    • 2
  • Raymond Chiong
    • 2
  • David Cornforth
    • 2
  • Ewa Lux
    • 1
  1. 1.Institute of Information Systems and MarketingKarlsruhe Institute of TechnologyKarlsruheGermany
  2. 2.School of Electrical Engineering and ComputingThe University of NewcastleCallaghanAustralia

Personalised recommendations