Skip to main content
SpringerLink
Log in

Remote heart rate measurement using low-cost RGB face video: a technical literature review

  • Review Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Remote photoplethysmography (rPPG) allows remote measurement of the heart rate using low-cost RGB imaging equipment. In this study, we review the development of the field of rPPG since its emergence in 2008. We also classify existing rPPG approaches and derive a framework that provides an overview of modular steps. Based on this framework, practitioners can use our classification to design algorithms for an rPPG approach that suits their specific needs. Researchers can use the reviewed and classified algorithms as a starting point to improve particular features of an rPPG algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang Z L, Pi Z Y, Liu B Y. Troika: a general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 522–531

    Article  Google Scholar 

  2. Adam M T P, Krämer J, Weinhardt C. Excitement up! Price down! Measuring emotions in dutch auctions. International Journal of Electronic Commerce, 2012, 13(2): 7–39

    Article  Google Scholar 

  3. Adam M T P, Krämer J, Müller MB. Auction fever! How time pressure and social competition affect bidders’ arousal and bids in retail auctions. Journal of Retailing, 2015, 91(3): 468–485

    Article  Google Scholar 

  4. Astor P J, Adam M T P, Jerčić P, Schaaff K, Weinhardt C. Integrating biosignals into information systems: a neurois tool for improving emotion regulation. Journal of Management Information Systems, 2013, 30(3): 247–278

    Article  Google Scholar 

  5. Riedl R. On the biology of technostress: literature review and research agenda. ACM SIGMIS Database, 2013, 44(1): 18–55

    Article  Google Scholar 

  6. Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Optics Express, 2008, 16(26): 21434–21445

    Article  Google Scholar 

  7. McDuff D J, Estepp J R, Piasecki A M, Blackford E B. A survey of remote optical photoplethysmographic imaging methods. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015, 6398–6404

    Google Scholar 

  8. Liu H, Wang Y D, Wang L. A review of non-contact, low-cost physiological information measurement based on photoplethysmographic imaging. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2088–2091

    Google Scholar 

  9. Kranjec J, Beguš S, Geršak G, Drnovšek J. Non-contact heart rate and heart rate variability measurements: a review. Biomedical Signal Processing and Control, 2014, 13(1): 102–112

    Article  Google Scholar 

  10. Poh M-Z, McDuff D J, Picard R W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express, 2010, 18(10): 10762–10774

    Article  Google Scholar 

  11. Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement, 2007, 28(3): R1–R39

    Article  Google Scholar 

  12. Lindberg L-G, Öberg P A. Optical properties of blood in motion. Optical Engineering, 1993, 32(2): 253–257

    Article  Google Scholar 

  13. Balakrishnan G, Durand F, Guttag J. Detecting pulse from head motions in video. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3430–3437

    Google Scholar 

  14. Starr I, Rawson A J, Schroeder H A, Joseph N R. Studies on the estimation of cardiac ouptut in man, and of abnormalities in cardiac function, from the heart’s recoil and the blood’s impacts; the ballistocardiogram. American Journal of Physiology, 1939, 127(1): 1–28

    Google Scholar 

  15. Hertzman A B, Spealman C R. Observations on the finger volume pulse recorded photoelectrically. American Journal of Physiology, 1937, 119(2): 334–335

    Google Scholar 

  16. Poh M-Z, McDuff D J, Picard RW. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 2011, 58(1): 7–11

    Article  Google Scholar 

  17. Lewandowska M, Ruminski J, Kocejko T. Measuring pulse rate with a webcam: a non-contact method for evaluating cardiac activity. In: Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS). 2011, 405–410

    Google Scholar 

  18. Kwon S, Kim H, Park K S. Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2174–2177

    Google Scholar 

  19. Lee K-Z, Hung P-C, Tsai L-W. Contact-free heart rate measurement using a camera. In: Proceedings of the 9th Conference on Computer and Robot Vision. 2012, 147–152

    Google Scholar 

  20. Wei L, Tian Y H, Wang Y W, Ebrahimi T, Huang T. Automatic webcam-based human heart rate measurements using laplacian eigenmap. In: Proceedings of Asian Conference on Computer Vision. 2013, 281–292

    Google Scholar 

  21. Wu H-Y, Rubinstein M, Shih E, Guttag J V, Durand F, Freeman W T. Eulerian video magnification for revealing subtle changes in the world. ACM Transactions on Graphics, 2012, 31(4): 1–8

    Article  Google Scholar 

  22. Wu H-Y. Eulerian video processing and medical applications. Dissertation for the Master Degree. Cambridge, MA: Massachusetts Institute of Technology, 2012

    Google Scholar 

  23. Shan L, Yu M H. Video-based heart rate measurement using head motion tracking and ica. In: Proceedings of the 6th International Congress on Image and Signal Processing. 2013, 160–164

    Google Scholar 

  24. Irani R, Nasrollahi K, Moeslund T B. Improved pulse detection from head motions using dct. In: Proceedings of the 9th International Conference on Computer Vision Theory and Applications. 2014, 118–124

    Google Scholar 

  25. De Haan G, Jeanne V. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 2013, 60(10): 2878–2886

    Article  Google Scholar 

  26. De Haan G, Van Leest A. Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiological Measurement, 2014, 35(9): 1913–1926

    Article  Google Scholar 

  27. Lempe G, Zaunseder S, Wirthgen T, Zipser S, Malberg H. ROI selection for remote photoplethysmography. In: Meinzer H-P, Deserno MT, Handels H, et al, eds. Bildverarbeitung für die Medizin 2013. Berlin: Springer, 2013, 99–103

    Chapter  Google Scholar 

  28. Datcu D, Cidota M, Lukosch S, Rothkrantz L. Noncontact automatic heart rate analysis in visible spectrum by specific face regions. In: Proceedings of the 14th International Conference on Computer Systems and Technologies. 2013, 120–127

    Google Scholar 

  29. Li X B, Chen J, Zhao G Y, Pietikäinen M. Remote heart rate measurement from face videos under realistic situations. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014, 4264–4271

    Google Scholar 

  30. McDuff D, Gontarek S, Picard RW. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(12): 2948–2954

    Article  Google Scholar 

  31. Kumar M, Veeraraghavan A, Sabharwal A. DistancePPG: robust noncontact vital signs monitoring using a camera. Biomedical Optics Express, 2015, 6(5): 1565–1588

    Article  Google Scholar 

  32. Tasli H E, Gudi A, Den Uyl M. Remote PPG based vital sign measurement using adaptive facial regions. In: Proceedings of IEEE International Conference on Image Processing. 2014, 1410–1414

    Google Scholar 

  33. Feng L T, Po L-M, Xu X Y, Li Y M. Motion artifacts suppression for remote imaging photoplethysmography. In: Proceedings of the 19th International Conference on Digital Signal Processing. 2014, 18–23

    Google Scholar 

  34. Feng L T, Po L-M, Xu X Y, Li Y M, Ma R Y. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(5): 879–891

    Article  Google Scholar 

  35. Wang W J, Stuijk S, De Haan G. Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 415–425

    Article  Google Scholar 

  36. McDuff D, Gontarek S, Picard R W. Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(10): 2593–2601

    Article  Google Scholar 

  37. Chwyl B, Chung A G, Deglint J, Wong A, Clausi D A. Remote heart rate measurement through broadband video via stochastic bayesian estimation. Vision Letters, 2015, 1(1): 5

    Article  Google Scholar 

  38. Monkaresi H, Calvo R A, Yan H. A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE Journal of Biomedical and Health Informatics, 2014, 18(4): 1153–1160

    Article  Google Scholar 

  39. Hsu Y, Lin Y L, Hsu W. Learning-based heart rate detection from remote photoplethysmography features. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2014, 4433–4437

    Google Scholar 

  40. Tran D N, Lee H, Kim C. A robust real time system for remote heart rate measurement via a camera. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2015, 1–6

    Google Scholar 

  41. Li M-C, Lin Y-H. A real-time non-contact pulse rate detector based on smartphone. In: Proceedings of International Symposium on Next-Generation Electronics. 2015, 1–3

    Google Scholar 

  42. Yu Y-P, Kwan B-H, Lim C-L, Wong S-L, Raveendran P. Video-based heart rate measurement using short-time fourier transform. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems. 2013, 704–707

    Google Scholar 

  43. Holton B D, Mannapperuma K, Lesniewski P J, Thomas J C. Signal recovery in imaging photoplethysmography. Physiological Measurement, 2013, 34(11): 1499–1511

    Article  Google Scholar 

  44. Xu S C, Sun L Y, Rohde G K. Robust efficient estimation of heart rate pulse from video. Biomedical Optics Express, 2014, 5(4): 1124–1135

    Article  Google Scholar 

  45. Feng L T, Po L-M, Xu X Y, Li Y M, Cheung C-H, Cheung K-W, Yuan F. Dynamic ROI based on K-means for remote photoplethysmography. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2015, 1310–1314

    Google Scholar 

  46. Han B, Ivanov K, Wang L, Yan Y. Exploration of the optimal skincamera distance for facial photoplethysmographic imaging measurement using cameras of different types. In: Proceedings of the 5th EAI International Conference onWireless Mobile Communication and Healthcare. 2015, 186–189

    Google Scholar 

  47. Zhang K H, Zhang L, Yang M-H. Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision. 2012, 864–877

    Google Scholar 

  48. Fernández A, Carúz J L, Usamentiaga R, Alvarez E, Casado R. Unobtrusive health monitoring system using video-based physiological information and activity measurements. In: Proceedings of International Conference on Computer, Information and Telecommunication Systems. 2012, 1–5

    Google Scholar 

  49. Danelljan M, Häger G, Felsberg M. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014, 1–10

    Google Scholar 

  50. Hoffmann K-P. Biosignale erfassen und verarbeiten. In: Kramme R, ed. Medizintechnik. Berlin: Springer, 2011, 667–688

    Chapter  Google Scholar 

  51. Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2001, 511–518

    Google Scholar 

  52. Cootes T F, Edwards G J, Taylor C J. Active appearance models. In: Proceedings of European Conference on Computer Vision. 1998, 484–498

    Google Scholar 

  53. Asthana A, Zafeiriou S, Cheng S Y, Pantic M. Robust discriminative response map fitting with constrained local models. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3444–3451

    Google Scholar 

  54. Saragih J M, Lucey S, Cohn J F. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 2011, 91(2): 200–215

    Article  MathSciNet  MATH  Google Scholar 

  55. Martinez B, Valstar M F, Binefa X, Pantic M. Local evidence aggregation for regression-based facial point detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(5): 1149–1163

    Article  Google Scholar 

  56. Shi J B, Tomasi C. Good features to track. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1994, 593–600

    Google Scholar 

  57. Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence. 1981, 674–679

    Google Scholar 

  58. Bay H, Tuytelaars T, Van Gool L. SURF: speeded up robust features. In: Proceedings of European Conference on Computer Vision. 2006, 404–417

    Google Scholar 

  59. Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564–577

    Article  Google Scholar 

  60. Henriques J F, Caseiro R, Martins P, Batista J. Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of European Conference on Computer Vision. 2012, 702–715

    Google Scholar 

  61. Farnebäck G. Two-frame motion estimation based on polynomial expansion. In: Proceedings of Scandinavian Conference on Image Analysis. 2003, 363–370

    Google Scholar 

  62. Tarvainen M P, Ranta-Aho P O, Karjalainen P A. An advanced detrending method with application to hrv analysis. IEEE Transactions on Biomedical Engineering, 2002, 49(2): 172–175

    Article  Google Scholar 

  63. Rouast P V, Adam M T P, Cornforth D J, Lux E, Weinhardt C. Using contactless heart rate measurements for real-time assessment of affective states. In: Davis F D, Riedl R, Vom Brocke J, et al, eds. Information Systems and Neuroscience. Springer International Publishing, 2016, 157–163

    Google Scholar 

  64. Monkaresi H, Hussain M S, Calvo R A. Using remote heart rate measurement for affect detection. In: Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference. 2014, 118–123

    Google Scholar 

  65. Zhao F, Li M, Qian Y, Tsien J Z. Remote measurements of heart and respiration rates for telemedicine. PLoS ONE, 2013, 8(10): e71384

    Article  Google Scholar 

  66. McDuff D, Gontarek S, Picard R. Remote measurement of cognitive stress via heart rate variability. In: Proceedings of the 36th IEEE Annual International Conference of Engineering in Medicine and Biology Society. 2014, 2957–2960

    Google Scholar 

  67. Rahman M A, Barai A, Islam M A, Hashem M M A. Development of a device for remote monitoring of heart rate and body temperature. In: Proceedings of the 15th International Conference on Computer and Information Technology. 2012, 411–416

    Google Scholar 

Download references

Acknowledgements

This work was supported by a fellowship within the FITweltweit programme of the German Academic Exchange Service (DAAD).

Author information

Authors and Affiliations

  1. Institute of Information Systems and Marketing, Karlsruhe Institute of Technology, Karlsruhe, 76133, Germany

    Philipp V. Rouast & Ewa Lux

  2. School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW, 2308, Australia

    Marc T. P. Adam, Raymond Chiong & David Cornforth

Authors
  1. Philipp V. Rouast
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc T. P. Adam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raymond Chiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Cornforth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ewa Lux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raymond Chiong.

Additional information

Philipp V. Rouast holds a BS degree in industrial engineering from Karlsruhe Institute of Technology, Germany. As part of his MS, he is working on a real-time application of Remote Photoplethysmography at the University of Newcastle, Australia. His research interests include machine learning, computer vision and data analytics.

Marc T. P. Adam is a senior lecturer in information technology at the University of Newcastle, Australia. He received a Diploma in computer science from the University of Applied Sciences Würzburg, Germany, and a PhD in economics of information systems from Karlsruhe Institute of Technology, Germany. In his research, he investigates the interplay of cognitive and affective processes of human users in electronic commerce.

Raymond Chiong is a senior lecturer at the University of Newcastle, Australia. He is also a guest research professor with the Center for Modern Information Management at Huazhong University of Science and Technology, China, and a visiting scholar with the Department of Automation, Tsinghua University, China. He obtained his PhD degree from the University of Melbourne, Australia, and an MS degree from the University of Birmingham, UK. His research interests include optimization, machine learning and data analytics. He is the Editor-in-Chief of the Journal of Systems and Information Technology, an Editor of Engineering Applications of Artificial Intelligence, and an Associate Editor of the IEEE Computational Intelligence Magazine. To date, he has produced over 130 refereed publications.

David Cornforth received the BS degree in electrical and electronic engineering from Nottingham Trent University, UK in 1982, and the PhD degree in computer science from the University of Nottingham, UK in 1994. He has been an educator and researcher at Charles Sturt University, the University of New South Wales, and now at the University of Newcastle, Australia. He has also been a research scientist at the Commonwealth Scientific and Industrial Research Organisation (CSIRO), Newcastle, Australia. His research interests are in health information systems, pattern recognition, artificial intelligence, multi-agent simulation, and optimisation.

Ewa Lux is a PhD candidate at the Institute of Information Systems and Marketing, Karlsruhe Institute of Technology, Germany. She received a Bachelor degree in business information systems from Baden-Wuerttemberg Cooperative State University, Germany in cooperation with the German Central Bank, and a Master degree in information engineering and management from Karlsruhe Institute of Technology, Germany. Her research interest focuses on emotional processes of economic decision making in electronic markets.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rouast, P.V., Adam, M.T.P., Chiong, R. et al. Remote heart rate measurement using low-cost RGB face video: a technical literature review. Front. Comput. Sci. 12, 858–872 (2018). https://doi.org/10.1007/s11704-016-6243-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-6243-6

Keywords

Search

Navigation