Skip to main content
SpringerLink
Log in

Facial Video-Based Remote Heart Rate Measurement via Spatial-Temporal Convolutional Network

  • Conference paper
  • First Online:
Intelligent Life System Modelling, Image Processing and Analysis (LSMS 2021, ICSEE 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1467))

  • 694 Accesses

Abstract

In this paper, in order to remotely measure the human heart rate (HR) with consumer-level cameras, we propose an end-to-end framework for reducing the influence of non-physiological signals (such as head movement and illumination variation). First, face tracking and skin segmentation are carried out on the input facial video, and then the designed model based on small-scale spatial-temporal convolutional network is used to classify the sub-regions of interest. Finally, the predicted results of each sub-block are aggregated to achieve the regression of the HR. Experiment on the LGI-PPGI dataset in four scenarios with increasing difficulty shows that the proposed approach achieves better performance compared with the benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16, 21434−21445 (2008)

    Google Scholar 

  2. Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10762−10774 (2010)

    Google Scholar 

  3. De, H.G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed. Eng. 60, 2878–2886 (2013)

    Article  Google Scholar 

  4. Wang, W., Brinker, A.C., Stuijk, S., Haan, G.: Algorithmic principles of remote PPG. IEEE Trans. Biomed. Eng. 64, 1479–1491 (2017)

    Article  Google Scholar 

  5. Li, X., Chen, J., Zhao, G., Pietikainen, M.: Remote heart rate measurement from face videos under realistic situations. In: 27th IEEE Conference on Computer Vision and Pattern Recognition, pp. 4264−271. IEEE Press (2014)

    Google Scholar 

  6. Wang, W., Brinker, A.C., Stuijk, S., Haan, G.: Robust heart rate from fitness videos. Physiol. Meas. 38, 1023−1044 (2017)

    Google Scholar 

  7. Pilz, C.S., Zaunseder, S., Krajewski, J., Blazek, V.: Local group invariance for heart rate estimation from face videos in the wild. In: 31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1335−1343. IEEE Press (2018)

    Google Scholar 

  8. Chen, W., McDuff, D.: DeepPhys: video-based physiological measurement using convolutional attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 356–373. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01216-8_22

    Chapter  Google Scholar 

  9. Niu, X., Shan, S., Han, H., Chen, X.: RhythmNet: end-to-end heart rate estimation from face via spatial-temporal representation. IEEE Trans. Image Process. 29, 2409–2423 (2020)

    Article  Google Scholar 

  10. Yu, Z., Peng, W., Li, X., Hong, X., Zhao, G.: Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement. In: 17th IEEE/CVF International Conference on Computer Vision, pp. 151−160. IEEE Press (2019)

    Google Scholar 

  11. Spetlik, R., Franc, V., Cech, J., Matas, J.: Visual heart rate estimation with convolutional neural network. In: 29th British Machine Vision Conference (2018)

    Google Scholar 

  12. Cai, Z., Liu, Q., Wang, S., Yang, B.: Joint head pose estimation with multi-task cascaded convolutional networks for face alignment. In: 24th International Conference on Pattern Recognition, pp. 495−500. IEEE Press (2018)

    Google Scholar 

  13. Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35, 221−231 (2013)

    Google Scholar 

  14. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (2015)

    Google Scholar 

  15. Rajendra Acharya, U., Paul Joseph, K., Kannathal, N., Lim, C.M., Suri, J.S.: Heart rate variability: a review. Med. Bio. Eng. Comput. 44, 1031−1051 (2006)

    Google Scholar 

  16. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: 14th International Conference on Artificial Intelligence and Statistics, pp. 315--323. IEEE Press (2011)

    Google Scholar 

  17. Wang, W., Stuijk, S., De Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63, 1974–1984 (2016)

    Article  Google Scholar 

  18. Boccignone, G., Conte, D., Cuculo, V., D’Amelio, A., Grossi, G., Lanzarotti, R.: An open framework for remote-PPG methods and their assessment. IEEE Access. 8, 216083–216103 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200072, China

    Chunlin Xu & Xin Li

Authors
  1. Chunlin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Minrui Fei

  2. University of Tokyo, Tokyo, Japan

    Luonan Chen

  3. Shanghai University, Shanghai, China

    Shiwei Ma

  4. Shanghai University, Shanghai, China

    Xin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, C., Li, X. (2021). Facial Video-Based Remote Heart Rate Measurement via Spatial-Temporal Convolutional Network. In: Fei, M., Chen, L., Ma, S., Li, X. (eds) Intelligent Life System Modelling, Image Processing and Analysis. LSMS ICSEE 2021 2021. Communications in Computer and Information Science, vol 1467. Springer, Singapore. https://doi.org/10.1007/978-981-16-7207-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-7207-1_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-7206-4

  • Online ISBN: 978-981-16-7207-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation