Artificial Life and Robotics

, Volume 23, Issue 1, pp 15–22 | Cite as

Improvements in remote video based estimation of heart rate variability using the Welch FFT method

  • Munenori Fukunishi
  • Daniel Mcduff
  • Norimichi Tsumura
Original Article


Non-contact heart rate and heart rate variability measurements have applications in healthcare and affective computing. Recently, a system utilizing a five-band camera (RGBCO: red, green, blue, cyan, orange) was proposed, and shown to improve both remote measurement of heart rate and heart rate variability over an RGB camera. In this paper, we propose an improved method for video-based estimation of heart rate variability. We introduce three advancements over previous work utilizing five-band cameras: (1) an adaptive non-rectangular region of interest identified using automatically detected facial feature points, (2) improved peak detection within the blood volume pulse (BVP) signal, and (3) improved HRV calculation using the Welch periodogram. We apply our method to a test dataset of subjects at rest and under cognitive stress and show qualitative improvements in the stability of HRV spectrogram estimation. Although we evaluate our method using a five-band camera, the method could be applied to video recorded with any camera.


Heart rate Heart rate variability Non-contact measurement Five-band cameras 


  1. 1.
    Verkruysse W, Svaasand LO, Nelson JS (2008) Remote plethysmographic imaging using ambient light. Opt Express 16(26):21434–21445CrossRefGoogle Scholar
  2. 2.
    Poh M-Z, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774CrossRefGoogle Scholar
  3. 3.
    Poh M-Z, McDuff D, Picard RW (2011) Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans Biomed Eng 58(1):7–11CrossRefGoogle Scholar
  4. 4.
    Pagani M, Furlan R, Pizzinelli P, Crivellaro W, Cerutti S, Malliani A (1989) Spectral analysis of R-R and arterial pressure variabilities to assess sympatho-vagal interaction during mental stress in humans. J Hypertens 7(Suppl 6):S14–S15CrossRefGoogle Scholar
  5. 5.
    McDuff D, Gontarek S, Picard RW (2014) Improvements in remote cardio-pulmonary measurement using a five band digital camera. IEEE Trans Biomed Eng 61(10):2593–2601CrossRefGoogle Scholar
  6. 6.
    Monno Y, Tanaka M, Okutomi M (2012) Multispectral demosaicking using guided filter. In: IS&T/SPIE electronic imaging. International society for optics and photonics, pp 82 990O-82 990OGoogle Scholar
  7. 7.
    MonnoY Kikuchi S, Tanaka M, Okutomi M (2015) A practical one-shot multispectral imaging system using a single image sensor. IEEE Trans Image Process 24(10):3048–3059MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lomb N (1976) Least-squares frequency analysis of unequally spaced data. Astrophys Space Sci 39(2):447–462CrossRefGoogle Scholar
  9. 9.
    Scargle JD (1982) Studies in astronomical time series analysis II-statistical aspects of spectral analysis of unevenly spaced data. Astrophys J 1:835–853CrossRefGoogle Scholar
  10. 10.
    Press William H, Rybicki George B (1989) Fast algorithm for spectral analysis of unevenly sampled data. Astrophys J 338:277–280CrossRefGoogle Scholar
  11. 11.
    Schulz M, Stattegger K (1997) Spectrum: spectral analysis of unevenly spaced paleoclimatic time series. Comput Geosci 23(9):929–945CrossRefGoogle Scholar
  12. 12.
    Welch P (1967) The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans Audio Electroacoust 15(2):70–73CrossRefGoogle Scholar
  13. 13.
    Martinez B, Valstar MF, Binefa X, Pantic M (2013) Local evidence aggregation for re-gression-based facial point detection. IEEE Trans Pattern Anal Mach Intell 35(5):1149–1163CrossRefGoogle Scholar
  14. 14.
    Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1867–1874Google Scholar
  15. 15.
    De Haan G, Jeanne V (2013) Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed Engineering 60(10):2878–2886CrossRefGoogle Scholar
  16. 16.
    Wang Wenjin, den Brinker Albertus C, Stuijk Sander, de Haan Gerard (2017) Algorithmic principles of remote PPG. IEEE Trans Biomed Eng 64(7):1479–1491Google Scholar

Copyright information

© ISAROB 2017

Authors and Affiliations

  • Munenori Fukunishi
    • 1
  • Daniel Mcduff
    • 2
  • Norimichi Tsumura
    • 1
  1. 1.Graduate School of Advanced Integration ScienceChiba UniversityChibaJapan
  2. 2.Microsoft ResearchRedmondUSA

Personalised recommendations