Using Contactless Heart Rate Measurements for Real-Time Assessment of Affective States

  • Philipp V. Rouast
  • Marc T. P. Adam
  • David J. Cornforth
  • Ewa Lux
  • Christof Weinhardt
Conference paper
Part of the Lecture Notes in Information Systems and Organisation book series (LNISO, volume 16)

Abstract

Heart rate measurements contain valuable information about a person’s affective state. There is a wide range of application domains for heart rate-based measures in information systems. To date, heart rate is typically measured using skin contact methods, where users must wear a measuring device. A non-contact and easy to use mobile approach, allowing heart rate measurements without interfering with the users’ natural environment, could prove to be a valuable NeuroIS tool. Hence, our two research objectives are (1) to develop an application for mobile devices that allows for non-contact, real-time heart rate measurement and (2) to evaluate this application in an IS context by benchmarking the results of our approach against established measurements. The proposed algorithm is based on non-contact photoplethysmography and hence takes advantage of slight skin color variations that occurs periodically with the user’s pulse.

Keywords

Heart rate Photoplethysmography Mobile NeuroIS Information systems 

References

  1. 1.
    Riedl, R., Davis, F.D., Hevner, A.R.: Towards a NeuroIS research methodology: Intensifying the discussion on methods, tools, and measurement. J. Assoc. Inf. Syst. 15, i–xxxv (2014)Google Scholar
  2. 2.
    Dimoka, A., Banker, R.D., Benbasat, I., Davis, F.D., Dennis, A.R., Gefen, D., Gupta, A., Ischebeck, A., Kenning, P., Pavlou, P.A., Müller-Putz, G., Riedl, R., vom Brocke, J., Weber, B.: On the use of neurophysiological tools in IS research: Developing a research agenda for NeuroIS. MIS Q. 36, 679–702 (2012)Google Scholar
  3. 3.
    Teubner, T., Adam, M.T.P., Riordan, R.: The impact of computerized agents on immediate emotions, overall arousal and bidding behavior in electronic auctions. J. Assoc. Inf. Syst. 16, 838–879 (2015)Google Scholar
  4. 4.
    Hariharan, A., Adam, M.T.P.: Blended emotion detection for decision support. IEEE Trans. Hum.-Mach. Syst. 45, 510–517 (2015)CrossRefGoogle Scholar
  5. 5.
    Léger, P.-M., Davis, F.D., Cronan, T.P., Perret, J.: Neurophysiological correlates of cognitive absorption in an enactive training context. Comput. Hum. Behav. 34, 273–283 (2014)CrossRefGoogle Scholar
  6. 6.
    Adam, M.T.P., Gimpel, H., Maedche, A., Riedl, R.: Design blueprint for stress-sensitive adaptive enterprise systems. Bus. Inf. Syst. Eng. (in press)Google Scholar
  7. 7.
    Shen, L., Wang, M., Shen, R.: Affective e-learning: Using “emotional” data to improve learning in pervasive learning environment. Educ. Technol. Soc. 12, 176–189 (2009)Google Scholar
  8. 8.
    Adam, M.T.P., Krämer, J., Müller, M.B.: Auction fever! How time pressure and social competition affect bidders’ arousal and bids in retail auctions. J. Retail. 91, 468–485 (2015)CrossRefGoogle Scholar
  9. 9.
    Schaaff, K., Degen, R., Adler, N., Adam, M.T.P.: Measuring affect using a standard mouse device. Biomed. Eng. (NY) 57, 761–764 (2012)Google Scholar
  10. 10.
    Vom Brocke, J., Riedl, R., Léger, P.-M.: Application strategies for neuroscience in information systems design science research. J. Comput. Inf. Syst. 53, 1–13 (2013)Google Scholar
  11. 11.
    Poh, M.-Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express. 18, 10762–10774 (2010)CrossRefGoogle Scholar
  12. 12.
    Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express. 16, 21434–21445 (2008)CrossRefGoogle Scholar
  13. 13.
    Hoffmann, K.-P.: Biosignale Erfassen und Verarbeiten. In: Kramme, R. (ed.) Medizintechnik, pp. 667–688. Springer, Berlin (2011)Google Scholar
  14. 14.
    Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: Proceedings of the 2013 I.E. Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3430–3437 (2013)Google Scholar
  15. 15.
    Irani, R., Nasrollahi, K., Moeslund, T.B.: Improved pulse detection from head motions using DCT. In: Proceedings of the 9th International Conference on Computer Vision Theory and Applications, pp. 118–124 (2014)Google Scholar
  16. 16.
    Shan, L., Yu, M.: Video-based heart rate measurement using head motion tracking and ICA. In: Proceedings of the 2013 6th International Congress on Image and Signal Processing, pp. 160–164 (2013)Google Scholar
  17. 17.
    Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J.V., Durand, F., Freeman, W.T.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 31, 1–8 (2012)CrossRefGoogle Scholar
  18. 18.
    Lewandowska, M., Ruminski, J., Kocejko, T.: Measuring pulse rate with a webcam: A non-contact method for evaluating cardiac activity. In: Proceedings of the 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 405–410 (2011)Google Scholar
  19. 19.
    Kwon, S., Kim, H., Park, K.S.: Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: Proceedings of the 2012 I.E. Annual International Conference of the Engineering in Medicine and Biology Society, pp. 2174–2177 (2012)Google Scholar
  20. 20.
    Li, X., Chen, J., Zhao, G., Pietikäinen, M.: Remote heart rate measurement from face videos under realistic situations. In: Proceedings of the 2014 I.E. Computer Society Conference on Computer Vision and Pattern Recognition, pp. 4264–4271 (2014)Google Scholar
  21. 21.
    Tarvainen, M.P., Ranta-Aho, P.O., Karjalainen, P.A.: An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng. 49, 172–175 (2002)CrossRefGoogle Scholar
  22. 22.
    Wu, H.-Y.: Eulerian Video Processing and Medical Applications. Master’s Thesis, Massachusetts Institute of Technology (2012)Google Scholar
  23. 23.
    Müller, M.B., Adam, M.T.P., Cornforth, D.J., Chiong, R., Krämer, J., Weinhardt, C.: Selecting physiological features for predicting bidding behavior in electronic auctions. In: Proceedings of the Forty-Ninth Annual Hawaii International Conference on System Sciences (HICSS), pp. 396–405 (2016)Google Scholar
  24. 24.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 I.E. Computer Society Conference on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  25. 25.
    Riedl, R.: On the biology of technostress: Literature review and research agenda. ACM SIGMIS Database 44, 18–55 (2013)CrossRefGoogle Scholar
  26. 26.
    Fairclough, S.: Physiological data must remain confidential. Nature 505, 263 (2014)CrossRefGoogle Scholar
  27. 27.
    Astor, P.J., Adam, M.T.P., Jerčić, P., Schaaff, K., Weinhardt, C.: Integrating biosignals into information systems: A NeuroIS tool for improving emotion regulation. J. Manag. Inf. Syst. 30, 247–278 (2013)CrossRefGoogle Scholar
  28. 28.
    Lux, E., Hawlitschek, F., Adam, M.T.P., Pfeiffer, J.: Using live biofeedback for decision support: Investigating influences of emotion regulation in financial decision making. In: ECIS 2015 Research-in-Progress Papers, pp. 1–12 (2015)Google Scholar
  29. 29.
    Astor, P.J., Adam, M.T.P., Jähnig, C., Seifert, S.: Measuring regret: Emotional aspects of auction design. In: ECIS 2011 Proceedings, pp. 1129–1140 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Philipp V. Rouast
    • 1
  • Marc T. P. Adam
    • 2
  • David J. Cornforth
    • 2
  • Ewa Lux
    • 1
  • Christof Weinhardt
    • 1
  1. 1.Karlsruhe Institute of TechnologyKarlsruheGermany
  2. 2.University of NewcastleNewcastleAustralia

Personalised recommendations