A Methodological Outline and Utility Assessment of Sensor-based Biosignal Measurement in Human-Robot Interaction

A System for Determining Correlations Between Robot Sensor Data and Subjective Human Data in HRI

Abstract

Sensor data taken during a human-robot interaction (HRI) have high potential for usage as new, objective measures of an interaction, either replacing or supplementing survey techniques that are currently most common in HRI research. Sensor data can be taken in large quantities quickly, naturally, and discreetly. They also have the potential to reflect a user’s biosignals—information about the user’s inner state (such as stress and attention) when interacting with the robot. We previously conducted three studies attempting to use sensor data as a measurement in HRI, with methodological differences in three different experimental environments. In this paper, we reanalyze and add new data to the previous findings under a consistent methodology, consolidate what correlations we find, and can conclude that sensor data is a useful metric in HRI across a wide range of experimental setups and subject pools. We fully describe the methodology we determined to be most effective, from selection of sensors to data analysis techniques to HRI experiment setup, as a basis for how this methodology can be used in other HRI studies. We describe necessary steps in the analysis of a large amount of sensor data (over 100,000 sets) and how sensor data can be compared with survey and behavioral data. Based on these correlations, we find that the most effective sensors are temperature sensors, tactile sensors, and face distance measurements. We also find that higher measurements across all of these sensors are more correlated with both survey and behavioral measurements reflecting positive thinking towards a robot (including non-technophobia, reciprocal behaviors, and positive ratings of the robot) during an interaction. Based on these results, we argue that robot sensor usage is an important and objective metric for HRI research.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Paulhus DL (1991) Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightman LS (eds) Measures of personality and social psychological attitudes. Academic Press, San Diego, pp 17–59

    Google Scholar 

  2. 2.

    Gödert HW, Rill H-G, Vossel G (2001) Psychophysiological differentiation of deception: the effects of electrodermal lability and mode of responding on skin conductance and heart rate. Int J Psychophysiol 40(1):61–75

    Article  Google Scholar 

  3. 3.

    Crawford DG, Friesen DD, Tomlinson-Keasey C (1977) Effects of cognitively induced anxiety on hand temperature. Appl Psychophysiol Biofeedback 2(2):139–146

    Google Scholar 

  4. 4.

    Popescu F, Fazli S, Badower Y, Blankertz B, Müller K-R (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2:7

    Article  Google Scholar 

  5. 5.

    Kennedy DO, Scholey AB (1999) Glucose administration, heart rate and cognitive performance: effects of increasing mental effort. Psychopharmacology 149(1):63–71

    Article  Google Scholar 

  6. 6.

    Storm H, Myre K, Rostrup M, Stokland O, Lien MD, Ræder JC (2002) Skin conductance correlates with perioperative stress. Acta Anaesthesiol Scand 46(7):887–895

    Article  Google Scholar 

  7. 7.

    Boudewyns PA (1976) A comparison of the effects of stress vs. relaxation instruction on the finger temperature response. Behav Ther 7(1):54–67

    Article  Google Scholar 

  8. 8.

    Strauss M, Reynolds C, Hughes S, Park K, McDarby G, Picard RW (2005) The HandWave Bluetooth skin conductance sensor. In: Affective computing and intelligent interaction, vol 3784, pp 699–706

    Google Scholar 

  9. 9.

    Munekata N, Yoshida N, Sakurazawa S, Tsukahara Y, Matsubara H (2006) Design of positive biofeedback using a robot’s behaviors as motion media. In: Lecture notes in computer science, vol 4161/2006. Springer, Berlin, pp 340–349

    Google Scholar 

  10. 10.

    Shipps EM, Freeman HR (2003) Handshake: Its relation to first impressions and measured personality traits. Psi Chi J Undergrad Res 8(4):144–148

    Google Scholar 

  11. 11.

    Stiehl WD, Lieberman J, Breazeal C, Basel L, Lalla L, Wolf M (2005) Design of a therapeutic robotic companion for relational, affective touch. In: Proceedings of the 2005 IEEE international workshop on robots and human interactive communication, pp 408–415, Nashville, TN, USA, August, 2005

    Google Scholar 

  12. 12.

    Morwitz VG, Johnson E, Schmittlein D (1993) Does measuring intent change behavior? J Consum Res 20(1):46–61

    Article  Google Scholar 

  13. 13.

    Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Understanding expectations of a robot’s identity through multi-user interactions. In: Proceedings of the human-robot interaction 2011 workshop on expectations in intuitive human-robot interaction, Lausanne, Switzerland, March, 2011

    Google Scholar 

  14. 14.

    Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Using biofeedback to analyze human-robot interaction experiments. In: Proceedings of the JSME robotics and mechatronics conference 2011, Okayama, Japan, May, 2011 (Japanese language text)

    Google Scholar 

  15. 15.

    Bainbridge WA, Nozawa S, Ueda R, Okada K, Inaba M (2011) Robot sensor data as a means to measure human reactions to an interaction. In: Proceedings of the IEEE-RAS 2011 international conference on humanoid robots, Bled, Slovenia, October, 2011

    Google Scholar 

  16. 16.

    Matsui T, Inaba M (1990) Euslisp: an object-based implementation of Lisp. J Inf Process 13:3

    Google Scholar 

  17. 17.

    Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: IEEE international conference on robotics and automation: workshop on open source software, Kobe, Japan, May, 2009

    Google Scholar 

  18. 18.

    Aquest (2010). AquesTalk—text-to-speech synthesis middleware. Retrieved November 1, 2010, from: http://www.aquest.com/download/index.html

  19. 19.

    Taylor P, Black AW, Caley R (1998) The architecture of the Festival speech synthesis system. In: Proceedings of the 3rd ESCA workshop on speech synthesis, pp 141–147, Jenolan Caves, Australia

    Google Scholar 

  20. 20.

    Sears DC, Rosen LD, Weil MM (1988). Technophobia measurement instruments information. Retrieved April 15, 2011 from: http://www.technostress.com/WRexam.htm

  21. 21.

    Silverman AF, Pressman ME, Bartel HW (1973) Self-esteem and tactile communication. J Humanist Psychol 13(2):73–77

    Article  Google Scholar 

  22. 22.

    Bainbridge WA, Hart J, Kim ES, Scassellati B (2009–2010) The benefits of iterations with physically present robots over video-displayed agents. Int J Soc Robot, 1–2

  23. 23.

    Walters ML, Dautenhahn K, Koay KL, Kaouri C, Boekhorst R, Nehaniv C, Werry I, Lee D (2005) Close encounters: spatial distances between people and a robot of mechanistic appearance. In: Proceedings of the 5th IEEE-RAS international conference on humanoid robots, Tsukuba, Japan, December, 2005

    Google Scholar 

  24. 24.

    Selvin HC, Stuart A (1966) Data-Dredging procedures in survey analysis. Am Stat 20(3):20–23

    Google Scholar 

  25. 25.

    Falk B, Burstein R, Rosenblum J, Shapiro Y, Zylber-Katz E, Bashan N (1990) Effects of caffeine ingestion on body fluid balance and thermoregulation during exercise. Can J Physiol Pharm 68(7):889–892

    Article  Google Scholar 

Download references

Acknowledgements

Wilma A. Bainbridge would like to thank Yale University, The Gordon Grand Fellowship, and the Fox International Fellowship for their support for her research at the University of Tokyo.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Wilma A. Bainbridge.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bainbridge, W.A., Nozawa, S., Ueda, R. et al. A Methodological Outline and Utility Assessment of Sensor-based Biosignal Measurement in Human-Robot Interaction. Int J of Soc Robotics 4, 303–316 (2012). https://doi.org/10.1007/s12369-012-0146-y

Download citation

Keywords

  • Biosignal
  • Sensor data
  • Human-robot interaction
  • HRI measurement
  • Handshake
  • Touch-based interaction
  • Hand temperature
  • Tactile measurements
  • Face distances