Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A Methodological Outline and Utility Assessment of Sensor-based Biosignal Measurement in Human-Robot Interaction

A System for Determining Correlations Between Robot Sensor Data and Subjective Human Data in HRI

  • 274 Accesses

  • 2 Citations

Abstract

Sensor data taken during a human-robot interaction (HRI) have high potential for usage as new, objective measures of an interaction, either replacing or supplementing survey techniques that are currently most common in HRI research. Sensor data can be taken in large quantities quickly, naturally, and discreetly. They also have the potential to reflect a user’s biosignals—information about the user’s inner state (such as stress and attention) when interacting with the robot. We previously conducted three studies attempting to use sensor data as a measurement in HRI, with methodological differences in three different experimental environments. In this paper, we reanalyze and add new data to the previous findings under a consistent methodology, consolidate what correlations we find, and can conclude that sensor data is a useful metric in HRI across a wide range of experimental setups and subject pools. We fully describe the methodology we determined to be most effective, from selection of sensors to data analysis techniques to HRI experiment setup, as a basis for how this methodology can be used in other HRI studies. We describe necessary steps in the analysis of a large amount of sensor data (over 100,000 sets) and how sensor data can be compared with survey and behavioral data. Based on these correlations, we find that the most effective sensors are temperature sensors, tactile sensors, and face distance measurements. We also find that higher measurements across all of these sensors are more correlated with both survey and behavioral measurements reflecting positive thinking towards a robot (including non-technophobia, reciprocal behaviors, and positive ratings of the robot) during an interaction. Based on these results, we argue that robot sensor usage is an important and objective metric for HRI research.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Paulhus DL (1991) Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightman LS (eds) Measures of personality and social psychological attitudes. Academic Press, San Diego, pp 17–59

  2. 2.

    Gödert HW, Rill H-G, Vossel G (2001) Psychophysiological differentiation of deception: the effects of electrodermal lability and mode of responding on skin conductance and heart rate. Int J Psychophysiol 40(1):61–75

  3. 3.

    Crawford DG, Friesen DD, Tomlinson-Keasey C (1977) Effects of cognitively induced anxiety on hand temperature. Appl Psychophysiol Biofeedback 2(2):139–146

  4. 4.

    Popescu F, Fazli S, Badower Y, Blankertz B, Müller K-R (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2:7

  5. 5.

    Kennedy DO, Scholey AB (1999) Glucose administration, heart rate and cognitive performance: effects of increasing mental effort. Psychopharmacology 149(1):63–71

  6. 6.

    Storm H, Myre K, Rostrup M, Stokland O, Lien MD, Ræder JC (2002) Skin conductance correlates with perioperative stress. Acta Anaesthesiol Scand 46(7):887–895

  7. 7.

    Boudewyns PA (1976) A comparison of the effects of stress vs. relaxation instruction on the finger temperature response. Behav Ther 7(1):54–67

  8. 8.

    Strauss M, Reynolds C, Hughes S, Park K, McDarby G, Picard RW (2005) The HandWave Bluetooth skin conductance sensor. In: Affective computing and intelligent interaction, vol 3784, pp 699–706

  9. 9.

    Munekata N, Yoshida N, Sakurazawa S, Tsukahara Y, Matsubara H (2006) Design of positive biofeedback using a robot’s behaviors as motion media. In: Lecture notes in computer science, vol 4161/2006. Springer, Berlin, pp 340–349

  10. 10.

    Shipps EM, Freeman HR (2003) Handshake: Its relation to first impressions and measured personality traits. Psi Chi J Undergrad Res 8(4):144–148

  11. 11.

    Stiehl WD, Lieberman J, Breazeal C, Basel L, Lalla L, Wolf M (2005) Design of a therapeutic robotic companion for relational, affective touch. In: Proceedings of the 2005 IEEE international workshop on robots and human interactive communication, pp 408–415, Nashville, TN, USA, August, 2005

  12. 12.

    Morwitz VG, Johnson E, Schmittlein D (1993) Does measuring intent change behavior? J Consum Res 20(1):46–61

  13. 13.

    Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Understanding expectations of a robot’s identity through multi-user interactions. In: Proceedings of the human-robot interaction 2011 workshop on expectations in intuitive human-robot interaction, Lausanne, Switzerland, March, 2011

  14. 14.

    Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Using biofeedback to analyze human-robot interaction experiments. In: Proceedings of the JSME robotics and mechatronics conference 2011, Okayama, Japan, May, 2011 (Japanese language text)

  15. 15.

    Bainbridge WA, Nozawa S, Ueda R, Okada K, Inaba M (2011) Robot sensor data as a means to measure human reactions to an interaction. In: Proceedings of the IEEE-RAS 2011 international conference on humanoid robots, Bled, Slovenia, October, 2011

  16. 16.

    Matsui T, Inaba M (1990) Euslisp: an object-based implementation of Lisp. J Inf Process 13:3

  17. 17.

    Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: IEEE international conference on robotics and automation: workshop on open source software, Kobe, Japan, May, 2009

  18. 18.

    Aquest (2010). AquesTalk—text-to-speech synthesis middleware. Retrieved November 1, 2010, from: http://www.aquest.com/download/index.html

  19. 19.

    Taylor P, Black AW, Caley R (1998) The architecture of the Festival speech synthesis system. In: Proceedings of the 3rd ESCA workshop on speech synthesis, pp 141–147, Jenolan Caves, Australia

  20. 20.

    Sears DC, Rosen LD, Weil MM (1988). Technophobia measurement instruments information. Retrieved April 15, 2011 from: http://www.technostress.com/WRexam.htm

  21. 21.

    Silverman AF, Pressman ME, Bartel HW (1973) Self-esteem and tactile communication. J Humanist Psychol 13(2):73–77

  22. 22.

    Bainbridge WA, Hart J, Kim ES, Scassellati B (2009–2010) The benefits of iterations with physically present robots over video-displayed agents. Int J Soc Robot, 1–2

  23. 23.

    Walters ML, Dautenhahn K, Koay KL, Kaouri C, Boekhorst R, Nehaniv C, Werry I, Lee D (2005) Close encounters: spatial distances between people and a robot of mechanistic appearance. In: Proceedings of the 5th IEEE-RAS international conference on humanoid robots, Tsukuba, Japan, December, 2005

  24. 24.

    Selvin HC, Stuart A (1966) Data-Dredging procedures in survey analysis. Am Stat 20(3):20–23

  25. 25.

    Falk B, Burstein R, Rosenblum J, Shapiro Y, Zylber-Katz E, Bashan N (1990) Effects of caffeine ingestion on body fluid balance and thermoregulation during exercise. Can J Physiol Pharm 68(7):889–892

Download references

Acknowledgements

Wilma A. Bainbridge would like to thank Yale University, The Gordon Grand Fellowship, and the Fox International Fellowship for their support for her research at the University of Tokyo.

Author information

Correspondence to Wilma A. Bainbridge.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bainbridge, W.A., Nozawa, S., Ueda, R. et al. A Methodological Outline and Utility Assessment of Sensor-based Biosignal Measurement in Human-Robot Interaction. Int J of Soc Robotics 4, 303–316 (2012). https://doi.org/10.1007/s12369-012-0146-y

Download citation

Keywords

  • Biosignal
  • Sensor data
  • Human-robot interaction
  • HRI measurement
  • Handshake
  • Touch-based interaction
  • Hand temperature
  • Tactile measurements
  • Face distances