Applied Intelligence

, Volume 33, Issue 1, pp 79–92 | Cite as

Using physiological signals to detect natural interactive behavior

  • Yasser MohammadEmail author
  • Toyoaki Nishida


Many researchers in the Human Robot Interaction (HRI) and Embodied Conversational Agents (ECA) domains try to build robots and agents that exhibit human-like behavior in real-world close encounter situations. One major requirement for comparing such robots and agents is to have an objective quantitative metric for measuring naturalness in various kinds of interactions. Some researchers have already suggested techniques for measuring stress level, awareness etc using physiological signals like Galvanic Skin Response (GSR) and Blood Volume Pulse (BVP). One problem of available techniques is that they are only tested with extreme situations and cannot according to the analysis provided in this paper distinguish the response of human subjects in natural interaction situations. One other problem of the available techniques is that most of them require calibration and some times ad-hoc adjustment for every subject. This paper explores the usefulness of various kinds of physiological signals and statistics in distinguishing natural and unnatural partner behavior in a close encounter situation. The paper also explores the usefulness of these statistics in various time slots of the interaction. Based on this analysis a regressor was designed to measure naturalness in close encounter situations and was evaluated using human-human and human-robot interactions and shown to achieve statistically significant distinction between natural and unnatural situations.


RSST Psychophysiology HRI Data mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mohammad Y, Xu Y, Matsumura K, Nishida T (2008) The h 3 r explanation corpus:human-human and base human-robot interaction dataset. In: The fourth international conference on intelligent sensors, sensor networks and information processing (ISSNIP2008) Google Scholar
  2. 2.
    Qazi Z, Wang Z, Haq I (2006) Human likeness of humanoid robots exploring the uncanny valley. In: International conference on emerging technologies, ICET’06, pp 650–656 Google Scholar
  3. 3.
    Shi Y, Choi EHC, Ruiz N, Chen F, Taib R (2007) Galvanic skin respons (gsr) as an index of cognitive load. In: CHI 2007, pp 2651–2656 Google Scholar
  4. 4.
    Mandryk RL, Inkpen KM (2004) Physiological indicators for the evaluation of co-located collaborative play. In: CSCW’04 Google Scholar
  5. 5.
    Lin T, Hu W, Omata M, Imamiya A (2005) Do physiological data relate to traditional usability indexes? In: OZCHI 2005 Google Scholar
  6. 6.
    Lang PJ (1995) The emotion probe: studies of motivation and attention. Am Psychologiest 50(5):372–285 CrossRefGoogle Scholar
  7. 7.
    Mower E, Feil-Seifer DJ, Mataric MJ, Narayanan S (2007) Investigating implicit cues for user state estimation in human-robot interaction using physiological measurements. In: 16th international conference on robot & human interactive communication, pp 1125–1130 Google Scholar
  8. 8.
    Bradley MPL (2000) Measuring emotions: behavior, feeling and physiology. In: Cognitive neuroscience of emotion. Oxford University Press, New York Google Scholar
  9. 9.
    Argyle M (2001) Bodily Communication, new edn. Routledge, London Google Scholar
  10. 10.
    Mohammad Y, Nishida T (2007) TalkBack: Feedback from a miniature robot. In: AI 2007: Advances in artificial intelligence, 20th Australian joint conference on artificial intelligence, Gold Coast, Australia, December 2007, pp 357–366 Google Scholar
  11. 11.
    Papillo JF, Shapiro D (1990) The cardiovascular system. In: Principles of psychophysiology: physical, social, and inferential elements. Cambridge University Press, Cambridge Google Scholar
  12. 12.
    Rowe DW, Sibert J, Irwin D (1998) Heart rate variability: indicator of user states an aid to human-computer interaction. In: Conference on human factors in computing systems (CHI98) Google Scholar
  13. 13.
    Kidd CD, Breazeal C (2005) Human-robot interaction experiments: lessons learned. In: Robot companions: hard problems and open challenges in robot human interaction symposioum of social intelligence and interaction in animals robots and agents, pp 141–142 Google Scholar
  14. 14.
    Safavian SR, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern 21(3):660–674 CrossRefMathSciNetGoogle Scholar
  15. 15.
    Steinwart I, Christmann A (2008) Support vector machines. Springer, Berlin zbMATHGoogle Scholar
  16. 16.
    Rosner B (1975) On the detection of many outliers. Technometrics 17(2):221–227 zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Mohammad Y, Nishdia T (2008) Human adaptation to a miniature robot: Precursors of mutual adaptation. In: IEEE 17th international symposium on robot and human interactive communication (IEEE Ro-Man), pp 124–129 Google Scholar
  18. 18.
    Liao W, Zhang W, Zhu Z, Ji O (2005) A decision theoretic model for stress recognition and user assistance. In: AAAI 2005, pp 529–534 Google Scholar
  19. 19.
    Stern RM, Ray WJ, Quigley KS (2001) Psychological recording, 2nd edn. Oxford University Press, Oxford Google Scholar
  20. 20.
    Mohammad Y, Nishida T (2009) Robust singular spectrum transform. In: IEA/AIE, pp 123–132 Google Scholar
  21. 21.
    Mohammad Y, Nishida T (2009) Measuring naturalness during close encounters using physiological signal processing. In: IEA/AIE, pp 281–290 Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Electrical Engineering DepartmentAssiut UniversityAssiutEgypt
  2. 2.Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations