Skip to main content
SpringerLink
Log in

Non-facial and non-verbal affective expression for appearance-constrained robots used in victim management

  • Published:
Paladyn

Abstract

Non-facial and non-verbal methods of affective expression are essential for social interaction in appearance-constrained robots such as those used in search and rescue, law enforcement, and military applications. This research identified five main methods of non-facial and non-verbal affective expression (body movements, postures, orientation, color, and sound). Based on an extensive review of literature, prescriptive design recommendations were developed for the appropriate non-facial and non-verbal affective expression methods for three proximity zones of interest (intimate, personal, and social). These design recommendations serve as guidelines to add retroactively affective expression through software with minimal or no physical modification to a robot. A large-scale, complex human-robot interaction study was conducted to validate these design recommendations using 128 participants and four methods of evaluation. The study was conducted in a high-fidelity, confined-space simulated disaster site with all robot interactions performed in the dark. Statistically significant results indicated that participants felt the robots that exhibited affective expressions were more calming, friendly, and attentive, which improved the social human-robot interactions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Breazeal, C.L.: Designing Sociable Robots. Intelligent Robots and Autonomous Agents. MIT Press, Cambridge, Mass. (2002)

    Google Scholar 

  2. Cañamero, L.D., Fredslund, J.: How Does It Feel? Emotional Interaction with a Humanoid Lego Robot. In: K. Dautenhahn (ed.) AAAI 2000 Fall Symposium — Socially Intelligent Agents: The Human in the Loop, vol. Technical Report FS-00-04, pp. 23–28. AAAI, Menlo Park, CA (2000)

    Google Scholar 

  3. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A Survey of Socially Interactive Robots. Robotics and Autonomous Systems 42(3–4), 143 (2003)

    Article  MATH  Google Scholar 

  4. Kirby, R., Forlizzi, J., Simmons, R.: Affective Social Robots. Robotics and Autonomous Systems 58, 322–332 (2010)

    Article  Google Scholar 

  5. Mizoguchi, H., Sato, T., Takagi, K., Nakao, M., Hatamura, Y.: Realization of Expressive Mobile Robot. In: 1997 IEEE International Conference on Robotics and Automation, vol. 1, pp. 581–586 (1997)

    Google Scholar 

  6. Scheeff, M., Pinto, J., Rahardja, K., Snibbe, S., Tow, R.: Experiences with Sparky, a Social Robot. In: Proceedings of the Workshop on Interactive Robot Entertainment (2000)

  7. Bethel, C.L., Murphy, R.R.: Survey of Non-Facial/Non-Verbal Affective Expressions for Appearance-Constrained Robots. IEEE Transactions on Systems, Man, Cybernetics, Part C 38(1) (2008)

  8. Argyle, M.: Bodily Communication. Methuen, London (1975)

    Google Scholar 

  9. Bull, P.: Posture and Gesture, 1st edn. Pergamon Press, Oxford Oxfordshire; New York (1987)

    Google Scholar 

  10. Bethel, C.L., Murphy, R.R.: Affective Expression in Appearance-Constrained Robots. In: 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI2006), pp. 327–328. ACM Press, Salt Lake City, UT (2006)

    Chapter  Google Scholar 

  11. Jones, A.E., Kline, J.A.: Shock. In: J.A. Marx (ed.) Rosen’s Emergency Medicine: Concepts and Clinical Practice, 7th edn. Elsevier, Philadelphia, PA (2009)

    Google Scholar 

  12. Buck, R.: The Communication of Emotion. Guilford Press, New York (1984)

    Google Scholar 

  13. Fast, J.: Body Language. M. Evans; distributed in association with Lippincott, New York (1970)

  14. Bickmore, T.W., Picard, R.W.: Towards Caring Machines. In: E. Dykstra-Erickson, M. Tscheligi (eds.) Conference on Human Factors in Computing Systems CHI’ 04 Extended Abstracts on Human factors in Computing Systems, pp. 1489–1492. ACM Press, New York, NY, USA, Vienna, Austria (2004)

    Chapter  Google Scholar 

  15. Bruce, A., Nourbakhsh, I., Simmons, R.: The Role of Expressiveness and Attention in Human-Robot Interaction. In: IEEE International Conference on Robotics and Automation, (ICRA’ 02), vol. 4, pp. 4138–4142 (2002)

    Google Scholar 

  16. Dautenhahn, K., Walters, M., Woods, S., Koay, K.L., Nehaniv, C.L., Sisbot, A., Alami, R., Siméon, T.: How May I Serve You?: A Robot Companion Approaching a Seated Person in a Helping Context. In: 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (HRI2006), pp. 172–179. ACM Press, New York, NY, USA, Salt Lake City, UT (2006)

    Chapter  Google Scholar 

  17. Fincannon, T., Barnes, L.E., Murphy, R.R., Riddle, D.L.: Evidence of the Need for Social Intelligence in Rescue Robots. In: IEEE/RSJ International Conference on International Conference on Intelligent Robots and Systems (IROS 2004), vol. 2, pp. 1089–1095 (2004)

    Google Scholar 

  18. Moshkina, L., Arkin, R.C.: Human perspective on Affective Robotic Behavior: A Longitudinal Study. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), pp. 2443–2450 (2005)

  19. Sugano, S., Ogata, T.: Emergence of Mind in Robots for Human Interface — Research Methodology and Robot Model. In: 1996 IEEE International Conference on Robotics and Automation, vol. 2, pp. 1191–1198 (1996)

    Google Scholar 

  20. Norman, D.A.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York (2004)

    Google Scholar 

  21. Balkwill, L., Thompson, W.F., Matsunaga, R.I.E.: Recognition of Emotion in Japanese, Western, and Hindustani Music by Japanese Listeners. Japanese Psychological Research 46(4), 337–349 (2004)

    Article  Google Scholar 

  22. Nass, C., Moon, Y.: Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues 56(1), 81–103 (2000)

    Article  Google Scholar 

  23. Takayama, L., Pantofaru, C.: Influences on Proxemic Behaviors in Human-Robot Interaction. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS’09) (2009)

  24. Walters, M.L., Dautenhahn, K., te Boekhorst, R., Kheng Lee, K., Kaouri, C., Woods, S., Nehaniv, C., Lee, D., Werry, I.: The Influence of Subjects’ Personality Traits on Personal Spatial Zones in a Human-Robot Interaction Experiment. In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN 2005), pp. 347–352 (2005)

  25. Bethel, C.L.: Robots Without Faces: Non-Verbal Social Human-Robot Interaction. Dissertation, University of South Florida (2009)

  26. Bethel, C.L., Murphy, R.R.: Non-Facial/Non-Verbal Methods of Affective Expression as Applied to Robot-Assisted Victim Assessment. In: 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI2007), pp. 287–294. ACM Press, Arlington, VA (2007)

    Google Scholar 

  27. Bethel, C.L., Murphy, R.R.: Review of Human Studies Methods in HRI and Recommendations. International Journal of Social Robotics (2010)

  28. Murphy, R.R., Riddle, D., Rasmussen, E.: Robot-Assisted Medical Reachback: A Survey of How Medical Personnel Expect to Interact with Rescue Robots. In: 13th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2004), pp. 301–306 (2004)

  29. Riddle, D.R., Murphy, R.R., Burke, J.L.: Robot-Assisted Medical Reachback: Using Shared Visual Information. In: IEEE International Workshop on Robot and Human Interactive Communication (ROMAN 2005), pp. 635–642. IEEE, Nashville, TN (2005)

    Chapter  Google Scholar 

  30. Paffenbarger, R.S.J., Wing, A., Hyde, R.: Physical Activity as an Index of Heart Attack Risk in College Alumni. American Journal of Epidemiology 108, 161–175 (1978)

    Google Scholar 

  31. Buysse, D.J., Reynolds, C.F., Monk, T.H., Berman, S.R., Kupfer, D.J.: Pittsburgh Sleep Quality Index: A New Instrument for Psychiatric Practice and Research. Psychiatry Research 28, 193–213 (1989)

    Article  Google Scholar 

  32. Cohen, S., Kamarck, T., Mermelstein, R.: A Global Measure of Perceived Stress. Journal of Health and Social Behavior 24, 385–396 (1983)

    Article  Google Scholar 

  33. Blake, R.L.J., McKay, D.A.: A Single-Item Measure of Social Supports as a Predictor of Morbidity. Journal of Family Practice 22(1), 82–84 (1986)

    Google Scholar 

  34. Spielberger, C., Gorsuch, R., Lushene, R.: Manual for the State-Trait Anxiety Inventory. Consulting Psychologist Press, Palo Alto, CA (1970)

    Google Scholar 

  35. Watson, D., Clark, L.A., Tellegen, A.: Development and Validation of Brief Measures of Positive and Negative Affect: The PANAS scales. Journal of Personality and Social Psychology 54(6), 1063–1070 (1988)

    Article  Google Scholar 

  36. Bradley, M.M., Lang, P.J.: Measuring Emotion: The Self-Assessment Manikin and the Semantic Differential. Journal of Behavioral Therapy and Experimental Psychiatry 25, 49–59 (1994)

    Article  Google Scholar 

  37. Mutlu, B., Osman, S., Forlizzi, J., Hodgins, J.K., Kiesler, S.: Task Structure and User Attributes as Elements of Human-Robot Interaction Design. In: 15th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2006). IEEE, University of Hertfordshire, Hatfield, UK (2006)

    Google Scholar 

  38. Bethel, C.L., Salomon, K., Burke, J.L., Murphy, R.R.: Psychophysiological Experimental Design for Use in Human-Robot Interaction Studies. In: The 2007 International Symposium on Collaborative Technologies and Systems (CTS 2007). IEEE, Orlando, FL (2007)

    Google Scholar 

  39. Bethel, C.L., Salomon, K., Murphy, R.R., Burke, J.L.: Survey of Psychophysiology Measurements Applied to Human-Robot Interaction. In: 16th IEEE International Symposium on Robot and Human Interactive Communication. Jeju Island, South Korea (2007)

    Google Scholar 

  40. Bartneck, C.: Affective Expressions of Machines. Masters thesis, Eindhoven University of Technology (2000)

  41. Dautenhahn, K., Ogden, B., Quick, T.: From Embodied to Socially Embedded Agents — Implications for Interaction-Aware Robots. Cognitive Systems Research 3(3), 397 (2002)

    Article  Google Scholar 

  42. Hall, E.T.: The Hidden Dimension. Doubleday, New York (1966)

    Google Scholar 

  43. Spiegel, J.P., Machotka, P.: Messages of the Body. Free Press, New York, NY (1974)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Yale University, New Haven, CT, 06511, USA

    Cindy L. Bethel

  2. Center for Robot-Assisted Search and Rescue, Texas A & M University, College Station, TX, 77843, USA

    Robin R. Murphy

Authors
  1. Cindy L. Bethel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robin R. Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cindy L. Bethel.

Additional information

This material is based upon work supported by the National Science Foundation under Grant#0937060 to the Computing Research Association for the CIFellows Project, a National Science Foundation Graduate Research Fellowship Award Number DGE-0135733, ARL Number W911NF-06-2-0041, IEEE Robotics and Automation Society Graduate Fellowship, and a Microsoft HRI grant.

Cindy L. Bethel is an NSF Computing Innovation Fellow in Computer Science at Yale University. In 2009, she received her Ph.D. in Computer Science and Engineering from the University of South Florida. She was an NSF Graduate Research Fellow and IEEE Robotics and Automation Society Graduate Fellow. Her research focuses in the areas of human-robot interaction, affective computing, robotics, and artificial intelligence. Cindy L. Bethel received a B.S. in Computer Science summa cum laude from the University of South Florida in 2004.

Robin R. Murphy received a B.M.E. in Mechanical Engineering, a M.S. and Ph.D. in Computer Science in 1980, 1989, and 1992, respectively, from Georgia Tech. She is the Raytheon Professor of Computer Science & Engineering at Texas A&M. Her research interests are artificial intelligence, human-robot interaction, and heterogeneous teams of robots. In 2008, she was awarded the Al Aube Outstanding Contributor award by the AUVSI Foundation, for her insertion of ground, air, and sea robots for urban search and rescue at the 9/11 World Trade Center disaster, Hurricanes Katrina and Charley, and the Crandall Canyon Utah mine collapse. She is an associate editor for IEEE Intelligent Systems, a Distinguished Speaker for the IEEE Robotics and Automation Society, and has served on numerous boards, including the Defense Science Board, USAF SAB, NSF CISE Advisory Council, and DARPA ISAT.

About this article

Cite this article

Bethel, C.L., Murphy, R.R. Non-facial and non-verbal affective expression for appearance-constrained robots used in victim management. Paladyn 1, 219–230 (2010). https://doi.org/10.2478/s13230-011-0009-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2478/s13230-011-0009-5

Keywords

Search

Navigation