Skip to main content

The Relevance of Social Cues in Assistive Training with a Social Robot

Part of the Lecture Notes in Computer Science book series (LNAI,volume 11357)


This paper examines whether social cues, such as facial expressions, can be used to adapt and tailor a robot-assisted training in order to maximize performance and comfort. Specifically, this paper serves as a basis in determining whether key facial signals, including emotions and facial actions, are common among participants during a physical and cognitive training scenario. In the experiment, participants performed basic arm exercises with a social robot as a guide. We extracted facial features from video recordings of participants and applied a recursive feature elimination algorithm to select a subset of discriminating facial features. These features are correlated with the performance of the user and the level of difficulty of the exercises. The long-term aim of this work, building upon the work presented here, is to develop an algorithm that can eventually be used in robot-assisted training to allow a robot to tailor a training program based on the physical capabilities as well as the social cues of the users.


  • Social cues
  • Facial signals
  • Robot-assisted training

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    CrossRef  MATH  Google Scholar 

  2. Broekens, J.: Emotion and reinforcement: affective facial expressions facilitate robot learning. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) Artifical Intelligence for Human Computing. LNCS (LNAI), vol. 4451, pp. 113–132. Springer, Heidelberg (2007).

    CrossRef  Google Scholar 

  3. Calle, M.L., Urrea, V.: Letter to the editor: stability of random forest importance measures. Brief. Bioinform. 12(1), 86–89 (2010)

    CrossRef  Google Scholar 

  4. Danner, F.W., Lonky, E.: A cognitive-developmental approach to the effects of rewards on intrinsic motivation. Child Dev. 52, 1043–1052 (1981)

    CrossRef  Google Scholar 

  5. Deci, E.L., Ryan, R.M.: The “what” and “why” of goal pursuits: human needs and the self-determination of behavior. Psychol. Inq. 11(4), 227–268 (2000)

    CrossRef  Google Scholar 

  6. Ekman, P., Rosenberg, E.L.: What The Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (1997)

    Google Scholar 

  7. El Kaliouby, R., Robinson, P.: Real-time inference of complex mental states from facial expressions and head gestures. In: Kisačanin, B., Pavlović, V., Huang, T.S. (eds.) Real-Time Vision for Human-Computer Interaction, pp. 181–200. Springer, Boston (2005).

    CrossRef  Google Scholar 

  8. Fan, J., et al.: A robotic coach architecture for elder care (ROCARE) based on multi-user engagement models. IEEE Trans. Neural Syst. Rehabil. Eng. 25(8), 1153–1163 (2017)

    CrossRef  Google Scholar 

  9. Fasola, J., Mataric, M.J.: Using socially assistive human-robot interaction to motivate physical exercise for older adults. Proc. IEEE 100(8), 2512–2526 (2012)

    CrossRef  Google Scholar 

  10. Fasola, J., Matarić, M.J.: A socially assistive robot exercise coach for the elderly. J. Hum.-Robot Interact. 2(2), 3–32 (2013)

    CrossRef  Google Scholar 

  11. Gordon, G., et al.: Affective personalization of a social robot tutor for children’s second language skills. In: AAAI, pp. 3951–3957 (2016)

    Google Scholar 

  12. Görer, B., Salah, A.A., Levent Akın, H.: An autonomous robotic exercise tutor for elderly people. Auton. Robots 41(3), 657–678 (2017)

    CrossRef  Google Scholar 

  13. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002)

    CrossRef  MATH  Google Scholar 

  14. McDaniel, B., D’Mello, S., King, B., Chipman, P., Tapp, K., Graesser, A.: Facial features for affective state detection in learning environments. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 29 (2007)

    Google Scholar 

  15. McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., el Kaliouby, R.: AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3723–3726. ACM (2016)

    Google Scholar 

  16. Miller, G.A.: The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol. Rev. 63(2), 81 (1956)

    CrossRef  Google Scholar 

  17. Poggi, I., D’Errico, F.: Social signals: a psychological perspective. In: Salah, A., Gevers, T. (eds.) Computer Analysis of Human Behavior, pp. 185–225. Springer, London (2011).

    CrossRef  Google Scholar 

  18. Ritschel, H., Baur, T., André, E.: Adapting a robot’s linguistic style based on socially-aware reinforcement learning. In: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 378–384. IEEE (2017)

    Google Scholar 

  19. Vail, A.K., Grafsgaard, J.F., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Predicting learning from student affective response to tutor questions. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 154–164. Springer, Cham (2016).

    CrossRef  Google Scholar 

Download references


This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 721619 for the SOCRATES project.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Neziha Akalin .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Akalin, N., Kiselev, A., Kristoffersson, A., Loutfi, A. (2018). The Relevance of Social Cues in Assistive Training with a Social Robot. In: Ge, S., et al. Social Robotics. ICSR 2018. Lecture Notes in Computer Science(), vol 11357. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05203-4

  • Online ISBN: 978-3-030-05204-1

  • eBook Packages: Computer ScienceComputer Science (R0)