Autonomous Robots

, Volume 39, Issue 3, pp 313–329 | Cite as

Effective task training strategies for human and robot instructors

  • Allison Sauppé
  • Bilge Mutlu


From teaching in labs to training for assembly, a role that robots are expected to play is to instruct their users in completing physical tasks. While instruction requires a range of capabilities, such as use of verbal and nonverbal language, a fundamental requirement for an instructional robot is to provide its students with instructions in a way that maximizes their task performance. In this paper, we present an autonomous instructional robot and investigate how different instructional strategies affect user performance and experience. Our analysis of human instructor–trainee interactions identified two key instructional strategies: (1) grouping instructions together and (2) summarizing the outcome of subsequent instructions. We implemented these strategies into a humanlike robot that autonomously instructed its users in a pipe-assembly task. To achieve autonomous instruction, we also developed a repair mechanism that enabled the robot to correct mistakes and misunderstandings. An evaluation of the instructional strategies in a human–robot interaction study showed that employing the grouping strategy resulted in faster task completion and increased rapport with the robot, although it also increased the number of task breakdowns. A comparison of our results with the human instructor–trainee interactions revealed many similarities, areas where our model for robot instructors could be improved, and the nuanced ways in which human instructors use training strategies such as summarization. Our findings offer strong implications for the design of instructional robots and directions of future research.


Repair Interactive robot systems Human–robot interaction Instructional systems Instructional strategies  Task training Human instruction Autonomous robot systems 



We thank Brandi Hefty, Jilana Boston, Ross Luo, Chien-Ming Huang, and Catherine Steffel for their contributions to and National Science Foundation Awards 1149970 and 1426824 and Mitsubishi Heavy Industries, Ltd. for their support of this work. Some of the findings from the human–human and human–robot data presented here have been published in the Proceeding of Robotics: Science and Systems (Sauppé and Mutlu 2014a) and included in a book chapter in Robots that Talk and Listen (Markowitz 2015).


  1. Alfieri, L., Brooks, P., Aldrich, N., & Tenenbaum, H. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18.CrossRefGoogle Scholar
  2. Andrist, S., Spannan, E., & Mutlu, B. (2013). Rhetorical robots: Making robots more effective speakers using linguistic cues of expertise. In Proc. HRI’13 (pp. 341–348).Google Scholar
  3. Bicho, E., Erlhagen, W., Louro, L., & Costa e Silva, E. (2011). Neuro-cognitive mechanisms of decision making in joint action: A human-robot interaction study. Human Movement Science, 30(5), 846–868.CrossRefGoogle Scholar
  4. Blaylock, N., Allen, J., & Ferguson, G. (2003). Managing communicative intentions with collaborative problem solving. In K. JCJ & R. Smith (Eds.), Current and new directions in discourse and dialogue (pp. 63–84). Berlin: Springer.CrossRefGoogle Scholar
  5. Boucher, J. D., Pattacini, U., Lelong, A., Bailly, G., Elisei, F., Fagel, S., et al. (2012). I reach faster when I see you look: Gaze effects in human–human and human–robot face-to-face cooperation. Frontiers in Neurorobotics, 6, 1–11.CrossRefGoogle Scholar
  6. Brooks, A. G., & Breazeal, C. (2006). Working with robots and objects: Revisiting deictic reference for achieving spatial common ground. In Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction (pp. 297–304). New York: ACM.Google Scholar
  7. Clark, H. H. (1996). Using language (Vol. 1996). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  8. Clark, H. H. (2005). Coordinating with each other in a material world. Discourse Studies, 7(4–5), 507–525.CrossRefGoogle Scholar
  9. Foster, M., Giuliani, M., Isard, A., Matheson, C., Oberlander, J., & Knoll, A. (2009). Evaluating description and reference strategies in a cooperative human-robot dialogue system. In Proc. IJCAI’09 (pp. 1818–1823).Google Scholar
  10. Gallese, V., & Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2(12), 493–501.CrossRefGoogle Scholar
  11. Gonsior, B., Wollherr, D., & Buss, M. (2010). Towards a dialog strategy for handling miscommunication in human-robot dialog. In Proc. RO-MAN’10 Google Scholar
  12. Gray, J., Breazeal, C., Berlin, M., Brooks, A., & Lieberman, J. (2005). Action parsing and goal inference using self as simulator. In Proc. RO-MAN’05 Google Scholar
  13. Grosz, B., & Kraus, S. (1996). Collaborative plans for complex group action. Artificial Intelligence, 86(2), 269–357.MathSciNetCrossRefGoogle Scholar
  14. Grosz, B., & Sidner, C. (1986). Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3), 175–204.Google Scholar
  15. Hato, Y., Satake, S., Kanda, T., Imai, M., & Hagita, N. (2010). Pointing to space: Modeling of deictic interaction referring to regions. In Proceedings of the 5th ACM/IEEE international conference on human-robot interaction (pp. 301–308). New York: IEEE PressGoogle Scholar
  16. Hirst, G., McRoy, S., Heeman, P., Edmonds, P., & Horton, D. (1994). Repairing conversational misunderstandings and non-understandings. Speech Communication, 15(3), 213–229.CrossRefGoogle Scholar
  17. Hoey, J., Poupart, P., Boutilier, C., & Mihailidis, A. (2005). POMDP models for assistive technology. In Proceedings of the AAAI 2005 fall symposium.Google Scholar
  18. Huang, C. M., & Mutlu, B. (2012). Robot behavior toolkit: Generating effective social behaviors for robots. In Proc. HRI’12 (pp. 25–32).Google Scholar
  19. Kanda, T., Sato, R., Saiwaki, N., & Ishiguro, H. (2007). A two-month field trial in an elementary school for long-term human-robot interaction. IEEE Transactions on Robotics, 23(5), 962–971.CrossRefGoogle Scholar
  20. Koulouri, T., & Lauria, S. (2009). Exploring miscommunication and collaborative behaviour in HRI. In Proc. SIGDIAL’09.Google Scholar
  21. Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.zbMATHMathSciNetCrossRefGoogle Scholar
  22. Lee, M., Kiesler, S., Forlizzi, J., Srinivasa, S., & Rybski, P. (2010). Gracefully mitigating breakdowns in robotic services. In Proc. HRI’10.Google Scholar
  23. Markowitz, J. (2015). Robots that talk and listen. Boston: Walter de Gruyter.CrossRefGoogle Scholar
  24. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
  25. Nicolescu, M., & Mataric, M. (2003). Linking perception and action in a control architecture for human-robot domains. In Proc. HICSS’03.Google Scholar
  26. Norrick, N. (1991). On the organization of corrective exchanges in conversation. Journal of Pragmatics, 16(1), 59–83.CrossRefGoogle Scholar
  27. Reigeluth, C., Merrill, M., Wilson, B., & Spiller, R. (1980). The elaboration theory of instruction: A model for sequencing and synthesizing instruction. Instructional Science, 9(3), 195–219.CrossRefGoogle Scholar
  28. Sakita, K., Ogawara, K., Murakami, S., Kawamura, K., & Ikeuchi, K. (2004). Flexible cooperation between human and robot by interpreting human intention from gaze information. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004 (IROS 2004) Proceedings (Vol. 1, pp. 846–851). New York: IEEEGoogle Scholar
  29. Sauppé, A., & Mutlu, B. (2014a). Effective task training strategies for instructional robots. In Proceedings of the 10th annual robotics: science and systems conference.Google Scholar
  30. Sauppé, A., & Mutlu, B. (2014b). Robot deictics: How gesture and context shape referential communication. In Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction (pp. 342–349). ACM.Google Scholar
  31. Schegloff, E., Jefferson, G., & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53, 361–382.CrossRefGoogle Scholar
  32. Seedhouse, P. (1999). The relationship between context and the organization of repair in the l2 classroom. International Review of Applied Linguistics in Language Teaching, 37(1), 59–80.Google Scholar
  33. Staudte, M., & Crocker, M. (2009). Visual attention in spoken human-robot interaction. In Proc. HRI’09 (pp. 77–84).Google Scholar
  34. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.CrossRefGoogle Scholar
  35. Tanaka, F., & Movellan, J. (2006). Behavior analysis of children’s touch on a small humanoid robot: Long-term observation at a daily classroom over three months. In Proc. RO-MAN’06.Google Scholar
  36. Tanaka, R., & Kimura, T. (2009) The use of robots in early education: A scenario based on ethical consideration. In Proc. RO-MAN’09.Google Scholar
  37. Tomasello, M., Conti-Ramsden, G., & Ewert, B. (1990). Young children’s conversations with their mothers and fathers: Differences in breakdown and repair. Journal of Child Language, 17(01), 115–130.CrossRefGoogle Scholar
  38. Torrey, C., Powers, A., Marge, M., Fussell, S., & Kiesler, S. (2006). Effects of adaptive robot dialogue on information exchange and social relations. In Proc. HRI’06 (pp. 126–133).Google Scholar
  39. Torrey, C., Powers, A., Fussell, S., & Kiesler, S. (2007). Exploring adaptive dialogue based on a robot’s awareness of human gaze and task progress. In Proc. HRI’07.Google Scholar
  40. Trafton, J., Cassimatis, N., Bugajska, M., Brock, D., Mintz, F., & Schultz, A. (2005). Enabling effective human-robot interaction using perspective-taking in robots. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 35(4), 460–470.CrossRefGoogle Scholar
  41. Zahn, C. (1984). A reexamination of conversational repair. Communications Monographs, 51(1), 56–66.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Computer SciencesUniversity of Wisconsin–MadisonMadisonUSA

Personalised recommendations