Skip to main content
Book cover

International Conference on Social Robotics

ICSR 2020: Social Robotics pp 144–156Cite as

Human-Robot Collaboration and Dialogue for Fault Recovery on Hierarchical Tasks

Part of the Lecture Notes in Computer Science book series (LNAI,volume 12483)

Abstract

Robotic systems typically follow a rigid approach to task execution, in which they perform the necessary steps in a specific order, but fail when having to cope with issues that arise during execution. We propose an approach that handles such cases through dialogue and human-robot collaboration. The proposed approach contributes a hierarchical control architecture that 1) autonomously detects and is cognizant of task execution failures, 2) initiates a dialogue with a human helper to obtain assistance, and 3) enables collaborative human-robot task execution through extended dialogue in order to 4) ensure robust execution of hierarchical tasks with complex constraints, such as sequential, non-ordering, and multiple paths of execution. The architecture ensures that the constraints are adhered to throughout the entire task execution, including during failures. The recovery of the architecture from issues during execution is validated by a human-robot team on a building task.

Keywords

  • Human-robot collaboration
  • Human-robot dialogue
  • Dialogue-based fault recovery
  • Hierarchical planning

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-62056-1_13
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-62056-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Arkin, R.C.: An Behavior-based Robotics, 1st edn. MIT Press, Cambridge (1998)

    Google Scholar 

  2. Beetz, M., et al.: Robotic roommates making pancakes. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots, pp. 529–536, October 2011

    Google Scholar 

  3. Blankenburg, J., et al.: A distributed control architecture for collaborative multi-robot task allocation. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 585–592, November 2017

    Google Scholar 

  4. Bradski, G., Kaehler, A.: Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc. (2008)

    Google Scholar 

  5. Breazeal, C., Hoffman, G., Lockerd, A.: Teaching and working with robots as a collaboration. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 3, pp. 1030–1037. IEEE Computer Society (2004)

    Google Scholar 

  6. Fong, T., Thorpe, C., Baur, C.: Multi-robot remote driving with collaborative control. IEEE Trans. Ind. Electron. 50(4), 699–704 (2003)

    CrossRef  Google Scholar 

  7. Fong, T., Thorpe, C., Baur, C.: Robot, asker of questions. Robot. Autonomous Syst. 42(3), 235–243 (2003). https://doi.org/10.1016/S0921-8890(02)00378-0, socially Interactive Robots

  8. Fraser, L., Rekabdar, B., Nicolescu, M., Nicolescu, M., Feil-Seifer, D., Bebis, G.: A compact task representation for hierarchical robot control. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pp. 697–704, November 2016. https://doi.org/10.1109/HUMANOIDS.2016.7803350

  9. Gassend, B.: sound\_play ros package. http://wiki.ros.org/sound_play

  10. Hayes, B., Scassellati, B.: Autonomously constructing hierarchical task networks for planning and human-robot collaboration. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 5469–5476. IEEE (2016)

    Google Scholar 

  11. Hayes, B., Shah, J.A.: Improving robot controller transparency through autonomous policy explanation. In: 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 303–312. IEEE (2017)

    Google Scholar 

  12. Hernandez, C., et al.: Team delft’s robot winner of the amazon picking challenge 2016. In: Behnke, S., Sheh, R., Sarıel, S., Lee, D.D. (eds.) RoboCup 2016: Robot World Cup XX, pp. 613–624. Springer International Publishing, Cham (2017)

    CrossRef  Google Scholar 

  13. Huggins-Daines, D., Kumar, M., Chan, A., Black, A.W., Ravishankar, M., Rudnicky, A.I.: Pocketsphinx: a free, real-time continuous speech recognition system for hand-held devices. In: 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, vol. 1, p. I. IEEE (2006)

    Google Scholar 

  14. Knepper, R.A., Tellex, S., Li, A., Roy, N., Rus, D.: Recovering from failure by asking for help. Auton. Robots 39(3), 347–362 (2015). https://doi.org/10.1007/s10514-015-9460-1

    CrossRef  Google Scholar 

  15. Mohseni-Kabir, A., Rich, C., Chernova, S., Sidner, C.L., Miller, D.: Interactive hierarchical task learning from a single demonstration. In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 205–212. ACM (2015)

    Google Scholar 

  16. Quigley, M., et al.: ROS: an open-source Robot Operating System. In: ICRA Workshop on Open Source Software, vol. 3 (2009)

    Google Scholar 

  17. Rybski, P.E., Yoon, K., Stolarz, J., Veloso, M.M.: Interactive robot task training through dialog and demonstration. In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, pp. 49–56. ACM (2007)

    Google Scholar 

  18. Schillinger, P., Kohlbrecher, S., von Stryk, O.: Human-robot collaborative high-level control with application to rescue robotics. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2796–2802. IEEE (2016)

    Google Scholar 

  19. Sucan, I.A., Chitta, S.: Moveit! (2013). http://moveit.ros.org

Download references

Acknowledgment

This work was supported by the National Science Foundation (IIS-1757929) and by the Office of Naval Research (ONR) award #N00014-16-1-2312.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, 89557, USA

    Janelle Blankenburg, Mariya Zagainova, Gabrielle Talavera, Monica Nicolescu & David Feil-Seifer

  2. Department of Computer Science, Brigham Young University, Provo, UT, 84602, USA

    S. Michael Simmons

Authors
  1. Janelle Blankenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariya Zagainova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Michael Simmons
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabrielle Talavera
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Monica Nicolescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Feil-Seifer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janelle Blankenburg .

Editor information

Editors and Affiliations

  1. Pennsylvania State University, University Park, PA, USA

    Alan R. Wagner

  2. University of Nevada Reno, Reno, NV, USA

    Ph.D. David Feil-Seifer

  3. University of Denver, Denver, CO, USA

    Kerstin S. Haring

  4. University of Naples Federico II, Naples, Italy

    Prof. Silvia Rossi

  5. Colorado School of Mines, Golden, CO, USA

    Thomas Williams

  6. Wichita State University, Wichita, KS, USA

    Hongsheng He

  7. National University of Singapore, Singapore, Singapore

    Prof. Dr. Shuzhi Sam Ge

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Blankenburg, J., Zagainova, M., Simmons, S.M., Talavera, G., Nicolescu, M., Feil-Seifer, D. (2020). Human-Robot Collaboration and Dialogue for Fault Recovery on Hierarchical Tasks. In: , et al. Social Robotics. ICSR 2020. Lecture Notes in Computer Science(), vol 12483. Springer, Cham. https://doi.org/10.1007/978-3-030-62056-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62056-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62055-4

  • Online ISBN: 978-3-030-62056-1

  • eBook Packages: Computer ScienceComputer Science (R0)