Advertisement

Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments

  • A. Hong
  • O. Igharoro
  • Y. Liu
  • F. Niroui
  • G. Nejat
  • B. Benhabib
Article
  • 46 Downloads

Abstract

Teams of semi-autonomous robots can provide valuable assistance in Urban Search and Rescue (USAR) by efficiently exploring cluttered environments and searching for potential victims. Their advantage over solely teleoperated robots is that they can address the task handling and situation awareness limitations of human operators by providing some level of autonomy to the multi-robot team. Our research focuses on developing learning-based semi-autonomous controllers for rescue robot teams. In this paper, we specifically investigate the influence of the operator-to-robot ratio on the performance of our proposed MAXQ hierarchical reinforcement learning based semi-autonomous controller for USAR missions. In particular, we propose a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots. A rigorous comparative study of our learning-based semi-autonomous controller versus a fully teleoperation-based approach was conducted in a 3D simulation environment. The results, as expected, show that, for both semi-autonomous and teleoperation modes, the total scene exploration time increases as the number of robots utilized increases. However, when using the proposed learning-based semi-autonomous controller, the rate of exploration-time increase and operator-interaction effort are significantly lower, while task performance is significantly higher. Furthermore, an additional case study showed that our learning-based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.

Keywords

Urban search and rescue Multi-robot rescue teams Semi-autonomous control Operator-to-robot ratio 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

This work was funded by the Natural Science and Engineering Research Council of Canada (NSERC) and the Canada Research Chairs (CRC) Program.

References

  1. 1.
    Casper, J., Micire, M., Murphy, R.: Issues in intelligent robots for search and rescue. In: Proceedings SPIE Unmanned Ground Vehicle Technology II, pp 292–302 (2000)Google Scholar
  2. 2.
    Casper, J., Murphy, R.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans. Syst. Man Cybern. B 33(3), 367–385 (2003)CrossRefGoogle Scholar
  3. 3.
    Liu, Y., Nejat, G.: Robotic urban search and rescue: a survey from the control perspective. J. Intell. Robot. Syst. 72(2), 147–165 (2013)CrossRefGoogle Scholar
  4. 4.
    Wong, C., Seet, G., Sim, S.: Multiple-robot systems for USAR: key design attributes and deployment issues. Int. J. Adv. Robot. Syst. 8(1), 85–101 (2011)CrossRefGoogle Scholar
  5. 5.
    Gatsoulis, Y., Virk, G.S., Dehghani-Sanij, A.A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Decis. Mak. 4(1), 69–98 (2010)CrossRefGoogle Scholar
  6. 6.
    Adams, J.A.: Multiple robot/single human interaction: effects on perceived workload. Behav. Inf. Technol. 28 (2), 183–198 (2009)CrossRefGoogle Scholar
  7. 7.
    de Visser, E., Parasuraman, R.: Adaptive aiding of human-robot teaming effects of imperfect automation on performance, trust, and workload. J. Cogn. Eng. Decis. Mak. 5(2), 209–231 (2011)CrossRefGoogle Scholar
  8. 8.
    Chen, J.Y.C., Durlach, P.J., Sloan, J.A., Bowens, L.D.: Human-robot interaction in the context of simulated route reconnaissance missions. Mil. Psychol. 20(3), 135–149 (2008)CrossRefGoogle Scholar
  9. 9.
    Breslow, L.A., Gartenberg, D., McCurry, J.M., Trafton, J.G.: Dy7namic operator overload: a model for predicting workload during supervisory control. IEEE Trans. Hum. Mach. Syst. 44(1), 30–40 (2014)CrossRefGoogle Scholar
  10. 10.
    McKendrick, R., Shaw, T, de Visser, E., Sager, H., Kidwell, B., Parasuraman, R.: Team performance in networked supervisory control of unmanned air vehicles effects of automation, working memory, and communication content. Hum. Factors 56(3), 463–475 (2014)CrossRefGoogle Scholar
  11. 11.
    Fincannon, T.D., Evans, A.W., Phillips, E., Jentsch, F., Keebler, J.: The influence of team size and communication modality on team effectiveness with unmanned systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 419–423 (2009)Google Scholar
  12. 12.
    de Visser, E., Kidwell, B., Payne, J., Lu, L., Parker, J., Brooks, N., Chabuk, T., Spriggs, S., Freedy, A., Scerri, P., Parasuraman, R.: Best of both worlds design and evaluation of an adaptive delegation interface. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 255–259 (2013)Google Scholar
  13. 13.
    Cummings, M.L., Bertucelli, L.F., Macbeth, J., Surana, A.: Task versus vehicle-based control paradigms in multiple unmanned vehicle supervision by a single operator. IEEE Trans. Hum. Mach. Syst. 44(3), 353–361 (2014)CrossRefGoogle Scholar
  14. 14.
    Prewett, M.S., Saboe, K.N., Johnson, R.C., Coovert, M.D., Elliot, L.R.: Workload in human-robot interaction: a review of manipulations and outcomes. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 1393–1397 (2009)Google Scholar
  15. 15.
    Velagapudi, P., Scerri, P., Sycara, K., Wang, H., Lewis, M., Wang, J.: Scaling effects in multi-robot control. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2121–2126 (2008)Google Scholar
  16. 16.
    Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., Grocholsky, B., Kumar, V., Taylor, C.J., Endo, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situation awareness. J. Field Robot. 24(11), 991–1014 (2007)CrossRefGoogle Scholar
  17. 17.
    Zhang, K., Xiaobo, L.: Human-robot team coordination that considers human fatigue. J. Adv. Robot. Syst. 11(6) (2014)Google Scholar
  18. 18.
    Olsen, D.R., Goodrich, M.A.: Metrics for evaluating human–robot interactions. In: Proceedings of the PERMIS (2003)Google Scholar
  19. 19.
    Gatsoulis, Y., Virk, G., Dehghani-Sanij, A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Dec. Mak. 4(1), 69–98 (2010)CrossRefGoogle Scholar
  20. 20.
    Gao, F., Cummings, M.L., Solovey, E.T.: Modeling teamwork in supervisory control of multiple robots. IEEE Trans. Hum. Mach. Syst. 44(4), 441–453 (2014)CrossRefGoogle Scholar
  21. 21.
    Lewis, M., Wang, H., Chien, S.Y.: Process and performance in human-robot teams. J. Cogn. Eng. Decis. Mak. 5(2), 186–208 (2011)CrossRefGoogle Scholar
  22. 22.
    Gunn, T., Anderson, J.: Dynamic heterogeneous team formation for robotic urban search and rescue. J. Comp. Syst. Sci. 81(3), 553–567 (2015)CrossRefGoogle Scholar
  23. 23.
    Rosenfeld, A., Agmon, N., Maksimov, O., Azaria, A., Kraus, S.: Intelligent agent supporting human-multi-robot team collaboration. In: Proceedings of the AAMAS Workshop ARMS (2015)Google Scholar
  24. 24.
    Nourjou, R., Smith, S.F., Hatayama, M., Szekely, P.: Intelligent algorithm for assignment of agents to human strategy in centralized multi-agent coordination. J. Softw. 9(10), 2586–2597 (2014)CrossRefGoogle Scholar
  25. 25.
    Chen, J.Y.C., Barnes, M.: Supervisory control of multiple robots: effects of imperfect automation and individual differences. Hum. Factors: J. Hum. Factors Ergon. Soc. 54(2), 157–174 (2012)CrossRefGoogle Scholar
  26. 26.
    Vilela, J., Liu, Y., Nejat, G.: Semi-autonomous exploration with robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2013)Google Scholar
  27. 27.
    Doroodgar, B., Liu, Y., Nejat, G.: A learning-based semi-autonomous controller for robotic exploration of unknown disaster scenes while searching for victims. IEEE Trans. Cybern. 44(12), 2719–2732 (2014)CrossRefGoogle Scholar
  28. 28.
    Niroui, F., Sprenger, B., Nejat, G.: Robot exploration in unknown cluttered environments when dealing with uncertainty. In: Proceedings of the IEEE International Symposium on Robotics and Intelligent Sensors (2017)Google Scholar
  29. 29.
    Liu, Y., Nejat, G., Doroodgar, B.: Learning-based semi-autonomous control for robots in urban search and rescue. In: Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2012)Google Scholar
  30. 30.
    Liu, Y., Nejat, G.: Multirobot cooperative learning for semiautonomous control in urban search and rescue applications. J. Field Robot. 33(4), 512–536 (2016)CrossRefGoogle Scholar
  31. 31.
    Brooks, N., Scerri, P., Sycara, K.: Asynchronous control with ATR for large robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 444–448 (2011)Google Scholar
  32. 32.
    Wang, H., Lewis, M., Chien, S., Velagapudi, P.: Scaling effects for synchronous vs. asynchronous video in multi-robot search. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 364–368 (2009)Google Scholar
  33. 33.
    Sato, N., Matsuno, F., Yamasaki, T., Kamegawa, T., Shiroma, N., Igarashi, H.: Cooperative task execution by a multiple robot team and its operators in search and rescue operations. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1083–1088 (2004)Google Scholar
  34. 34.
    Lewis, M., Wang, H., Chien, S.Y.: Choosing autonomy modes for multirobot search. Hum. Factors 52 (2), 225–233 (2010)CrossRefGoogle Scholar
  35. 35.
    Mouaddib, A., Zilberstein, S., Beynier, A., Jeanpierre, L.: A decision-theoretic approach to cooperative control and adjustable autonomy. In: Proceedings of the European Conference on Artificial Intelligence, pp 971–972 (2010)Google Scholar
  36. 36.
    Côté, N., Canu, A., Bouzid, M., Mouaddib, A.: Humans-robots sliding collaboration control in complex environments with adjustable autonomy. In: Proceedings of the IEEE/WIC/ACM International Conference Web Intelligence and Intelligent Agent Technology, pp 146–153 (2012)Google Scholar
  37. 37.
    Wray, K.H., Pineda, L., Zilberstein, S.: Hierarchical approach to transfer of control in semi-autonomous systems. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp 517–523 (2016)Google Scholar
  38. 38.
    Zhang, K., Collins, E.G., Shi, D.: Centralized and distributed task allocation in multi-robot teams via a stochastic clustering auction. J. Auton. Adapt. Syst. 7(2) (2012)Google Scholar
  39. 39.
    Zhang, K., Collins, E.G., Barbu, A.: An efficient stochastic clustering auction for heterogenous robotic collaborative teams. J. Intell. Robot. Syst. 72(3–4), 541–558 (2013)CrossRefGoogle Scholar
  40. 40.
    Zhang, K., Collins, E.G., Barbu, A.: A novel stochastic clustering auction for task allocation in multi-robot teams. In: Proceedings of the IEEE/RSJ International Conference on Intelligent and Robotic Systems, pp 3300–3307 (2010)Google Scholar
  41. 41.
    Zhang, K., Collins, E.G., Shi, D., Liu, X., Chuy, O.: A stochastic clustering auction (SCA) for centralized and distributed task allocation in multi-agent teams. Distrib. Auton. Robot. Syst. 8, 345–354 (2009)Google Scholar
  42. 42.
    Zhang, Z., Littman, M., Chen, X.: Covering number as a complexity measure for POMDP planning and learning. In: AAAI Conference on Artificial Intelligence, pp 1853–1859 (2012)Google Scholar
  43. 43.
    Dietterich, D.: Hierarchical reinforcement learning with MAXQ value function decomposition. J. Artif. Intell. Res. 13, 227–303 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Liu, Y., Nejat, G., Vilela, J.: Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics (2013)Google Scholar
  45. 45.
    Doroodgar, B., Nejat, G.: A hierarchical reinforcement learning based control architecture for semi-autonomous rescue robots in cluttered environments. In: IEEE International Conference on Automation Science and Engineering, pp 948–953 (2010)Google Scholar
  46. 46.
    Chien, S., Wang, H., Lewis, M.: Effects of alarms on control of robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 434–438 (2011)Google Scholar
  47. 47.
    Oppermann, R.: Adaptive User Support: Ergonomic Design of Manually and Automatically Adaptable Software. CRC Press, Boca Raton (1994)Google Scholar
  48. 48.
    Parasuraman, R., Galster, S., Squire, P., Furukawa, H., Miller, C.: A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with RoboFlag. IEEE Trans. Cybern. 35(4), 481–493 (2005)CrossRefGoogle Scholar
  49. 49.
    Clare, A., Cummings, M., How, J., Whitten, A., Toupet, O.: Operator object function guidance for a real-time unmanned vehicle scheduling algorithm. J. Aerosp. Comput. Inf. Commun. 9(4), 161–173 (2012)CrossRefGoogle Scholar
  50. 50.
    Sourceforge. USARSim. [Online]. http://sourceforge.net/projects/usarsim/
  51. 51.
    Epic Games Inc. UDK. [Online]. http://www.unrealengine.com/udk
  52. 52.
    Lewis, M., Wang, J., Hughes, S.: USARSim: simulation for the study of human-robot interaction. J. Cognitive Eng. Decis. Mak. 1(1), 98–120 (2007)CrossRefGoogle Scholar
  53. 53.
    Wang, J., Lewis, M., Hughes, S., Koes, M.: Validating USARsim for use in HRI Research. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 457–461 (2005)Google Scholar
  54. 54.
    Carpin, S., Lewis, M., Wang, J., Balakirsky, S., Scrapper, C.: USARSim: a robot simulator for research and education. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1400–1405 (2007)Google Scholar
  55. 55.
    Balaguer, B., Balakirsky, S., Carpin, S., Visser, A.: Evaluating maps produced by urban search and rescue robots: lessons learned from RoboCup. J. Auton. Robot. 27(4), 449–464 (2009)CrossRefGoogle Scholar
  56. 56.
    Shiroma, N., Chiu, Y., Sato, N., Matsuno, F.: Cooperative task execution of a search and rescue mission by a multi-robot team. J. Adv. Robot. 19(3), 311–329 (2005)CrossRefGoogle Scholar
  57. 57.
    Calisi, D., Farinelli, A., Iocchi, L., Nardi, D.: Multi-objective exploration and search for autonomous rescue robots. J. Field Robot. 24(8-9), 763–777 (2007)CrossRefGoogle Scholar
  58. 58.
    Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: Proceeding of the ACM SIGCHI/SIGART Conference on Human-robot Interaction, pp 33–40 (2006)Google Scholar
  59. 59.
    Donmez, B., Pina, P.E., Cummings, M.L.: Evaluation criteria for human-automation performance metrics. In: Performance Evaluation and Benchmarking of Intelligent Systems, ch. 2, pp 21–40 (2009)Google Scholar
  60. 60.
    Tubre, T.C., Collins, J.M.: Jackson and Schuler (1985) Revisited: a meta-analysis of the relationships between role ambiguity, role conflict, and job performance. J. Manag. 26(1), 155–169 (2000)Google Scholar
  61. 61.
    Barde, M.P., Barde, P.J.: What to use to express the variability of data: Standard deviation or standard error of mean Perspect. Clin. Res. 3(3), 113–116 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Autonomous Systems and Biomechatronics Laboratory, Department of Mechanical and Industrial EngineeringUniversity of TorontoTorontoCanada
  2. 2.Department of Mechanical and Industrial EngineeringUniversity of TorontoTorontoCanada

Personalised recommendations