Abstract
Teams of semi-autonomous robots can provide valuable assistance in Urban Search and Rescue (USAR) by efficiently exploring cluttered environments and searching for potential victims. Their advantage over solely teleoperated robots is that they can address the task handling and situation awareness limitations of human operators by providing some level of autonomy to the multi-robot team. Our research focuses on developing learning-based semi-autonomous controllers for rescue robot teams. In this paper, we specifically investigate the influence of the operator-to-robot ratio on the performance of our proposed MAXQ hierarchical reinforcement learning based semi-autonomous controller for USAR missions. In particular, we propose a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots. A rigorous comparative study of our learning-based semi-autonomous controller versus a fully teleoperation-based approach was conducted in a 3D simulation environment. The results, as expected, show that, for both semi-autonomous and teleoperation modes, the total scene exploration time increases as the number of robots utilized increases. However, when using the proposed learning-based semi-autonomous controller, the rate of exploration-time increase and operator-interaction effort are significantly lower, while task performance is significantly higher. Furthermore, an additional case study showed that our learning-based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Casper, J., Micire, M., Murphy, R.: Issues in intelligent robots for search and rescue. In: Proceedings SPIE Unmanned Ground Vehicle Technology II, pp 292–302 (2000)
Casper, J., Murphy, R.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans. Syst. Man Cybern. B 33(3), 367–385 (2003)
Liu, Y., Nejat, G.: Robotic urban search and rescue: a survey from the control perspective. J. Intell. Robot. Syst. 72(2), 147–165 (2013)
Wong, C., Seet, G., Sim, S.: Multiple-robot systems for USAR: key design attributes and deployment issues. Int. J. Adv. Robot. Syst. 8(1), 85–101 (2011)
Gatsoulis, Y., Virk, G.S., Dehghani-Sanij, A.A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Decis. Mak. 4(1), 69–98 (2010)
Adams, J.A.: Multiple robot/single human interaction: effects on perceived workload. Behav. Inf. Technol. 28 (2), 183–198 (2009)
de Visser, E., Parasuraman, R.: Adaptive aiding of human-robot teaming effects of imperfect automation on performance, trust, and workload. J. Cogn. Eng. Decis. Mak. 5(2), 209–231 (2011)
Chen, J.Y.C., Durlach, P.J., Sloan, J.A., Bowens, L.D.: Human-robot interaction in the context of simulated route reconnaissance missions. Mil. Psychol. 20(3), 135–149 (2008)
Breslow, L.A., Gartenberg, D., McCurry, J.M., Trafton, J.G.: Dy7namic operator overload: a model for predicting workload during supervisory control. IEEE Trans. Hum. Mach. Syst. 44(1), 30–40 (2014)
McKendrick, R., Shaw, T, de Visser, E., Sager, H., Kidwell, B., Parasuraman, R.: Team performance in networked supervisory control of unmanned air vehicles effects of automation, working memory, and communication content. Hum. Factors 56(3), 463–475 (2014)
Fincannon, T.D., Evans, A.W., Phillips, E., Jentsch, F., Keebler, J.: The influence of team size and communication modality on team effectiveness with unmanned systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 419–423 (2009)
de Visser, E., Kidwell, B., Payne, J., Lu, L., Parker, J., Brooks, N., Chabuk, T., Spriggs, S., Freedy, A., Scerri, P., Parasuraman, R.: Best of both worlds design and evaluation of an adaptive delegation interface. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 255–259 (2013)
Cummings, M.L., Bertucelli, L.F., Macbeth, J., Surana, A.: Task versus vehicle-based control paradigms in multiple unmanned vehicle supervision by a single operator. IEEE Trans. Hum. Mach. Syst. 44(3), 353–361 (2014)
Prewett, M.S., Saboe, K.N., Johnson, R.C., Coovert, M.D., Elliot, L.R.: Workload in human-robot interaction: a review of manipulations and outcomes. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 1393–1397 (2009)
Velagapudi, P., Scerri, P., Sycara, K., Wang, H., Lewis, M., Wang, J.: Scaling effects in multi-robot control. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2121–2126 (2008)
Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., Grocholsky, B., Kumar, V., Taylor, C.J., Endo, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situation awareness. J. Field Robot. 24(11), 991–1014 (2007)
Zhang, K., Xiaobo, L.: Human-robot team coordination that considers human fatigue. J. Adv. Robot. Syst. 11(6) (2014)
Olsen, D.R., Goodrich, M.A.: Metrics for evaluating human–robot interactions. In: Proceedings of the PERMIS (2003)
Gatsoulis, Y., Virk, G., Dehghani-Sanij, A.: On the measurement of situation awareness for effective human-robot interaction in teleoperated systems. J. Cogn. Eng. Dec. Mak. 4(1), 69–98 (2010)
Gao, F., Cummings, M.L., Solovey, E.T.: Modeling teamwork in supervisory control of multiple robots. IEEE Trans. Hum. Mach. Syst. 44(4), 441–453 (2014)
Lewis, M., Wang, H., Chien, S.Y.: Process and performance in human-robot teams. J. Cogn. Eng. Decis. Mak. 5(2), 186–208 (2011)
Gunn, T., Anderson, J.: Dynamic heterogeneous team formation for robotic urban search and rescue. J. Comp. Syst. Sci. 81(3), 553–567 (2015)
Rosenfeld, A., Agmon, N., Maksimov, O., Azaria, A., Kraus, S.: Intelligent agent supporting human-multi-robot team collaboration. In: Proceedings of the AAMAS Workshop ARMS (2015)
Nourjou, R., Smith, S.F., Hatayama, M., Szekely, P.: Intelligent algorithm for assignment of agents to human strategy in centralized multi-agent coordination. J. Softw. 9(10), 2586–2597 (2014)
Chen, J.Y.C., Barnes, M.: Supervisory control of multiple robots: effects of imperfect automation and individual differences. Hum. Factors: J. Hum. Factors Ergon. Soc. 54(2), 157–174 (2012)
Vilela, J., Liu, Y., Nejat, G.: Semi-autonomous exploration with robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2013)
Doroodgar, B., Liu, Y., Nejat, G.: A learning-based semi-autonomous controller for robotic exploration of unknown disaster scenes while searching for victims. IEEE Trans. Cybern. 44(12), 2719–2732 (2014)
Niroui, F., Sprenger, B., Nejat, G.: Robot exploration in unknown cluttered environments when dealing with uncertainty. In: Proceedings of the IEEE International Symposium on Robotics and Intelligent Sensors (2017)
Liu, Y., Nejat, G., Doroodgar, B.: Learning-based semi-autonomous control for robots in urban search and rescue. In: Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 1–6 (2012)
Liu, Y., Nejat, G.: Multirobot cooperative learning for semiautonomous control in urban search and rescue applications. J. Field Robot. 33(4), 512–536 (2016)
Brooks, N., Scerri, P., Sycara, K.: Asynchronous control with ATR for large robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 444–448 (2011)
Wang, H., Lewis, M., Chien, S., Velagapudi, P.: Scaling effects for synchronous vs. asynchronous video in multi-robot search. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 364–368 (2009)
Sato, N., Matsuno, F., Yamasaki, T., Kamegawa, T., Shiroma, N., Igarashi, H.: Cooperative task execution by a multiple robot team and its operators in search and rescue operations. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1083–1088 (2004)
Lewis, M., Wang, H., Chien, S.Y.: Choosing autonomy modes for multirobot search. Hum. Factors 52 (2), 225–233 (2010)
Mouaddib, A., Zilberstein, S., Beynier, A., Jeanpierre, L.: A decision-theoretic approach to cooperative control and adjustable autonomy. In: Proceedings of the European Conference on Artificial Intelligence, pp 971–972 (2010)
Côté, N., Canu, A., Bouzid, M., Mouaddib, A.: Humans-robots sliding collaboration control in complex environments with adjustable autonomy. In: Proceedings of the IEEE/WIC/ACM International Conference Web Intelligence and Intelligent Agent Technology, pp 146–153 (2012)
Wray, K.H., Pineda, L., Zilberstein, S.: Hierarchical approach to transfer of control in semi-autonomous systems. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp 517–523 (2016)
Zhang, K., Collins, E.G., Shi, D.: Centralized and distributed task allocation in multi-robot teams via a stochastic clustering auction. J. Auton. Adapt. Syst. 7(2) (2012)
Zhang, K., Collins, E.G., Barbu, A.: An efficient stochastic clustering auction for heterogenous robotic collaborative teams. J. Intell. Robot. Syst. 72(3–4), 541–558 (2013)
Zhang, K., Collins, E.G., Barbu, A.: A novel stochastic clustering auction for task allocation in multi-robot teams. In: Proceedings of the IEEE/RSJ International Conference on Intelligent and Robotic Systems, pp 3300–3307 (2010)
Zhang, K., Collins, E.G., Shi, D., Liu, X., Chuy, O.: A stochastic clustering auction (SCA) for centralized and distributed task allocation in multi-agent teams. Distrib. Auton. Robot. Syst. 8, 345–354 (2009)
Zhang, Z., Littman, M., Chen, X.: Covering number as a complexity measure for POMDP planning and learning. In: AAAI Conference on Artificial Intelligence, pp 1853–1859 (2012)
Dietterich, D.: Hierarchical reinforcement learning with MAXQ value function decomposition. J. Artif. Intell. Res. 13, 227–303 (2000)
Liu, Y., Nejat, G., Vilela, J.: Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue. In: IEEE International Symposium on Safety, Security, and Rescue Robotics (2013)
Doroodgar, B., Nejat, G.: A hierarchical reinforcement learning based control architecture for semi-autonomous rescue robots in cluttered environments. In: IEEE International Conference on Automation Science and Engineering, pp 948–953 (2010)
Chien, S., Wang, H., Lewis, M.: Effects of alarms on control of robot teams. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 434–438 (2011)
Oppermann, R.: Adaptive User Support: Ergonomic Design of Manually and Automatically Adaptable Software. CRC Press, Boca Raton (1994)
Parasuraman, R., Galster, S., Squire, P., Furukawa, H., Miller, C.: A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with RoboFlag. IEEE Trans. Cybern. 35(4), 481–493 (2005)
Clare, A., Cummings, M., How, J., Whitten, A., Toupet, O.: Operator object function guidance for a real-time unmanned vehicle scheduling algorithm. J. Aerosp. Comput. Inf. Commun. 9(4), 161–173 (2012)
Sourceforge. USARSim. [Online]. http://sourceforge.net/projects/usarsim/
Epic Games Inc. UDK. [Online]. http://www.unrealengine.com/udk
Lewis, M., Wang, J., Hughes, S.: USARSim: simulation for the study of human-robot interaction. J. Cognitive Eng. Decis. Mak. 1(1), 98–120 (2007)
Wang, J., Lewis, M., Hughes, S., Koes, M.: Validating USARsim for use in HRI Research. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 457–461 (2005)
Carpin, S., Lewis, M., Wang, J., Balakirsky, S., Scrapper, C.: USARSim: a robot simulator for research and education. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1400–1405 (2007)
Balaguer, B., Balakirsky, S., Carpin, S., Visser, A.: Evaluating maps produced by urban search and rescue robots: lessons learned from RoboCup. J. Auton. Robot. 27(4), 449–464 (2009)
Shiroma, N., Chiu, Y., Sato, N., Matsuno, F.: Cooperative task execution of a search and rescue mission by a multi-robot team. J. Adv. Robot. 19(3), 311–329 (2005)
Calisi, D., Farinelli, A., Iocchi, L., Nardi, D.: Multi-objective exploration and search for autonomous rescue robots. J. Field Robot. 24(8-9), 763–777 (2007)
Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: Proceeding of the ACM SIGCHI/SIGART Conference on Human-robot Interaction, pp 33–40 (2006)
Donmez, B., Pina, P.E., Cummings, M.L.: Evaluation criteria for human-automation performance metrics. In: Performance Evaluation and Benchmarking of Intelligent Systems, ch. 2, pp 21–40 (2009)
Tubre, T.C., Collins, J.M.: Jackson and Schuler (1985) Revisited: a meta-analysis of the relationships between role ambiguity, role conflict, and job performance. J. Manag. 26(1), 155–169 (2000)
Barde, M.P., Barde, P.J.: What to use to express the variability of data: Standard deviation or standard error of mean Perspect. Clin. Res. 3(3), 113–116 (2012)
Acknowledgments
This work was funded by the Natural Science and Engineering Research Council of Canada (NSERC) and the Canada Research Chairs (CRC) Program.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hong, A., Igharoro, O., Liu, Y. et al. Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments. J Intell Robot Syst 94, 669–686 (2019). https://doi.org/10.1007/s10846-018-0899-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-018-0899-0