Autonomous Agents and Multi-Agent Systems

, Volume 30, Issue 1, pp 82–111 | Cite as

Human–agent collaboration for disaster response

  • Sarvapali D. RamchurnEmail author
  • Feng Wu
  • Wenchao Jiang
  • Joel E. Fischer
  • Steve Reece
  • Stephen Roberts
  • Tom Rodden
  • Chris Greenhalgh
  • Nicholas R. Jennings


In the aftermath of major disasters, first responders are typically overwhelmed with large numbers of, spatially distributed, search and rescue tasks, each with their own requirements. Moreover, responders have to operate in highly uncertain and dynamic environments where new tasks may appear and hazards may be spreading across the disaster space. Hence, rescue missions may need to be re-planned as new information comes in, tasks are completed, or new hazards are discovered. Finding an optimal allocation of resources to complete all the tasks is a major computational challenge. In this paper, we use decision theoretic techniques to solve the task allocation problem posed by emergency response planning and then deploy our solution as part of an agent-based planning tool in real-world field trials. By so doing, we are able to study the interactional issues that arise when humans are guided by an agent. Specifically, we develop an algorithm, based on a multi-agent Markov decision process representation of the task allocation problem and show that it outperforms standard baseline solutions. We then integrate the algorithm into a planning agent that responds to requests for tasks from participants in a mixed-reality location-based game, called AtomicOrchid, that simulates disaster response settings in the real-world. We then run a number of trials of our planning agent and compare it against a purely human driven system. Our analysis of these trials show that human commanders adapt to the planning agent by taking on a more supervisory role and that, by providing humans with the flexibility of requesting plans from the agent, allows them to perform more tasks more efficiently than using purely human interactions to allocate tasks. We also discuss how such flexibility could lead to poor performance if left unchecked.


Human–agent interaction Human–agent collectives Disaster response 



This work was done as part of the EPSRC-funded ORCHID Project (EP/I011587/1). We also wish to thank Trung Dong Huynh for generating the traces of the player movements as well as Davide Zilli and Sebastian Stein for initial input on the Responder App. Finally we wish thank the anonymous reviewers for their constructive comments that helped improve the paper.


  1. 1.
    Abbott, K. R. & Sarin, S. K. (1994). Experiences with workflow management: Issues for the next generation. In Proceedings of the 1994 ACM conference on computer supported cooperative work (CSCW) (pp. 113–120).Google Scholar
  2. 2.
    Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multi-armed bandit problem. Machine Learning, 47(2–3), 235–256.zbMATHCrossRefGoogle Scholar
  3. 3.
    Bader, T., Meissner, A., & Tscherney, R. (2008). Digital map table with fovea-tablett \(\textregistered \): Smart furniture for emergency operation centers. In proceedings of the 5th international conference on information systems for crisis response and management (pp. 679–688).Google Scholar
  4. 4.
    Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995). Learning to act using real-time dynamic programming. Artificial Intelligence, 72(1), 81–138.CrossRefGoogle Scholar
  5. 5.
    Benford, S., Magerkurth, C., & Ljungstrand, P. (2005). Bridging the physical and digital in pervasive gaming. Communications of the ACM, 48(3), 54.CrossRefGoogle Scholar
  6. 6.
    Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002). The complexity of decentralized control of markov decision processes. Mathematics of Operations Research, 27(4), 819–840.zbMATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Boutilier, C. (1996). Planning, learning and coordination in multi-agent decision processes. Proceedings of TARK, 1996, 195–210.Google Scholar
  8. 8.
    Boutilier, C., Dearden, R., & Goldszmidt, M. (2000). Stochastic dynamic programming with factored representations. Artificial Intelligence, 121(1), 49–107.zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Bowers, J., Button, G., & Sharrock, W. (1994). Workflow from within and without: Technology and cooperative work on the print industry shopfloor introduction: Workflow systems and work practice. In Fourth European conference on computer-supported cooperative work (pp. 51–66).Google Scholar
  10. 10.
    Bradshaw, J. M., Feltovich, P., & Johnson, M. (2011). Human–agent interaction. In G. Boy (Ed.), Handbook of human–machine interaction (Chap. 13) (pp. 293–302). Surrey: Ashgate.Google Scholar
  11. 11.
    Brown, B., Reeves, S., & Sherwood, S. (2011). Into the wild: Challenges and opportunities for field trial methods. In Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’11 (pp. 1657–1666). New York, NY: ACM.Google Scholar
  12. 12.
    Chapman, A., Micillo, R. A., Kota, R., & Jennings, N. R. (2009). Decentralised dynamic task allocation: A practical game-theoretic approach. Proceedings of AAMAS, 2009, 915–922.Google Scholar
  13. 13.
    Chen, R., Sharman, R., Rao, H. R., & Upadhyaya, S. J. (2005). Design principles of coordinated multi-incident emergency response systems. Simulation, 3495, 177–202.Google Scholar
  14. 14.
    Convertino, G., Mentis, H. M., Slavkovic, A., Rosson, M. B., & Carroll, J. M. (2011). Supporting common ground and awareness in emergency management planning. ACM Transactions on Computer–Human Interaction, 18(4), 1–34.CrossRefGoogle Scholar
  15. 15.
    Cooke, G. J. N., & Harry, B. B. P. (2006). Distributed mission environments: Effects of geographic distribution on team cognition, process, and performance. Towards a science of distributed learning and training. Washington, DC: American Psychological Association.Google Scholar
  16. 16.
    Crabtree, A., Benford, S., Greenhalgh, C., Tennent, P., Chalmers, M., & Brown, B. (2006). Supporting ethnographic studies of ubiquitous computing in the wild. In Proceedings of the 6th ACM conference on designing interactive systems—DIS ’06 (p. 60). New York, NY: ACM.Google Scholar
  17. 17.
    Drury, J., Cocking, C., & Reicher, S. (2009). Everyone for themselves? A comparative study of crowd solidarity among emergency survivors. The British Journal of Social Psychology/The British Psychological Society, 48(Pt 3), 487–506.CrossRefGoogle Scholar
  18. 18.
    Fischer, J. E., Jiang, W., Kerne, A., Greenhalgh, C., Ramchurn, S. D., Reece, S., Pantidi, N., & Rodden, T. (2014). Supporting team coordination on the ground: Requirements from a mixed reality game. In COOP 2014-proceedings of the 11th international conference on the design of cooperative systems, 27–30 May 2014, Nice (France) (pp. 49–67). Berlin: Springer.Google Scholar
  19. 19.
    Fischer, J. E., Reeves, S., Rodden, T., Reece, S., Ramchurn, S. D., & Jones, D. (2015). Building a birds eye view: Collaborative work in disaster response. In Proceedings of the SIGCHI conference on human computer interaction (CHI 2015)—to appear.Google Scholar
  20. 20.
    Guestrin, C., Koller, D., & Parr, R. (2001). Multiagent planning with factored MDPS. NIPS, 1, 1523–1530.Google Scholar
  21. 21.
    Guestrin, C., Koller, D., Parr, R., & Venkataraman, S. (2003). Efficient solution algorithms for factored MDPS. Journal of Artificial Intelligence Research, 19, 399–468.zbMATHMathSciNetGoogle Scholar
  22. 22.
    Hawe, G. I., Coates, G., Wilson, D. T., & Crouch, R. S. (2012). Agent-based simulation for large-scale emergency response. ACM Computing Surveys, 45(1), 1–51.CrossRefGoogle Scholar
  23. 23.
    Initiative, H. H. et al. (2010). Disaster relief 2.0: The future of information sharing in humanitarian emergencies. In Disaster Relief 2.0: The future of information sharing in humanitarian emergencies. HHI; United Nations Foundation, OCHA; The Vodafone Foundation.Google Scholar
  24. 24.
    Jennings, N. R., Moreau, L., Nicholson, D., Ramchurn, S. D., Roberts, S. J., Rodden, T., et al. (2014). On human–agent collectives. Communications of the ACM, 57(12), 80–88.CrossRefGoogle Scholar
  25. 25.
    Jiang, W., Fischer, J. E., Greenhalgh, C., Ramchurn, S. D., Wu, F., Jennings, N. R., Rodden, T. (2014). Social implications of agent-based planning support for human teams. In International conference on collaboration technologies and systems (pp, 310–317).Google Scholar
  26. 26.
    Khan, M. A., Turgut, D., & Bölöni, L. (2011). Optimizing coalition formation for tasks with dynamically evolving rewards and nondeterministic action effects. Journal of Autonomous Agents and Multi-agent Systems, 22(3), 415–438.CrossRefGoogle Scholar
  27. 27.
    Kitano, H., & Tadokoro, S. (2001). Robocup rescue: A grand challenge for multiagent and intelligent systems. AI Magazine, 22(1), 39–52.Google Scholar
  28. 28.
    Kleiner, A., Farinelli, A., Ramchurn, S., Shi, B., Mafioletti, F., & Refatto, R. (2013). Rmasbench: A benchmarking system for multi-agent coordination in urban search and rescue. In International conference on autonomous agents and multi-agent systems (AAMAS 2013).Google Scholar
  29. 29.
    Kocsis, L., & Szepesvári, C. (2006). Bandit based Monte-Carlo planning. Proceedings of ECML, 2006, 282–293.Google Scholar
  30. 30.
    Koes, M., Nourbakhsh, I., & Sycara, K. (2006). Constraint optimization coordination architecture for search and rescue robotics. In Proceedings of IEEE international conference on robotics and automation (pp. 3977–3982). IEEE.Google Scholar
  31. 31.
    Koller, D. & Parr, R. (2000). Policy iteration for factored MDPS. In Proceedings of the sixteenth conference on Uncertainty in artificial intelligence (pp. 326–334). San Francisco, CA: Morgan Kaufmann Publishers Inc.Google Scholar
  32. 32.
    Lee, Y. M., Ghosh, S., & Ettl, M. (2009, Dec). Simulating distribution of emergency relief supplies for disaster response operations. In Proceedings of the 2009 winter simulation conference (WSC) (pp. 2797–2808). IEEE.Google Scholar
  33. 33.
    Lenox, T. L., Payne, T., Hahn, S., Lewis, M., & Sycara, K. (2000). Agent-based aiding for individual and team planning tasks. Proceedings of the human factors and Ergonomics Society Annual Meeting, 44(1), 65–68.CrossRefGoogle Scholar
  34. 34.
    Malone, T. W. & Crowston, K. (1990). What is coordination theory and how can it help design cooperative work systems? In Proceedings of the 1990 ACM conference on computer-supported cooperative work—CSCW ’90 (pp. 357–370). New York, NY: ACM.Google Scholar
  35. 35.
    Mausam, & Kolobov, A. (2012). Planning with markov decision processes: An AI perspective. Synthesis Lectures on AI and Machine Learning, 6(1), 1–210.CrossRefGoogle Scholar
  36. 36.
    Monares, A., Ochoa, S. F., Pino, J. A., Herskovic, V., Rodriguez-Covili, J., & Neyem, A. (2011). Mobile computing in urban emergency situations: Improving the support to firefighters in the field. Expert Systems with Applications, 38(2), 1255–1267.CrossRefGoogle Scholar
  37. 37.
    Moran, S., Pantidi, N., Bachour, K., Fischer, J. E., Flintham, M., Rodden, T., Evans, S., & Johnson, S. (2013). Team reactions to voiced agent instructions in a pervasive game. In Proceedings of the 2013 international conference on Intelligent user interfaces—IUI ’13 (p. 371).Google Scholar
  38. 38.
    Murthy, S., Akkiraju, R., Rachlin, J., & Wu, F. (1997). Agent-based cooperative scheduling. In Proceedings of AAAI workshop on constraints and agents (pp. 112–117).Google Scholar
  39. 39.
    Musliner, D. J., Durfee, E. H., Wu, J., Dolgov, D. A., Goldman, R. P., & Boddy, M. S. (2006). Coordinated plan management using multiagent MDPS. In AAAI spring symposium: Distributed plan and schedule management (pp. 73–80).Google Scholar
  40. 40.
    Nakajima, Y., Shiina, H., Yamane, S., Ishida, T., & Yamaki, H. (2007, Jan). Disaster evacuation guide: Using a massively multiagent server and gps mobile phones. In International symposium on applications and the internet, 2007. SAINT 2007 (p. 2).Google Scholar
  41. 41.
    Padilha, R. P., Gomes, J. O., & Canós, J. H. (2010). The design of collaboration support between command and operation teams during emergency response. Current, 759–763.Google Scholar
  42. 42.
    Proper, S., & Tadepalli, P. (2009). Solving multi-agent assignment Markov decision processes. Proceedings of AAMAS, 2009, 681–688.Google Scholar
  43. 43.
    Pujol-Gonzalez, M., Cerquides, J., Farinelli, A., Meseguer, P., & Rodríguez-Aguilar, J. A. (2014). Binary max-sum for multi-team task allocation in robocup rescue. In Optimisation in multi-agent systems and distributed constraint reasoning (OptMAS-DCR), Paris, France, 05/05/2014.Google Scholar
  44. 44.
    Pynadath, D. V., & Tambe, M. (2002). The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of Artificial Intelligence Research, 16, 389–423.zbMATHMathSciNetGoogle Scholar
  45. 45.
    Ramchurn, S. D., Farinelli, A., Macarthur, K. S., & Jennings, N. R. (2010). Decentralized coordination in robocup rescue. The Computer Journal, 53(9), 1447–1461.CrossRefGoogle Scholar
  46. 46.
    Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge, MA: MIT.zbMATHGoogle Scholar
  47. 47.
    Reece, S. & Roberts, S. (2010). An introduction to Gaussian processes for the Kalman filter expert. In Proceedings of International conference on information fusion (FUSION) (pp. 1–9). IEEE.Google Scholar
  48. 48.
    Reece, S., Ghosh, S., Roberts, S., Rogers, A., & Jennings, N. R. (2014). Efficient state–space inference of periodic latent force models. Journal of Machine Learning Research, 15, 2337–2397.zbMATHMathSciNetGoogle Scholar
  49. 49.
    Robinson, C. & Brown, D. (2005). First responder information flow simulation: A tool for technology assessment. In Proceedings of the winter simulation conference, 2005 (pp. 919–925). IEEE.Google Scholar
  50. 50.
    Scerri, P., Farinelli, A., Okamoto, S., & Tambe, M. (2005). Allocating tasks in extreme teams. In Proceedings of AAMAS (pp. 727–734). New York, NY: ACM.Google Scholar
  51. 51.
    Scerri, P., Pynadath, D., Johnson, L., Rosenbloom, P., Si, M., Schurr, N., & Tambe, M. (2003). A prototype infrastructure for distributed robot-agent-person teams. In Proceedings of the second international joint conference on autonomous agents and multiagent systems, AAMAS ’03 (pp. 433–440). New York, NY: ACM.Google Scholar
  52. 52.
    Scerri, P., Tambe, M., & Pynadath, D. V. (2002). Towards adjustable autonomy for the real-world. Journal of Artificial Intelligence Research, 17(1), 171–228.zbMATHMathSciNetGoogle Scholar
  53. 53.
    Schurr, N., Marecki, J., Lewis, J. P., Tambe, M., & Scerri, P. (2005). The defacto system: Training tool for incident commanders. In National conference on artificial intelligence (AAAI) (pp. 1555–1562).Google Scholar
  54. 54.
    Searle, J. (1975). A taxonomy of illocutionary acts. In K. Günderson (Ed.), Language, mind, and knowledge (Vol. 7, pp. 344–369)., Studies in the philosophy of science Minneapolis: University of Minneapolis Press.Google Scholar
  55. 55.
    Simonović, S. P. (2010). Systems approach to management of disasters. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  56. 56.
    Skinner, C. & Ramchurn, S. D. (2010). The robocup rescue simulation platform. In AAMAS (pp. 1647–1648). IFAAMAS.Google Scholar
  57. 57.
    Sukthankar, G., Sycara, K., Giampapa, J. A., & Burnett, C. (2009). Communications for agent-based human team support. In: Handbook of research on multi-agent dystems: Semantics and dynamics of organizational models (p. 285).Google Scholar
  58. 58.
    Tambe, M. (2011). Security and game theory: Algorithms, deployed systems lessons learned (1st ed.). New York, NY: Cambridge University Press.CrossRefGoogle Scholar
  59. 59.
    Toups, Z. O., Kerne, A., & Hamilton, W. A. (2011). The team coordination game: Zero-fidelity simulation abstracted from fire emergency response practice. ACM Transactions on Computer–Human Interaction, 18(4), 1–37.CrossRefGoogle Scholar
  60. 60.
    Wagner, T., Phelps, J., Guralnik, V., & VanRiper, R. (2004). An application view of coordinators: Coordination managers for first responders. In AAAI.Google Scholar
  61. 61.
    Wirz, M., Roggen, D., & Tröster, G. (2010). User acceptance study of a mobile system for assistance during emergency situations at large-scale events. In The 3rd international conference on human-centric computing.Google Scholar

Copyright information

© The Author(s) 2015

Authors and Affiliations

  • Sarvapali D. Ramchurn
    • 1
    Email author
  • Feng Wu
    • 1
  • Wenchao Jiang
    • 2
  • Joel E. Fischer
    • 2
  • Steve Reece
    • 3
  • Stephen Roberts
    • 3
  • Tom Rodden
    • 2
  • Chris Greenhalgh
    • 2
  • Nicholas R. Jennings
    • 1
  1. 1.Department of Electronics and Computer ScienceUniversity of SouthamptonSouthamptonUK
  2. 2.Mixed Reality LabUniversity of NottinghamNottinghamUK
  3. 3.Pattern Recognition GroupUniversity of OxfordOxfordUK

Personalised recommendations