Autonomous Robots

, Volume 43, Issue 6, pp 1375–1391 | Cite as

A multiagent framework for learning dynamic traffic management strategies

  • Jen Jen ChungEmail author
  • Carrie Rebhuhn
  • Connor Yates
  • Geoffrey A. Hollinger
  • Kagan Tumer


There is strong commercial interest in the use of large scale automated transport robots in industrial settings (e.g. warehouse robots) and we are beginning to see new applications extending these systems into our urban environments in the form of autonomous cars and package delivery drones. This new technology comes with new risks—increasing traffic congestion and concerns over safety; it also comes with new opportunities—massively distributed information and communication systems. In this paper, we present a method that leverages the distributed nature of the autonomous traffic to provide improved traffic throughput while maintaining strict capacity constraints across the network. Our proposed multiagent-based dynamic traffic management strategy borrows concepts from both air traffic control and highway metering lights. We introduce controller agents whose actions are to adjust the robots’ perceived “costs” of traveling across different parts of the traffic network. This approach allows each robot the flexibility of using its own (potentially proprietary) navigation algorithm, while still being bound by the “rules of the road.” The control policies of the agents are defined as neural networks whose weights are learned via cooperative coevolution across the entire traffic management team. Results in a real world road network and a simulated warehouse domain demonstrate that our multiagent traffic management system provides substantial improvements to overall traffic throughput in terms of number of successful trips in a fixed amount of time, as well as faster average traversal times.


Multiagent systems Traffic management Learning for coordination 



  1. Agogino, A. K., & Tumer, K. (2004). Efficient evaluation functions for multi-rover systems. In Genetic and evolutionary computation conference (pp. 1–11).Google Scholar
  2. Agogino, A. K., & Tumer, K. (2012). A multiagent approach to managing air traffic flow. Autonomous Agents and Multi-Agent Systems, 24(1), 1–25.CrossRefGoogle Scholar
  3. Arnott, R., De Palma, A., & Lindsey, R. (1993). A structural model of peak-period congestion: A traffic bottleneck with elastic demand. The American Economic Review, 83, 161–179.Google Scholar
  4. Arnott, R., & Kraus, M. (1998). When are anonymous congestion charges consistent with marginal cost pricing? Journal of Public Economics, 67(1), 45–64.CrossRefGoogle Scholar
  5. Aubert, M. C., Üzümcü, S. C., Hutchins, A. R., & Cummings, M. L. (2015). Toward the development of a low-altitude air traffic control paradigm for networks of small, autonomous unmanned aerial vehicles. In AIAA infotech @ aerospace (pp. 1110–1117).Google Scholar
  6. Baskar, L. D., De Schutter, B., Hellendoorn, J., & Papp, Z. (2011). Traffic control and intelligent vehicle highway systems: A survey. IET Intelligent Transport Systems, 5(1), 38–52.CrossRefGoogle Scholar
  7. Bellemans, T., De Schutter, B., & De Moor, B. (2006). Model predictive control for ramp metering of motorway traffic: A case study. Control Engineering Practice, 14(7), 757–767.CrossRefGoogle Scholar
  8. Beria, P., Ramella, F., & Laurino, A. (2015). Motorways economic regulation: A worldwide survey. Transport Policy, 41, 23–32.CrossRefGoogle Scholar
  9. Bongiorno, C., Gurtner, G., Lillo, F., VAlori, L., Ducci, M., Monechi, B., & Pozzi, S. (2013). An agent based model of air traffic management. In Proceedings of the 3rd SESAR innovation days. Stockholm.Google Scholar
  10. Bullo, F., Frazzoli, E., Pavone, M., Savla, K., & Smith, S. L. (2011). Dynamic vehicle routing for robotic systems. Proceedings of the IEEE, 99(9), 1482–1504.CrossRefGoogle Scholar
  11. Carey, M., & Srinivasan, A. (1993). Externalities, average and marginal costs, and tolls on congested networks with time-varying flows. Operations Research, 41(1), 217–231.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Chung, J. J. (2018). Git repository. Accessed 21 February 2018.
  13. Digani, V., Sabattini, L., & Secchi, C. (2016). A probabilistic eulerian traffic model for the coordination of multiple AGVs in automatic warehouses. IEEE Robotics and Automation Letters, 1(1), 26–32.CrossRefGoogle Scholar
  14. Digani, V., Sabattini, L., Secchi, C., & Fantuzzi, C. (2015). Ensemble coordination approach in multi-AGV systems applied to industrial warehouses. IEEE Transactions on Automation Science and Engineering, 12(3), 922–934.CrossRefGoogle Scholar
  15. Emami, H., & Derakhshan, F. (2012). An overview on conflict detection and resolution methods in air traffic management using multi agent systems. In 16th CSI international symposium on artificial intelligence and signal processing (AISP) (pp. 293–298). IEEE.Google Scholar
  16. Ficici, S. G., Melnik, O., & Pollack, J. B. (2005). A game-theoretic and dynamical-systems analysis of selection methods in coevolution. IEEE Transactions on Evolutionary Computation, 9(6), 580–602.CrossRefGoogle Scholar
  17. Gawrilow, E., Köhler, E., Möhring, R. H., & Stenzel, B. (2008). Dynamic routing of automated guided vehicles in real-time. In H.-J. Krebs & W. Jäger (Eds.), Mathematics—Key technology for the future (pp. 165–177). Berlin: Springer. (Chapter 5).CrossRefGoogle Scholar
  18. Hegyi, A., De Schutter, B., & Hellendoorn, H. (2005a). Model predictive control for optimal coordination of ramp metering and variable speed limits. Transportation Research Part C: Emerging Technologies, 13(3), 185–209.CrossRefGoogle Scholar
  19. Hegyi, A., De Schutter, B., & Hellendoorn, J. (2005b). Optimal coordination of variable speed limits to suppress shock waves. IEEE Transactions on Intelligent Transportation Systems, 6(1), 102–112.CrossRefGoogle Scholar
  20. Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366.CrossRefzbMATHGoogle Scholar
  21. Huang, P.-C., Lehman, J., Mok, A. K., Miikkulainen, R., & Sentis, L. (2014). Grasping novel objects with a dexterous robotic hand through neuroevolution. In IEEE symposium on computational intelligence in control and automation (CICA) (pp. 1–8). IEEE.Google Scholar
  22. Kopardekar, P. (2015). Safely enabling low-altitude airspace operations: Unmanned aerial system traffic management (UTM). Technical Report ARC-E-DAA-TN22234, NASA, April 2015.Google Scholar
  23. Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., & Robinson, J. E. III (2016). Unmanned aircraft system traffic management (UTM) concept of operations. In AIAA aviation technology, integration, and operations conference (pp. 1–16).Google Scholar
  24. Lindsey, C. R., & Verhoef, E. T. (2000). Traffic congestion and congestion pricing. In Handbook of transport systems and traffic control (pp. 77–105).Google Scholar
  25. Land Transport Authority of Singapore (2017). Electronic road pricing (ERP). Accessed 11 October 2017.
  26. NSW Government Roads and Maritime Services (2017a). Motorway design guide: Capacity and flow analysis. Accessed: 22 December 2017.
  27. NSW Government Roads and Maritime Services (2017b). Sydney route number map. Accessed: 22 December 2017.
  28. NSW Government Roads and Maritime Services (2017c). Your guide to using the Sydney motorway network. Accessed: 22 December 2017.
  29. Open Source Robotics Foundation. (2016). \(\text{move}\_\text{ base }\). Accessed 21 February 2018.
  30. Open Source Robotics Foundation (2017). amcl. Accessed 21 February 2018.
  31. Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3), 387–434.CrossRefGoogle Scholar
  32. Papageorgiou, M., Hadj-Salem, H., & Middelham, F. (1997). ALINEA local ramp metering: Summary of field results. Transportation Research Record: Journal of the Transportation Research Board, 1603, 90–98.CrossRefGoogle Scholar
  33. Pechoucek, M., & Sislak, D. (2009). Agent-based approach to free-flight planning, control, and simulation. IEEE Intelligent Systems, 24(1), 14–17.CrossRefGoogle Scholar
  34. Pigou, A. C. (1920). The economics of welfare. London: Macmillian and Co.Google Scholar
  35. Prevot, T., Homola, J. R., Martin, L. H., Mercer, J. S., & Cabrall, C. D. (2012). Toward automated air traffic controlinvestigating a fundamental paradigm shift in human/systems interaction. International Journal of Human–Computer Interaction, 28(2), 77–98.CrossRefGoogle Scholar
  36. Qiu, L., Hsu, W.-J., Huang, S.-Y., & Wang, H. (2002). Scheduling and routing algorithms for AGVs: A survey. International Journal of Production Research, 40(3), 745–760.CrossRefzbMATHGoogle Scholar
  37. Rebhuhn, C., Skeele, R., Chung, J. J., Hollinger, G. A., & Tumer, K. (2015). Learning to trick cost-based planners into cooperative behavior. In 2015 IEEE/RSJ international conference on intelligent robots and systems (pp. 4627–4633).Google Scholar
  38. Rossi, F., Zhang, R., Hindy, Y., & Pavone, M. (2018). Routing autonomous vehicles in congested transportation networks: Structural properties and coordination algorithms. Autonomous Robots, 42, 1–16.CrossRefGoogle Scholar
  39. Stephanedes, Y. J., Kwon, E., & Chang, K. (1992). Control emulation method for evaluating and improving traffic-response ramp metering strategies. Transportation Research Record, 1360, 42–45.Google Scholar
  40. Taghaboni-Dutta, F., & Tanchoco, J. M. A. (1995). Comparison of dynamic routeing techniques for automated guided vehicle system. International Journal of Production Research, 33(10), 2653–2669.CrossRefzbMATHGoogle Scholar
  41. Tomlin, C., Pappas, G. J., & Sastry, S. (1998). Conflict resolution for air traffic management: A study in multiagent hybrid systems. IEEE Transactions on Automatic Control, 43(4), 509–521.MathSciNetCrossRefzbMATHGoogle Scholar
  42. Yliniemi, L., Agogino, A. K., & Tumer, K. (2014). Evolutionary agent-based simulation of the introduction of new technologies in air traffic management. In Proceedings of the 2014 annual conference on genetic and evolutionary computation (pp. 1215–1222).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Jen Jen Chung
    • 1
    Email author
  • Carrie Rebhuhn
    • 2
  • Connor Yates
    • 3
  • Geoffrey A. Hollinger
    • 3
  • Kagan Tumer
    • 3
  1. 1.Autonomous Systems LabETH ZürichZürichSwitzerland
  2. 2.The MITRE CorporationMcLeanUSA
  3. 3.School of Mechanical, Industrial and Manufacturing EngineeringOregon State UniversityCorvallisUSA

Personalised recommendations