Abstract
There is strong commercial interest in the use of large scale automated transport robots in industrial settings (e.g. warehouse robots) and we are beginning to see new applications extending these systems into our urban environments in the form of autonomous cars and package delivery drones. This new technology comes with new risks—increasing traffic congestion and concerns over safety; it also comes with new opportunities—massively distributed information and communication systems. In this paper, we present a method that leverages the distributed nature of the autonomous traffic to provide improved traffic throughput while maintaining strict capacity constraints across the network. Our proposed multiagent-based dynamic traffic management strategy borrows concepts from both air traffic control and highway metering lights. We introduce controller agents whose actions are to adjust the robots’ perceived “costs” of traveling across different parts of the traffic network. This approach allows each robot the flexibility of using its own (potentially proprietary) navigation algorithm, while still being bound by the “rules of the road.” The control policies of the agents are defined as neural networks whose weights are learned via cooperative coevolution across the entire traffic management team. Results in a real world road network and a simulated warehouse domain demonstrate that our multiagent traffic management system provides substantial improvements to overall traffic throughput in terms of number of successful trips in a fixed amount of time, as well as faster average traversal times.
Similar content being viewed by others
References
Agogino, A. K., & Tumer, K. (2004). Efficient evaluation functions for multi-rover systems. In Genetic and evolutionary computation conference (pp. 1–11).
Agogino, A. K., & Tumer, K. (2012). A multiagent approach to managing air traffic flow. Autonomous Agents and Multi-Agent Systems, 24(1), 1–25.
Arnott, R., De Palma, A., & Lindsey, R. (1993). A structural model of peak-period congestion: A traffic bottleneck with elastic demand. The American Economic Review, 83, 161–179.
Arnott, R., & Kraus, M. (1998). When are anonymous congestion charges consistent with marginal cost pricing? Journal of Public Economics, 67(1), 45–64.
Aubert, M. C., Üzümcü, S. C., Hutchins, A. R., & Cummings, M. L. (2015). Toward the development of a low-altitude air traffic control paradigm for networks of small, autonomous unmanned aerial vehicles. In AIAA infotech @ aerospace (pp. 1110–1117).
Baskar, L. D., De Schutter, B., Hellendoorn, J., & Papp, Z. (2011). Traffic control and intelligent vehicle highway systems: A survey. IET Intelligent Transport Systems, 5(1), 38–52.
Bellemans, T., De Schutter, B., & De Moor, B. (2006). Model predictive control for ramp metering of motorway traffic: A case study. Control Engineering Practice, 14(7), 757–767.
Beria, P., Ramella, F., & Laurino, A. (2015). Motorways economic regulation: A worldwide survey. Transport Policy, 41, 23–32.
Bongiorno, C., Gurtner, G., Lillo, F., VAlori, L., Ducci, M., Monechi, B., & Pozzi, S. (2013). An agent based model of air traffic management. In Proceedings of the 3rd SESAR innovation days. Stockholm.
Bullo, F., Frazzoli, E., Pavone, M., Savla, K., & Smith, S. L. (2011). Dynamic vehicle routing for robotic systems. Proceedings of the IEEE, 99(9), 1482–1504.
Carey, M., & Srinivasan, A. (1993). Externalities, average and marginal costs, and tolls on congested networks with time-varying flows. Operations Research, 41(1), 217–231.
Chung, J. J. (2018). Git repository. https://github.com/JenJenChung. Accessed 21 February 2018.
Digani, V., Sabattini, L., & Secchi, C. (2016). A probabilistic eulerian traffic model for the coordination of multiple AGVs in automatic warehouses. IEEE Robotics and Automation Letters, 1(1), 26–32.
Digani, V., Sabattini, L., Secchi, C., & Fantuzzi, C. (2015). Ensemble coordination approach in multi-AGV systems applied to industrial warehouses. IEEE Transactions on Automation Science and Engineering, 12(3), 922–934.
Emami, H., & Derakhshan, F. (2012). An overview on conflict detection and resolution methods in air traffic management using multi agent systems. In 16th CSI international symposium on artificial intelligence and signal processing (AISP) (pp. 293–298). IEEE.
Ficici, S. G., Melnik, O., & Pollack, J. B. (2005). A game-theoretic and dynamical-systems analysis of selection methods in coevolution. IEEE Transactions on Evolutionary Computation, 9(6), 580–602.
Gawrilow, E., Köhler, E., Möhring, R. H., & Stenzel, B. (2008). Dynamic routing of automated guided vehicles in real-time. In H.-J. Krebs & W. Jäger (Eds.), Mathematics—Key technology for the future (pp. 165–177). Berlin: Springer. (Chapter 5).
Hegyi, A., De Schutter, B., & Hellendoorn, H. (2005a). Model predictive control for optimal coordination of ramp metering and variable speed limits. Transportation Research Part C: Emerging Technologies, 13(3), 185–209.
Hegyi, A., De Schutter, B., & Hellendoorn, J. (2005b). Optimal coordination of variable speed limits to suppress shock waves. IEEE Transactions on Intelligent Transportation Systems, 6(1), 102–112.
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366.
Huang, P.-C., Lehman, J., Mok, A. K., Miikkulainen, R., & Sentis, L. (2014). Grasping novel objects with a dexterous robotic hand through neuroevolution. In IEEE symposium on computational intelligence in control and automation (CICA) (pp. 1–8). IEEE.
Kopardekar, P. (2015). Safely enabling low-altitude airspace operations: Unmanned aerial system traffic management (UTM). Technical Report ARC-E-DAA-TN22234, NASA, April 2015.
Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., & Robinson, J. E. III (2016). Unmanned aircraft system traffic management (UTM) concept of operations. In AIAA aviation technology, integration, and operations conference (pp. 1–16).
Lindsey, C. R., & Verhoef, E. T. (2000). Traffic congestion and congestion pricing. In Handbook of transport systems and traffic control (pp. 77–105).
Land Transport Authority of Singapore (2017). Electronic road pricing (ERP). https://www.lta.gov.sg/content/ltaweb/en/roads-and-motoring/managing-traffic-and-congestion/electronic-road-pricing-erp.html. Accessed 11 October 2017.
NSW Government Roads and Maritime Services (2017a). Motorway design guide: Capacity and flow analysis. http://www.rms.nsw.gov.au/business-industry/partners-suppliers/documents/motorway-design/motorway-design-guide-capacity-flow-analysis.pdf. Accessed: 22 December 2017.
NSW Government Roads and Maritime Services (2017b). Sydney route number map. http://www.rms.nsw.gov.au/documents/roads/using-roads/alpha-numeric/sydney-map.jpg. Accessed: 22 December 2017.
NSW Government Roads and Maritime Services (2017c). Your guide to using the Sydney motorway network. http://www.rms.nsw.gov.au/sydney-motorways/documents/sydney-motorways-map.pdf. Accessed: 22 December 2017.
Open Source Robotics Foundation. (2016). \(\text{move}\_\text{ base }\). http://wiki.ros.org/move_base. Accessed 21 February 2018.
Open Source Robotics Foundation (2017). amcl. http://wiki.ros.org/amcl. Accessed 21 February 2018.
Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3), 387–434.
Papageorgiou, M., Hadj-Salem, H., & Middelham, F. (1997). ALINEA local ramp metering: Summary of field results. Transportation Research Record: Journal of the Transportation Research Board, 1603, 90–98.
Pechoucek, M., & Sislak, D. (2009). Agent-based approach to free-flight planning, control, and simulation. IEEE Intelligent Systems, 24(1), 14–17.
Pigou, A. C. (1920). The economics of welfare. London: Macmillian and Co.
Prevot, T., Homola, J. R., Martin, L. H., Mercer, J. S., & Cabrall, C. D. (2012). Toward automated air traffic controlinvestigating a fundamental paradigm shift in human/systems interaction. International Journal of Human–Computer Interaction, 28(2), 77–98.
Qiu, L., Hsu, W.-J., Huang, S.-Y., & Wang, H. (2002). Scheduling and routing algorithms for AGVs: A survey. International Journal of Production Research, 40(3), 745–760.
Rebhuhn, C., Skeele, R., Chung, J. J., Hollinger, G. A., & Tumer, K. (2015). Learning to trick cost-based planners into cooperative behavior. In 2015 IEEE/RSJ international conference on intelligent robots and systems (pp. 4627–4633).
Rossi, F., Zhang, R., Hindy, Y., & Pavone, M. (2018). Routing autonomous vehicles in congested transportation networks: Structural properties and coordination algorithms. Autonomous Robots, 42, 1–16.
Stephanedes, Y. J., Kwon, E., & Chang, K. (1992). Control emulation method for evaluating and improving traffic-response ramp metering strategies. Transportation Research Record, 1360, 42–45.
Taghaboni-Dutta, F., & Tanchoco, J. M. A. (1995). Comparison of dynamic routeing techniques for automated guided vehicle system. International Journal of Production Research, 33(10), 2653–2669.
Tomlin, C., Pappas, G. J., & Sastry, S. (1998). Conflict resolution for air traffic management: A study in multiagent hybrid systems. IEEE Transactions on Automatic Control, 43(4), 509–521.
Yliniemi, L., Agogino, A. K., & Tumer, K. (2014). Evolutionary agent-based simulation of the introduction of new technologies in air traffic management. In Proceedings of the 2014 annual conference on genetic and evolutionary computation (pp. 1215–1222).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was conducted at Oregon State University and was supported by NASA Grant NNX14AI10G.
Appendix A Sydney orbital network traffic graph configuration
Appendix A Sydney orbital network traffic graph configuration
The fixed travel cost of each edge is the distance of each highway section between the defined vertices rounded to the nearest kilometer. Edges with common end vertices are taken to have the same travel costs and capacities, e.g. edges \(\left( 0,1\right) \) and \(\left( 1,0\right) \) have the same parameters and so only edge \(\left( 0,1\right) \) is listed below (Table 5).
Rights and permissions
About this article
Cite this article
Chung, J.J., Rebhuhn, C., Yates, C. et al. A multiagent framework for learning dynamic traffic management strategies. Auton Robot 43, 1375–1391 (2019). https://doi.org/10.1007/s10514-018-9800-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-018-9800-z