Skip to main content
SpringerLink
Log in

A multiagent framework for learning dynamic traffic management strategies

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

There is strong commercial interest in the use of large scale automated transport robots in industrial settings (e.g. warehouse robots) and we are beginning to see new applications extending these systems into our urban environments in the form of autonomous cars and package delivery drones. This new technology comes with new risks—increasing traffic congestion and concerns over safety; it also comes with new opportunities—massively distributed information and communication systems. In this paper, we present a method that leverages the distributed nature of the autonomous traffic to provide improved traffic throughput while maintaining strict capacity constraints across the network. Our proposed multiagent-based dynamic traffic management strategy borrows concepts from both air traffic control and highway metering lights. We introduce controller agents whose actions are to adjust the robots’ perceived “costs” of traveling across different parts of the traffic network. This approach allows each robot the flexibility of using its own (potentially proprietary) navigation algorithm, while still being bound by the “rules of the road.” The control policies of the agents are defined as neural networks whose weights are learned via cooperative coevolution across the entire traffic management team. Results in a real world road network and a simulated warehouse domain demonstrate that our multiagent traffic management system provides substantial improvements to overall traffic throughput in terms of number of successful trips in a fixed amount of time, as well as faster average traversal times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Agogino, A. K., & Tumer, K. (2004). Efficient evaluation functions for multi-rover systems. In Genetic and evolutionary computation conference (pp. 1–11).

  • Agogino, A. K., & Tumer, K. (2012). A multiagent approach to managing air traffic flow. Autonomous Agents and Multi-Agent Systems, 24(1), 1–25.

    Article  Google Scholar 

  • Arnott, R., De Palma, A., & Lindsey, R. (1993). A structural model of peak-period congestion: A traffic bottleneck with elastic demand. The American Economic Review, 83, 161–179.

    Google Scholar 

  • Arnott, R., & Kraus, M. (1998). When are anonymous congestion charges consistent with marginal cost pricing? Journal of Public Economics, 67(1), 45–64.

    Article  Google Scholar 

  • Aubert, M. C., Üzümcü, S. C., Hutchins, A. R., & Cummings, M. L. (2015). Toward the development of a low-altitude air traffic control paradigm for networks of small, autonomous unmanned aerial vehicles. In AIAA infotech @ aerospace (pp. 1110–1117).

  • Baskar, L. D., De Schutter, B., Hellendoorn, J., & Papp, Z. (2011). Traffic control and intelligent vehicle highway systems: A survey. IET Intelligent Transport Systems, 5(1), 38–52.

    Article  Google Scholar 

  • Bellemans, T., De Schutter, B., & De Moor, B. (2006). Model predictive control for ramp metering of motorway traffic: A case study. Control Engineering Practice, 14(7), 757–767.

    Article  Google Scholar 

  • Beria, P., Ramella, F., & Laurino, A. (2015). Motorways economic regulation: A worldwide survey. Transport Policy, 41, 23–32.

    Article  Google Scholar 

  • Bongiorno, C., Gurtner, G., Lillo, F., VAlori, L., Ducci, M., Monechi, B., & Pozzi, S. (2013). An agent based model of air traffic management. In Proceedings of the 3rd SESAR innovation days. Stockholm.

  • Bullo, F., Frazzoli, E., Pavone, M., Savla, K., & Smith, S. L. (2011). Dynamic vehicle routing for robotic systems. Proceedings of the IEEE, 99(9), 1482–1504.

    Article  Google Scholar 

  • Carey, M., & Srinivasan, A. (1993). Externalities, average and marginal costs, and tolls on congested networks with time-varying flows. Operations Research, 41(1), 217–231.

    Article  MathSciNet  MATH  Google Scholar 

  • Chung, J. J. (2018). Git repository. https://github.com/JenJenChung. Accessed 21 February 2018.

  • Digani, V., Sabattini, L., & Secchi, C. (2016). A probabilistic eulerian traffic model for the coordination of multiple AGVs in automatic warehouses. IEEE Robotics and Automation Letters, 1(1), 26–32.

    Article  Google Scholar 

  • Digani, V., Sabattini, L., Secchi, C., & Fantuzzi, C. (2015). Ensemble coordination approach in multi-AGV systems applied to industrial warehouses. IEEE Transactions on Automation Science and Engineering, 12(3), 922–934.

    Article  Google Scholar 

  • Emami, H., & Derakhshan, F. (2012). An overview on conflict detection and resolution methods in air traffic management using multi agent systems. In 16th CSI international symposium on artificial intelligence and signal processing (AISP) (pp. 293–298). IEEE.

  • Ficici, S. G., Melnik, O., & Pollack, J. B. (2005). A game-theoretic and dynamical-systems analysis of selection methods in coevolution. IEEE Transactions on Evolutionary Computation, 9(6), 580–602.

    Article  Google Scholar 

  • Gawrilow, E., Köhler, E., Möhring, R. H., & Stenzel, B. (2008). Dynamic routing of automated guided vehicles in real-time. In H.-J. Krebs & W. Jäger (Eds.), Mathematics—Key technology for the future (pp. 165–177). Berlin: Springer. (Chapter 5).

    Chapter  Google Scholar 

  • Hegyi, A., De Schutter, B., & Hellendoorn, H. (2005a). Model predictive control for optimal coordination of ramp metering and variable speed limits. Transportation Research Part C: Emerging Technologies, 13(3), 185–209.

    Article  Google Scholar 

  • Hegyi, A., De Schutter, B., & Hellendoorn, J. (2005b). Optimal coordination of variable speed limits to suppress shock waves. IEEE Transactions on Intelligent Transportation Systems, 6(1), 102–112.

    Article  Google Scholar 

  • Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366.

    Article  MATH  Google Scholar 

  • Huang, P.-C., Lehman, J., Mok, A. K., Miikkulainen, R., & Sentis, L. (2014). Grasping novel objects with a dexterous robotic hand through neuroevolution. In IEEE symposium on computational intelligence in control and automation (CICA) (pp. 1–8). IEEE.

  • Kopardekar, P. (2015). Safely enabling low-altitude airspace operations: Unmanned aerial system traffic management (UTM). Technical Report ARC-E-DAA-TN22234, NASA, April 2015.

  • Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., & Robinson, J. E. III (2016). Unmanned aircraft system traffic management (UTM) concept of operations. In AIAA aviation technology, integration, and operations conference (pp. 1–16).

  • Lindsey, C. R., & Verhoef, E. T. (2000). Traffic congestion and congestion pricing. In Handbook of transport systems and traffic control (pp. 77–105).

  • Land Transport Authority of Singapore (2017). Electronic road pricing (ERP). https://www.lta.gov.sg/content/ltaweb/en/roads-and-motoring/managing-traffic-and-congestion/electronic-road-pricing-erp.html. Accessed 11 October 2017.

  • NSW Government Roads and Maritime Services (2017a). Motorway design guide: Capacity and flow analysis. http://www.rms.nsw.gov.au/business-industry/partners-suppliers/documents/motorway-design/motorway-design-guide-capacity-flow-analysis.pdf. Accessed: 22 December 2017.

  • NSW Government Roads and Maritime Services (2017b). Sydney route number map. http://www.rms.nsw.gov.au/documents/roads/using-roads/alpha-numeric/sydney-map.jpg. Accessed: 22 December 2017.

  • NSW Government Roads and Maritime Services (2017c). Your guide to using the Sydney motorway network. http://www.rms.nsw.gov.au/sydney-motorways/documents/sydney-motorways-map.pdf. Accessed: 22 December 2017.

  • Open Source Robotics Foundation. (2016). \(\text{move}\_\text{ base }\). http://wiki.ros.org/move_base. Accessed 21 February 2018.

  • Open Source Robotics Foundation (2017). amcl. http://wiki.ros.org/amcl. Accessed 21 February 2018.

  • Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3), 387–434.

    Article  Google Scholar 

  • Papageorgiou, M., Hadj-Salem, H., & Middelham, F. (1997). ALINEA local ramp metering: Summary of field results. Transportation Research Record: Journal of the Transportation Research Board, 1603, 90–98.

    Article  Google Scholar 

  • Pechoucek, M., & Sislak, D. (2009). Agent-based approach to free-flight planning, control, and simulation. IEEE Intelligent Systems, 24(1), 14–17.

    Article  Google Scholar 

  • Pigou, A. C. (1920). The economics of welfare. London: Macmillian and Co.

    Google Scholar 

  • Prevot, T., Homola, J. R., Martin, L. H., Mercer, J. S., & Cabrall, C. D. (2012). Toward automated air traffic controlinvestigating a fundamental paradigm shift in human/systems interaction. International Journal of Human–Computer Interaction, 28(2), 77–98.

    Article  Google Scholar 

  • Qiu, L., Hsu, W.-J., Huang, S.-Y., & Wang, H. (2002). Scheduling and routing algorithms for AGVs: A survey. International Journal of Production Research, 40(3), 745–760.

    Article  MATH  Google Scholar 

  • Rebhuhn, C., Skeele, R., Chung, J. J., Hollinger, G. A., & Tumer, K. (2015). Learning to trick cost-based planners into cooperative behavior. In 2015 IEEE/RSJ international conference on intelligent robots and systems (pp. 4627–4633).

  • Rossi, F., Zhang, R., Hindy, Y., & Pavone, M. (2018). Routing autonomous vehicles in congested transportation networks: Structural properties and coordination algorithms. Autonomous Robots, 42, 1–16.

    Article  Google Scholar 

  • Stephanedes, Y. J., Kwon, E., & Chang, K. (1992). Control emulation method for evaluating and improving traffic-response ramp metering strategies. Transportation Research Record, 1360, 42–45.

    Google Scholar 

  • Taghaboni-Dutta, F., & Tanchoco, J. M. A. (1995). Comparison of dynamic routeing techniques for automated guided vehicle system. International Journal of Production Research, 33(10), 2653–2669.

    Article  MATH  Google Scholar 

  • Tomlin, C., Pappas, G. J., & Sastry, S. (1998). Conflict resolution for air traffic management: A study in multiagent hybrid systems. IEEE Transactions on Automatic Control, 43(4), 509–521.

    Article  MathSciNet  MATH  Google Scholar 

  • Yliniemi, L., Agogino, A. K., & Tumer, K. (2014). Evolutionary agent-based simulation of the introduction of new technologies in air traffic management. In Proceedings of the 2014 annual conference on genetic and evolutionary computation (pp. 1215–1222).

Download references

Author information

Authors and Affiliations

  1. Autonomous Systems Lab, ETH Zürich, Zürich, Switzerland

    Jen Jen Chung

  2. The MITRE Corporation, McLean, VA, USA

    Carrie Rebhuhn

  3. School of Mechanical, Industrial and Manufacturing Engineering, Oregon State University, Corvallis, OR, USA

    Connor Yates, Geoffrey A. Hollinger & Kagan Tumer

Authors
  1. Jen Jen Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carrie Rebhuhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Connor Yates
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Geoffrey A. Hollinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kagan Tumer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jen Jen Chung.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was conducted at Oregon State University and was supported by NASA Grant NNX14AI10G.

Appendix A Sydney orbital network traffic graph configuration

Appendix A Sydney orbital network traffic graph configuration

The fixed travel cost of each edge is the distance of each highway section between the defined vertices rounded to the nearest kilometer. Edges with common end vertices are taken to have the same travel costs and capacities, e.g. edges \(\left( 0,1\right) \) and \(\left( 1,0\right) \) have the same parameters and so only edge \(\left( 0,1\right) \) is listed below (Table 5).

Table 5 Sydney orbital network traffic graph configuration
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chung, J.J., Rebhuhn, C., Yates, C. et al. A multiagent framework for learning dynamic traffic management strategies. Auton Robot 43, 1375–1391 (2019). https://doi.org/10.1007/s10514-018-9800-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9800-z

Keywords

Search

Navigation