Skip to main content
SpringerLink
Log in

Dynamic multi-robot task allocation under uncertainty and temporal constraints

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We consider the problem of dynamically allocating tasks to multiple agents under time window constraints and task completion uncertainty. Our objective is to minimize the number of unsuccessful tasks at the end of the operation horizon. We present a multi-robot allocation algorithm that decouples the key computational challenges of sequential decision-making under uncertainty and multi-agent coordination, and addresses them in a hierarchical manner. The lower layer computes policies for individual agents using dynamic programming with tree search, and the upper layer resolves conflicts in individual plans to obtain a valid multi-agent allocation. Our algorithm, Stochastic Conflict-Based Allocation (SCoBA), is optimal in expectation and complete under some reasonable assumptions. In practice, SCoBA is computationally efficient enough to interleave planning and execution online. On the metric of successful task completion, SCoBA consistently outperforms a number of baseline methods and shows strong competitive performance against an oracle with complete lookahead. It also scales well with the number of tasks and agents. We validate our results over a wide range of simulations on two distinct domains: multi-arm conveyor belt pick-and-place and multi-drone delivery dispatch in a city.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The code is available at https://github.com/sisl/SCoBA.jl.

References

  • Ahmed, S., & Garcia, R. (2003). Dynamic capacity acquisition and assignment under uncertainty. Annals of Operations Research, 124(1–4), 267–283.

    Article  MathSciNet  Google Scholar 

  • Al-Hinai, N., & ElMekkawy, T. Y. (2011). Robust and stable flexible job shop scheduling with random machine breakdowns using a hybrid genetic algorithm. International Journal of Production Economics, 132(2), 279–291.

    Article  Google Scholar 

  • Albers, S. (1999). Better bounds for online scheduling. SIAM Journal on Computing, 29(2), 459–473.

    Article  MathSciNet  Google Scholar 

  • Alonso-Mora, J., Samaranayake, S., Wallar, A., Frazzoli, E., & Rus, D. (2017). On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment. Proceedings of the National Academy of Sciences, 114(3), 462–467.

    Article  Google Scholar 

  • Barer, M., Sharon, G., Stern, R., & Felner, A. (2014). Suboptimal variants of the conflict-based search algorithm for the multi-agent pathfinding problem. In European conference on artificial intelligence (ECAI), pp. 961–962.

  • Behrens, J. K., Lange, R., & Mansouri, M. (2019). A constraint programming approach to simultaneous task allocation and motion scheduling for industrial dual-arm manipulation tasks. In IEEE international conference on robotics and automation (ICRA), IEEE, pp. 8705–8711.

  • Bertsekas, D. P. (2005). Dynamic programming and optimal control. Athena Scientific.

    MATH  Google Scholar 

  • Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A fresh approach to numerical computing. SIAM Review, 59(1), 65–98.

    Article  MathSciNet  Google Scholar 

  • Boutilier, C. (1996). Planning, learning and coordination in multiagent decision processes. In Conference on theoretical aspects of rationality and knowledge, Morgan Kaufmann Publishers Inc., pp. 195–210.

  • Boyarski, E., Felner, A., Stern, R., Sharon, G., Tolpin, D., Betzalel, O., & Shimony, S. E. (2015). ICBS: Improved conflict-based search algorithm for multi-agent pathfinding. In International joint conference on artificial intelligence (IJCAI), pp. 740–746.

  • Burkard, R. E., Dell’Amico, M., & Martello, S. (2009). Assignment problems, SIAM.

  • Campbell, T., Johnson, L., & How, J. P. (2013). Multiagent allocation of Markov decision process tasks. In American control conference (ACC), IEEE, pp. 2356–2361.

  • Chaari, T., Chaabane, S., Aissani, N., & Trentesaux, D. (2014). Scheduling under uncertainty: Survey and research directions. In International conference on advanced logistics and transport, ICALT, pp. 229–234.

  • Che, A., Kats, V., & Levner, E. (2017). An efficient bicriteria algorithm for stable robotic flow shop scheduling. European Journal of Operational Research, 260(3), 964–971.

    Article  MathSciNet  Google Scholar 

  • Cheung, R. K., Hang, D. D., & Shi, N. (2005). A labeling method for dynamic driver-task assignment with uncertain task durations. Operations Research Letters, 33(4), 411–420.

    Article  MathSciNet  Google Scholar 

  • Choudhury, S., Gupta, J. K., Kochenderfer, M. J., Sadigh, D., & Bohg, J. (2020a). Dynamic multi-robot task allocation under uncertainty and temporal constraints. Robotics: Science and Systems Foundation.

  • Choudhury, S., Solovey, K., Kochenderfer, M. J., & Pavone, M. (2020b). Efficient large-scale multi-drone delivery using transit networks. In IEEE international conference on robotics and automation (ICRA).

  • Church, L. K., & Uzsoy, R. (1992). Analysis of periodic and event-driven rescheduling policies in dynamic shops. International Journal of Computer Integrated Manufacturing, 5(3), 153–163.

    Article  Google Scholar 

  • Coltin, B., & Veloso, M. M. (2014). Online pickup and delivery planning with transfers for mobile robots. In IEEE international conference on robotics and automation (ICRA), IEEE, pp. 5786–5791.

  • Cordeau, J., & Laporte, G. (2007). The dial-a-ride problem: Models and algorithms. Annals of Operations Research, 153(1), 29–46.

    Article  MathSciNet  Google Scholar 

  • Dertouzos, M. L., & Mok, A. K. (1989). Multiprocessor online scheduling of hard-real-time tasks. IEEE Transactions on Software Engineering, 15(12), 1497–1506.

    Article  Google Scholar 

  • Egorov, M., Sunberg, Z. N., Balaban, E., Wheeler, T. A., Gupta, J. K., & Kochenderfer, M. J. (2017). POMDPs.jl: A framework for sequential decision making under uncertainty. Journal of Machine Learning Research (JMLR), 18(26), 1–5.

    MathSciNet  Google Scholar 

  • Felner, A., Stern, R., Shimony, S. E., Boyarski, E., Goldenberg, M., Sharon, G., Sturtevant, N., Wagner, G., & Surynek, P. (2017). Search-based optimal solvers for the multi-agent pathfinding problem: Summary and challenges. In Symposium on combinatorial search.

  • Framinan, J. M., Fernandez-Viagas, V., & Perez-Gonzalez, P. (2019). Using real-time information to reschedule jobs in a flowshop with variable processing times. Computers & Industrial Engineering, 129, 113–125.

    Article  Google Scholar 

  • Garey, M. R., & Johnson, D. S. (1975). Complexity results for multiprocessor scheduling under resource constraints. SIAM Journal on Computing, 4(4), 397–411.

    Article  MathSciNet  Google Scholar 

  • Gerkey, B. P., & Mataric, M. J. (2004). A formal analysis and taxonomy of task allocation in multi-robot systems. International Journal of Robotics Research, 23(9), 939–954.

    Article  Google Scholar 

  • Gini, M. L. (2017). Multi-robot allocation of tasks with temporal and ordering constraints. In AAAI conference on artificial intelligence (AAAI), pp. 4863–4869.

  • Gombolay, M. C., Wilcox, R., & Shah, J. A. (2018). Fast scheduling of robot teams performing tasks with temporospatial constraints. IEEE Transactions on Robotics (TRO), 34(1), 220–239.

    Article  Google Scholar 

  • González-Neira, E., Montoya-Torres, J., & Barrera, D. (2017). Flow-shop scheduling problem under uncertainties: Review and trends. International Journal of Industrial Engineering Computations, 8(4), 399–426.

    Article  Google Scholar 

  • Hönig, W., Kiesel, S., Tinka, A., Durham, J., & Ayanian, N. (2018). Conflict-based search with optimal task assignment. In International conference on autonomous agents and multiagent systems (AAMAS).

  • Hyland, M., & Mahmassani, H. S. (2018). Dynamic autonomous vehicle fleet operations: Optimization-based strategies to assign AVs to immediate traveler demand requests. Transportation Research Part C: Emerging Technologies, 92, 278–297.

    Article  Google Scholar 

  • Johannsmeier, L., & Haddadin, S. (2016). A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robotics and Automation Letters, 2(1), 41–48.

    Article  Google Scholar 

  • Kartal, B., Nunes, E., Godoy, J., & Gini, M. L. (2016). Monte Carlo tree search for multi-robot task allocation. In AAAI conference on artificial intelligence (AAAI), pp. 4222–4223.

  • Kochenderfer, M. J. (2015). Decision making under uncertainty: Theory and application. MIT Press.

    Book  Google Scholar 

  • Kok, J. R., Spaan, M. T., & Vlassis, N. (2003). Multi-robot decision making using coordination graphs. International Conference on Advanced Robotics (ICAR), 3, 1124–1129.

    Google Scholar 

  • Lanctot, M., Zambaldi, V., Gruslys, A., Lazaridou, A., Tuyls, K., Pérolat, J., Silver, D., & Graepel, T. (2017). A unified game-theoretic approach to multiagent reinforcement learning. In Advances in neural information processing systems, pp. 4190–4203.

  • Lau, H. C., Sim, M., & Teo, K. M. (2003a). Vehicle routing problem with time windows and a limited number of vehicles. European Journal of Operational Research, 148(3), 559–569.

    Article  MathSciNet  Google Scholar 

  • Lau, H. C., Sim, M., & Teo, K. M. (2003b). Vehicle routing problem with time windows and a limited number of vehicles. European Journal of Operational Research, 148(3), 559–569.

    Article  MathSciNet  Google Scholar 

  • Laumond, J. P., et al. (1998). Robot motion planning and control (Vol. 229). Springer.

    Book  Google Scholar 

  • Lenstra, J. K., Kan, A. R., & Brucker, P. (1977). Complexity of machine scheduling problems. In Annals of discrete mathematics, Vol. 1, Elsevier, pp. 343–362.

  • Lerman, K., Jones, C. V., Galstyan, A., & Mataric, M. J. (2006). Analysis of dynamic task allocation in multi-robot systems. International Journal of Robotics Researh (IJRR), 25(3), 225–241.

    Article  Google Scholar 

  • Lin, X., Janak, S. L., & Floudas, C. A. (2004). A new robust optimization approach for scheduling under uncertainty: I—Bounded uncertainty. Computers & Chemical Engineering, 28(6–7), 1069–1085.

    Article  Google Scholar 

  • Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Machine learning, Elsevier, pp. 157–163.

  • Liu, L., & Shell, D. A. (2011). Assessing optimal assignment under uncertainty: An interval-based algorithm. The International Journal of Robotics Research, 30(7), 936–953.

    Article  Google Scholar 

  • Mataric, M. J., Sukhatme, G. S., & Østergaard, E. H. (2003). Multi-robot task allocation in uncertain environments. Autonomous Robots, 14(2–3), 255–263.

    Article  Google Scholar 

  • Munkres, J. (1957). Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32–38.

    Article  MathSciNet  Google Scholar 

  • Nunes, E., Manner, M. D., Mitiche, H., & Gini, M. L. (2017). A taxonomy for task allocation problems with temporal and ordering constraints. Robotics and Autonomous Systems, 90, 55–70.

    Article  Google Scholar 

  • O’onovan, R., Uzsoy, R., & McKay, K. N. (1999). Predictable scheduling of a single machine with breakdowns and sensitive jobs. International Journal of Production Research, 37(18), 4217–4233.

    Article  Google Scholar 

  • Péret, L., & Garcia, F. (2013). Online resolution techniques. In Markov decision processes in artificial intelligence, pp. 153–184.

  • Pinedo, M. (2012). Scheduling (Vol. 29). Springer.

    Book  Google Scholar 

  • Rahmani, D., & Heydari, M. (2014). Robust and stable flow shop scheduling with unexpected arrivals of new jobs and uncertain processing times. Journal of Manufacturing Systems, 33(1), 84–92.

    Article  Google Scholar 

  • Raman, V., Donzé, A., Sadigh, D., Murray, R. M., & Seshia, S. A. (2015). Reactive synthesis from signal temporal logic specifications. In International conference on hybrid systems: Computation and control, pp. 239–248.

  • Sharon, G., Stern, R., Felner, A., & Sturtevant, N. (2012). Conflict-based search for optimal multi-agent path finding. In AAAI conference on artificial intelligence (AAAI).

  • Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.

    MATH  Google Scholar 

  • Szelke, E., & Kerr, R. M. (1994). Knowledge-based reactive scheduling. Production Planning & Control, 5(2), 124–145.

    Article  Google Scholar 

  • Timotheou, S. (2010). Asset-task assignment algorithms in the presence of execution uncertainty. The Computer Journal, 54(9), 1514–1525.

    Article  Google Scholar 

  • Timotheou, S. (2011). Network flow approaches for an asset-task assignment problem with execution uncertainty. In Computer and information sciences, Springer, pp. 33–38.

  • Vodopivec, T., Samothrakis, S., & Ster, B. (2017). On Monte Carlo tree search and reinforcement learning. Journal of Artificial Intelligence Research, 60, 881–936.

    Article  MathSciNet  Google Scholar 

  • Wang, Z., & Gombolay, M. (2020). Learning scheduling policies for multi-robot coordination with graph attention networks. IEEE Robotics and Automation Letters, 5(3), 4509–4516.

    Article  Google Scholar 

  • Yan, Z., Jouandeau, N., & Chérif, A. A. (2012). Multi-robot heuristic goods transportation. In IEEE international conference on intelligent systems, pp. 409–414.

  • Zhang, C., Song, W., Cao, Z., Zhang, J., Tan, P. S., & Chi, X. (2020). Learning to dispatch for job shop scheduling via deep reinforcement learning. In Advances in neural information processing systems (NIPS), Vol. 33.

Download references

Acknowledgements

This work was supported by the Ford Motor Company, National Science Foundation Grant Number 1941722 and National Science Foundation Grant Number 1849952.

Author information

Authors and Affiliations

  1. Department of Computer Science, Stanford University, Stanford, CA, 94305, USA

    Shushman Choudhury, Jayesh K. Gupta, Mykel J. Kochenderfer, Dorsa Sadigh & Jeannette Bohg

Authors
  1. Shushman Choudhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jayesh K. Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mykel J. Kochenderfer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dorsa Sadigh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeannette Bohg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shushman Choudhury.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Robotics: Science and Systems 2020.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choudhury, S., Gupta, J.K., Kochenderfer, M.J. et al. Dynamic multi-robot task allocation under uncertainty and temporal constraints. Auton Robot 46, 231–247 (2022). https://doi.org/10.1007/s10514-021-10022-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-021-10022-9

Keywords

Search

Navigation