Skip to main content
SpringerLink
Log in

Optimal Cislunar Architecture Design Using Monte Carlo Tree Search Methods

  • Original Article
  • Published:
The Journal of the Astronautical Sciences Aims and scope Submit manuscript

Abstract

A novel multi-objective Monte Carlo Tree Search (MO-MCTS) algorithm is developed and implemented for use in architecture design problems. This algorithm is used with two well-known problems with known solutions in order to verify its performance. It is then used in a highly nonlinear Cislunar architecture design problem with no known analytical solutions. The results of this implementation display the ability of MO-MCTS to effectively navigate the state space of mixed integer nonlinear programming problems and emphasize the versatility of MO-MCTS for designing critical Cislunar architecture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://ssd.jpl.nasa.gov/api.html.

References

  1. Duffy, L., Adams, J.: Cislunar systems architectures survey paper. In: 2022 IEEE International Systems Conference (SysCon), pp. 1–8 (2022). IEEE

  2. Holzinger, M., Chow, C., Garretson, P.: A Primer on Cislunar Space. Air Force Research Laboratory (2021)

  3. Belotti, P., Kirches, C., Leyffer, S., Linderoth, J., Luedtke, J., Mahajan, A.: Mixed-integer nonlinear optimization. Acta Numer. 22, 1–131 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Xu, M., Wang, J., Liu, S., Xu, S.: A new constellation configuration scheme for communicating architecture in cislunar space. Math. Probl. Eng. (2013). https://doi.org/10.1155/2013/864950

    Article  MathSciNet  Google Scholar 

  5. Vendl, J.K., Holzinger, M.J.: Cislunar periodic orbit analysis for persistent space object detection capability. J. Spacecr. Rocket. 58(4), 1174–1185 (2021)

    Article  Google Scholar 

  6. Lee, S.-U., Kim, J.-H., Lee, S.-P.: Communications satellite system by using Moon orbit satellite constellation. J. Astron. Space Sci. 20(4), 313–318 (2003)

    Article  Google Scholar 

  7. Frueh, C., Howell, K., DeMars, K., Bhadauria, S., Gupta, M.: Cislunar space traffic management: surveillance through Earth-Moon resonance orbits. In: 8th European Conference on Space Debris (2021)

  8. Fedeler, S., Holzinger, M., Whitacre, W.: Sensor tasking in the Cislunar regime using Monte Carlo tree search. Adv. Space Res. (2022). https://doi.org/10.1016/j.asr.2022.05.003

  9. Bai, F., Ju, X., Wang, S., Zhou, W., Liu, F.: Wind farm layout optimization using adaptive evolutionary algorithm with Monte Carlo tree search reinforcement learning. Energy Convers. Manag. 252, 115047 (2022)

    Article  Google Scholar 

  10. Rossi, L., Winands, M.H., Butenweg, C.: Monte Carlo tree search as an intelligent search tool in structural design problems. Eng. Comput. 38, 1–18 (2021)

    Google Scholar 

  11. Huang, X., Shao, Z., Yang, Y.: MIPS: instance placement for stream processing systems based on Monte Carlo tree search. arXiv e-prints arxiv:2008.00156 (2020)

  12. Hennes, D., Izzo, D.: Interplanetary trajectory planning with Monte Carlo tree search. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)

  13. Wang, W., Sebag, M.: Multi-objective Monte-Carlo tree search. In: Asian Conference on Machine Learning, pp. 507–522 (2012). PMLR

  14. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  15. Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) Machine Learning: ECML 2006. Lecture Notes in Computer Science, vol. 4212, pp 282–293. Springer, Berlin, Heidelberg (2006). https://doi.org/10.1007/11871842_29

  16. Vodopivec, T., Samothrakis, S., Ster, B.: On monte carlo tree search and reinforcement learning. J. Artif. Intell. Res. 60, 881–936 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  17. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  18. Chaslot, G., Bakkes, S., Szita, I., Spronck, P.: Monte-Carlo tree search: a new framework for game AI. In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. 4, pp. 216–217 (2008)

  19. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)

    Article  Google Scholar 

  20. Świechowski, M., Godlewski, K., Sawicki, B., Mańdziuk, J.: Monte Carlo tree search: a review of recent modifications and applications. Artif. Intell. Rev. 56, 2497–2562 (2023). https://doi.org/10.1007/s10462-022-10228-y (2021)

  21. Kocsis, L., Szepesvári, C., Willemson, J.: Improved Monte-Carlo search. Univ. Tartu, Estonia, Tech. Rep 1 (2006)

  22. Shah, D., Xie, Q., Xu, Z.: Non-asymptotic analysis of Monte Carlo tree search. In: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems, pp. 31–32 (2020)

  23. Couëtoux, A., Hoock, J.-B., Sokolovska, N., Teytaud, O., Bonnard, N.: Continuous upper confidence trees. In: Coello, C.A.C. (eds.) Learning and Intelligent Optimization. LION 2011. Lecture Notes in Computer Science, vol. 6683, pp 433–445. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25566-3_32

  24. Marler, R.T., Arora, J.S.: Survey of multi-objective optimization methods for engineering. Struct. Multidiscip. Optim. 26(6), 369–395 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  25. Blank, J., Deb, K.: Pymoo: multi-objective optimization in Python. IEEE Access 8, 89497–89509 (2020)

    Article  Google Scholar 

  26. Riquelme, N., Von Lücken, C., Baran, B.: Performance metrics in multi-objective optimization. In: 2015 Latin American Computing Conference (CLEI), pp. 1–11 (2015). IEEE

  27. Johnson, S.B., et al.: Space Exploration and Humanity: A Historical Encyclopedia, Vol. 1. ABC-CLIO (2010)

  28. Hendrickx, B.: A history of Soviet/Russian meteorological satellites. J. Br. Interplanet. Soc. 57(1), 56 (2004)

    Google Scholar 

  29. Coder, R.D., Holzinger, M.J.: Multi-objective design of optical systems for space situational awareness. Acta Astronaut. 128, 669–684 (2016)

    Article  Google Scholar 

  30. Annex, A.M., Pearson, B., Seignovert, B., Carcich, B.T., Eichhorn, H., Mapel, J.A., Von Forstner, J.L.F., McAuliffe, J., Del Rio, J.D., Berry, K.L., et al.: SpiceyPy: a Pythonic Wrapper for the SPICE Toolkit. J Open Source Softw 5(46), 2050 (2020)

    Article  Google Scholar 

  31. Greaves, J.A., Scheeres, D.J.: Observation and maneuver detection for cislunar vehicles. J. Astronaut. Sci. 68(4), 826–854 (2021)

    Article  Google Scholar 

  32. Fowler, E.E., Hurtt, S.B., Paley, D.A.: Observability metrics for space-based cislunar domain awareness. In: AAS/AIAA Astrodynamics Specialist Conference (2020)

Download references

Author information

Authors and Affiliations

  1. Smead Aerospace Engineering Science, University of Colorado Boulder, 3775 Discovery Drive, Boulder, 80303, CO, USA

    Michael Klonowski & Marcus J. Holzinger

  2. Ball Aerospace, 10 Longs Peak Dr, Broomfield, 80021, CO, USA

    Naomi Owens Fahrner

Authors
  1. Michael Klonowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus J. Holzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naomi Owens Fahrner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Klonowski.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klonowski, M., Holzinger, M.J. & Fahrner, N.O. Optimal Cislunar Architecture Design Using Monte Carlo Tree Search Methods. J Astronaut Sci 70, 17 (2023). https://doi.org/10.1007/s40295-023-00383-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40295-023-00383-x

Keywords

Search

Navigation