Abstract
The Deep Q-Network (DQN) is one of the deep reinforcement learning algorithms, which uses deep neural network structure to estimate the Q-value in Q-learning. In the previous work, we designed and implemented a DQN-based Autonomous Aerial Vehicle (AAV) testbed and proposed a Tabu List Strategy based DQN (TLS-DQN). In this paper, we consider corner environment as a new simulation scenario and carried out simulations for normal DQN and TLS-DQN for mobility control of AAV. Simulation results show that TLS-DQN performs better than normal DQN in the corner environment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Stöcker, C., Bennett, R., Nex, F., Gerke, M., Zevenbergen, J.: Review of the Current State of UAV Regulations. Remote Sens. 9(5), 1–26 (2017)
Artemenko, O., Dominic, O., Andryeyev, O., Mitschele-Thiel, A.: Energy-aware trajectory planning for the localization of mobile devices using an unmanned aerial vehicle. In: Proceedings of the 25th International Conference on Computer Communication and Networks, ICCCN-2016, pp. 1–9 (2016)
Popović, M., et al.: An informative path planning framework for UAV-based terrain monitoring. Auton. Robot. 44, 889–911 (2020)
Nguyen, H., Chen, F., Chesser, J., Rezatofighi, H., Ranasinghe, D.: LAVAPilot: lightweight UAV trajectory planner with situational awareness for embedded autonomy to track and locate radio-tags, pp. 1–8. arXiv:2007.15860 (2020)
Oda, T., Obukata, R., Ikeda, M., Barolli, L., Takizawa, M.: Design and implementation of a simulation system based on deep q-network for mobile actor node control in wireless sensor and actor networks. In: Proceedings of the 31th IEEE International Conference on Advanced Information Networking and Applications Workshops, IEEE AINA 2017, pp. 195–200 (2017)
Saito, N., Oda, T., Hirata, A., Hirota, Y., Hirota, M., Katayama, K.: Design and implementation of a DQN based AAV. In: Proceedings of the 15th International Conference on Broadband and Wireless Computing, Communication and Applications, BWCCA 2020, pp. 321–329 (2020)
Sandino, J., Vanegas, F., Maire, F., Caccetta, P., Sanderson, C., Gonzalez, F.: UAV framework for autonomous onboard navigation and people/object detection in cluttered indoor environments. Remote Sens. 12(20), 1–31 (2020)
Scherer, J., et al.: An autonomous multi-UAV system for search and rescue. In: Proceedings of the 6th ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications, DroNet 2015, pp. 33–38 (2015)
Moulton, J., et al.: An autonomous surface vehicle for long term operations. In: Proceedings of the MTS/IEEE OCEANS, pp. 1–10 (2018)
Oda, T., Ueda, C., Ozaki, R., Katayama, K.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Proceedings of the 33rd International Conference on Advanced Information Networking and Applications, AINA 2019, pp. 362–370 (2019)
Oda, T., Matsuo, K., Barolli, L., Yamada, M., Liu, Y.: Design and implementation of an IoT-based e-learning testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017)
Hirota, Y., Oda, T., Saito, N., Hirata, A., Hirota, M., Katatama, K.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Proceedings of the 15th International Conference on Broadband and Wireless Computing, Communication and Applications, BWCCA 2020, pp. 444-453 (2020)
Hayosh, D., Liu, X., Lee, K.: Woody: low-cost, open-source humanoid torso robot. In: Proceedings of the 17th International Conference on Ubiquitous Robots, ICUR 2020, pp. 247–252 (2020)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Mnih, V., et al.: Playing Atari with deep reinforcement learning, pp. 1–9. arXiv:1312.5602 (2013)
Lei, T., Ming, L.: A robot exploration strategy based on Q-learning network. In: IEEE International Conference on Real-time Computing and Robotics, IEEE RCAR-2016, pp. 57–62 (2016)
Riedmiller, M.: Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In: Proceedings of the 16th European Conference on Machine Learning, ECML-2005, pp. 317–328 (2005)
Lin, L.J.: Reinforcement Learning for Robots Using Neural Networks. Technical Report, DTIC Document (1993)
Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 2010, pp. 1–8 (2010)
Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)
Glover, F.: Tabu Search - Part I. ORSA J. Comput. 1(3), 190–206 (1989)
Takano, K., Oda, T., Kohata, M.: Design of a DSL for converting rust programming language into RTL. In: Proceedings of the 8th International Conference on Emerging Internet, Data & Web Technologies, EIDWT 2020, pp. 342–350 (2020)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, AISTATS-2010, pp. 249–256 (2010)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, AISTATS 2011, pp. 315–323 (2011)
Acknowledgements
This work was supported by JSPS KAKENHI Grant Number 20K19793.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saito, N., Oda, T., Hirata, A., Toyoshima, K., Hirota, M., Barolli, L. (2022). Simulation Results of a DQN Based AAV Testbed in Corner Environment: A Comparison Study for Normal DQN and TLS-DQN. In: Barolli, L., Yim, K., Chen, HC. (eds) Innovative Mobile and Internet Services in Ubiquitous Computing. IMIS 2021. Lecture Notes in Networks and Systems, vol 279. Springer, Cham. https://doi.org/10.1007/978-3-030-79728-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-79728-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79727-0
Online ISBN: 978-3-030-79728-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)