Abstract
The Deep Q-Network (DQN) is one of the deep reinforcement learning algorithms, which uses deep neural network structure to estimate the Q-value in Q-learning. In the previous work, we designed and implemented a DQN-based Autonomous Aerial Vehicle (AAV) testbed and proposed a Tabu List Strategy based DQN (TLS-DQN). In this paper, we propose a DQN-based AAV mobility control method. The performance evaluation results show that the proposed method can decide and reach the destination in an indoor single-path environment with a staircase.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Stöcker, C., et al.: Review of the current state of UAV regulations. Remote Sens. 9(5), 1–26 (2017)
Artemenko, O., et al.: Energy-aware trajectory planning for the localization of mobile devices using an unmanned aerial vehicle. In: Proceedings of the 25-th International Conference on Computer Communication and Networks (ICCCN-2016), pp. 1–9 (2016)
Popović, M., et al.: An informative path planning framework for UAV-based terrain monitoring. Auton. Robot. 44, 889–911 (2020)
Nguyen, H., et al.: LAVAPilot: lightweight UAV trajectory planner with situational awareness for embedded autonomy to track and locate radio-tags. arXiv:2007.15860, pp. 1–8 (2020)
Oda, T., et al.: Design and implementation of a simulation system based on deep Q-network for mobile actor node control in wireless sensor and actor networks. In: Proceedings of the 31-th IEEE International Conference on Advanced Information Networking and Applications Workshops (IEEE AINA-2017), pp. 195–200 (2017)
Oda, T., Elmazi, D., Cuka, M., Kulla, E., Ikeda, M., Barolli, L.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering three-dimensional environment. In: Barolli, L., Woungang, I., Hussain, O.K. (eds.) INCoS 2017. LNDECT, vol. 8, pp. 41–52. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-65636-6_4
Oda, T., Kulla, E., Katayama, K., Ikeda, M., Barolli, L.: A deep Q-network based simulation system for actor node mobility control in WSANs considering three-dimensional environment: a comparison study for normal and uniform distributions. In: Barolli, L., Javaid, N., Ikeda, M., Takizawa, M. (eds.) CISIS 2018. AISC, vol. 772, pp. 842–852. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93659-8_77
Sandino, J., et al.: UAV framework for autonomous onboard navigation and people/object detection in cluttered indoor environments. Remote Sens. 12(20), 1–31 (2020)
Moulton, J., et al.: An autonomous surface vehicle for long term operations. In: Proceedings of MTS/IEEE OCEANS, pp. 1–10 (2018)
Oda, T., Ueda, C., Ozaki, R., Katayama, K.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds.) WAINA 2019. AISC, vol. 927, pp. 362–370. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15035-8_34
Oda, T., et al.: Design and implementation of an IoT-based E-learning testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017)
Hirota, Y., Oda, T., Saito, N., Hirata, A., Hirota, M., Katatama, K.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Barolli, L., Takizawa, M., Enokido, T., Chen, H.-C., Matsuo, K. (eds.) BWCCA 2020. LNNS, vol. 159, pp. 444–453. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-61108-8_44
Hayosh, D., et al.: Woody: low-cost, open-source humanoid torso robot. In: Proceedings of the 17-th International Conference on Ubiquitous Robots (ICUR-2020), pp. 247–252 (2020)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv:1312.5602, pp. 1–9 (2013)
Lei, T., Ming, L.: A robot exploration strategy based on Q-learning network. In: IEEE International Conference on Real-time Computing and Robotics (IEEE RCAR-2016), pp. 57–62 (2016)
Riedmiller, M.: Neural fitted Q iteration – first experiences with a data efficient neural reinforcement learning method. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 317–328. Springer, Heidelberg (2005). https://doi.org/10.1007/11564096_32
Lin, L.J.: Reinforcement learning for robots using neural networks. In: Proceedings of Technical Report, DTIC Document (1993)
Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN-2010), pp. 1–8 (2010)
Kaelbling, L.P., et al.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)
Saito, N., et al.: A Tabu list strategy based DQN for AAV mobility in indoor single-path environment: implementation and performance evaluation. Internet Things 14, 100394 (2021)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13-th International Conference on Artificial Intelligence and Statistics (AISTATS-2010), pp. 249–256 (2010)
Glorot, X., et al.: Deep sparse rectifier neural networks. In: Proceedings of the 14-th International Conference on Artificial Intelligence and Statistics (AISTATS-2011), pp. 315–323 (2011)
Glover, F.: Tabu search - part I. ORSA J. Comput. 1(3), 190–206 (1989)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number JP20K19793 and Grant for Promotion of OUS Research Project (OUS-RP-20-3).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saito, N., Oda, T., Hirata, A., Yukawa, C., Hirota, M., Barolli, L. (2022). Performance Evaluation of a DQN-Based Autonomous Aerial Vehicle Mobility Control Method in an Indoor Single-Path Environment with a Staircase. In: Barolli, L., Kulla, E., Ikeda, M. (eds) Advances in Internet, Data & Web Technologies. EIDWT 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 118. Springer, Cham. https://doi.org/10.1007/978-3-030-95903-6_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-95903-6_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95902-9
Online ISBN: 978-3-030-95903-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)