Abstract
Optimization is an important branch which aims to conceptualize, analyze, and solve problems of minimization or maximization of a function on a specific dataset. Several optimization algorithms are discussed in machine learning and particularly in deep learning (DL) based systems such as the Gradient Descent (GD) algorithm. Given the importance and the efficiency of the gradient descent algorithm, several research works have made it possible to optimize it and demonstrate its performance, Otherwise, regularity of a train is essential to ensure the continuity of the entire rail system. Non-regularity can spread quickly and influence the rest of the means of transport: rail, road, air, navy etc. In this paper, we perform a comparative study of different optimizations algorithms which are largely used in context of machine learning on the prediction of the regularity of trains, the data used is publicly available. The optimization algorithms studied are Momentum, Adagrad, RMSprop Adam and Adamax. In our context, the overall experimental results obtained show that RMSprop performed better compared to other optimization techniques, while Momentum represents the lowest performances to improve regularity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kaffash, S., Nguyen, A.T., Zhu, J.: Big data algorithms and applications in intelligent transportation system: a review and bibliometric analysis. Int. J. Prod. Econ. 231, 107868 (2021). https://doi.org/10.1016/j.ijpe.2020.107868
Haji, S.H., Abdulazeez, A.M.: Comparison of optimization techniques based on gradient descent algorithm: a review. PJAEE 18(4) (2021)
Abbas, Q.: Glaucoma-deep: detection of glaucoma eye disease on retinal fundus images using deep learning. Int. J. Adv. Comput. Sci. Appl. 8(6) (2017). https://doi.org/10.14569/IJACSA.2017.080606
Bottou, L.: Stochastic gradient descent tricks. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 421–436. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_25
Makari, F., Teflioudi, C., Gemulla, R., Haas, P., Sismanis, Y.: Shared-memory and shared-nothing stochastic gradient descent algorithms for matrix completion. Knowl. Inf. Syst. 42(3), 493–523. (2015). https://doi.org/10.1007/s10115-013-0718-7
Luo, Z., Chen, S., Qian, Y.: Stochastic momentum method with double acceleration for regularized empirical risk minimization. IEEE Access 7, 166551–166563 (2019). https://doi.org/10.1109/ACCESS.2019.2953288
Mukkamala, M.C., Hein, M.: Variants of RMSProp and Adagrad with logarithmic regret bounds. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR, vol. 70, p. 9 (2017). arXiv:1706.05507v2
Xu, S., Niu, D., Tao, B., Li, G.: Convolutional Neural Network Based Traffic Sign Recognition System, no. Icsai, pp. 957–961 (2018)
Kosolsombat, S., Saraubon, K.: A review of the prediction method for intelligent transport system. In: 2018 18th International Symposium on Communications and Information Technologies (ISCIT), Bangkok, pp. 237–240 (Sept 2018)https://doi.org/10.1109/ISCIT.2018.8588015
Balasubramaniam, A., Gul, M.J.J., Menon, V.G., Paul, A.: Blockchain for intelligent transport system. IETE Tech. Rev. 1–12 (Mai 2020). https://doi.org/10.1080/02564602.2020.1766385
Barcelo, J., Codina, E., Casas, J., Ferrer, J.L., Garcia, D.: Microscopic traffic simulation: a tool for the design, analysis and evaluation of intelligent transport systems. J. Intell. Robot. Syst. 41(2–3), 173–203 (Janv 2005). https://doi.org/10.1007/s10846-005-3808-2
Veres, M., Moussa, M.: Deep learning for intelligent transportation systems: a survey of emerging trends. IEEE Trans. Intell. Transp. Syst. 21(8), 3152–3168 (Aouˆt 2020). https://doi.org/10.1109/TITS.2019.2929020
Li, D., Deng, L., Cai, Z., Yao, X.: Notice of retraction: intelligent transportation system in macao based on deep self-coding learning. IEEE Trans. Ind. Inform. 14(7), 3253–3260 (2018). https://doi.org/10.1109/TII.2018.2810291
Jing, Y., Su, Y.: Passenger travel behaviour on chinese high-speed railways using machine learning based on revealed preference data. Expert Syst. 1–12. Wiley (2019). No. July 2018
Hadj-mabrouk, H.: Analysis and prediction of railway accident risks using machine learning. AIMS Electron. Electr. Eng. 4, pp. 19–46 (2020). https://doi.org/10.3934/ElectrEng.2020.1.19. No. November 2019
Hadj-mabrouk, H.: A hybrid approach for the prevention of railway accidents based on artificial intelligence. In: ICO 2018, AISC 866, pp. 383–394. Springer (2019)
Mustapha, A., Mohamed, L., Ali, K.: Comparative study of optimization techniques in deep learning: application in the ophthalmology field. Journal of Physics: Conference Series, vol. 1743, no. 1. IOP Publishing (2021). https://doi.org/10.1088/1742-6596/1743/1/012002
Hamlich, M., Bellatreche, L., Mondal, A., Ordonez, C. (eds.): SADASC 2020. CCIS, vol. 1207. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45183-7
Bera, S., Shrivastava, V.K.: Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification. Int. J. Remote Sens. 41(7), 2664–2683 (2020)
Mukkamala, M.C., Hein, M.: Variants of RMSProp and Adagrad with logarithmic regret bounds variants of RMSProp and Adagrad with logarithmic regret bounds. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70 (2017)
Kovalenko, R.: Comparative analysis of stochastic optimization algorithms for image registration. In: IV International Conference on Information Technology and Nanotechnology (ITNT-2018) (2018). No. January, 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ragala, Z., Retbi, A., Bennani, S. (2023). Overview of Gradient Descent Algorithms: Application to Railway Regularity. In: Hassanien, A.E., Snášel, V., Tang, M., Sung, TW., Chang, KC. (eds) Proceedings of the 8th International Conference on Advanced Intelligent Systems and Informatics 2022. AISI 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 152. Springer, Cham. https://doi.org/10.1007/978-3-031-20601-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-20601-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20600-9
Online ISBN: 978-3-031-20601-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)