Abstract
Precise data input classification and identification are essential for developing high-quality intelligent systems, including defending systems against cyberattacks and guaranteeing data protection. The Internet of Things (IoT), in particular, offers specific problems for cybersecurity not because of the volumes of data generated in a short time but also for the diversity of data sources. Thus, creating an accurate identification system is crucial to preserve data integrity and security. It has been demonstrated that convolutional neural networks (CNNs) perform well in developing data-driven applications, including cybersecurity. However, hyperparameters play a significant role in their prediction quality which may increase/decrease the false alarm rates. CNNs’ performance may be improved by the activation and optimization functions utilized using nonlinear expression and reducing loss functions, enabling them to be more precisely fit data. This study evaluates the classification accuracy of CNNs while classifying inputs using various combinations of activation functions and optimization strategies. The performance of the CNN model with ReLU and PReLU activation functions that utilize RMSprop and Adam optimization strategies is examined. However, even the most minor enhancement may significantly impact cybersecurity. A handwriting text is one of the authentication mechanisms to identify persons by their signature and writing style. In this experiment, a CNN model with Adam optimizer and PReLU activation function achieved the highest accuracy of 98.60% when trained on the Kaggle handwritten digit dataset, making a significant difference compared to classical learning algorithms and a slight improvement to other CNN models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmad, K., Maabreh, M., Ghaly, M., Khan, K., Qadir, J., Al-Fuqaha, A.: Developing future human-centered smart cities: critical analysis of smart city security, data management, and ethical challenges. Comput. Sci. Rev. 43, 100452 (2022)
Reddy, R.V.K., Rao, B.S., Raju, K.P.: Handwritten Hindi digits recognition using convolutional neural network with RMSProp optimization. In: 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India (2018)
Li, J., Sun, G., Yi, L., Cao, Q., Liang, F., Sun, Y.: Handwritten digit recognition system based on convolutional neural network. In: 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), Dalian, China (2020)
Du, G., Jia, L., Wei, L.: A new algorithm of handwritten numeral recognition based on GPU multi-stream concurrent and parallel model. In: 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Weihai, China (2020)
Yin, Y., et al.: IGRF-RFE: a hybrid feature selection method for MLP-based network intrusion detection on UNSW-NB15 dataset. J. Big Data 10(1), 1–26 (2023)
Maabreh, M., Obeidat, I., Elsoud, E.A., Alnajjar, A., Alzyoud, R., Darwish, O.: Towards data-driven network intrusion detection systems: features dimensionality reduction and machine learning. Int. J. Interact. Mob. Technol. 17(14) (2022)
Siddique, F., Sakib, S., Siddique, M.A.B.: Recognition of handwritten digit using convolutional neural network in python with tensorflow and comparison of performance for various hidden layers. In: 2019 5th International Conference on Advances in Electrical Engineering (ICAEE), Dhaka, Bangladesh (2019)
Hao, W., Yizhou, W., Yaqin, L., Zhili, S.: The role of activation function in CNN. In: 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China (2020)
Maabreh, M., Darwish, O., Karajeh, O., Tashtoush, Y.: On developing deep learning models with particle swarm optimization in the presence of poisoning attacks. In: 2022 International Arab Conference on Information Technology (ACIT), pp. 1–5. IEEE (2022)
Maabreh, M., Maabreh, A., Qolomany, B., Al-Fuqaha, A.: The robustness of popular multiclass machine learning models against poisoning attacks: lessons and insights. Int. J. Distrib. Sens. Netw. 18(7), 15501329221105160 (2022)
Agrawal, A.K., Shrivas, A.K., Awasthi, V.K.: Robust model for handwritten digit recognition using machine and deep learning technique. In: 2021 2nd International Conference for Emerging Technology (INCET), Belagavi, India (2021)
Assegie, T.A.: Handwritten digits recognition with decision tree classification: a machine learning approach. Int. J. Electr. Comput. Eng. (IJECE) (2019)
Hegde, V., Usmani, S.: Parallel and distributed deep learning. https://web.stanford.edu/~rezab/classes/cme323/S16/projects_reports/hedge_usmani.pdf
Pethick, M. Liddle, M., Werstein, P., Huang, Z.: Parallelization of a backpropagation neural network on a cluster computer. In: International Conference on Parallel and Distributed Computing and Systems (PDCS 2003) (2003)
Vrtanoski, J., Stojanovski, T.D.: Pattern recognition with OpenCL heterogeneous platform. In: 2012 20th Telecommunications Forum (TELFOR) (2012)
Du, G., Jia, L., Wei, L.: A new algorithm of handwritten numeral recognition based on GPU multi-stream concurrent and parallel model. Presented at the 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Weihai, China (2020)
Polok, L., Smrz, P.: Fast linear algebra on GPU. In: 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems (2012)
Musaev, M., Rakhimov, M.: Accelerated training for convolutional neural networks. In: 2020 International Conference on Information Science and Communications Technologies (ICISCT) (2020)
Singh, S., Bhatele, A.: AxoNN: an asynchronous, message-driven parallel framework for extreme-scale deep learning. Presented at the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Lyon, France (2022)
Khomenko, V., Shyshkov, O., Radyvonenko, O., Bokhan, K.: Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization. In: 2016 IEEE First International Conference on Data Stream Mining & Processing (DSMP), pp. 100–103 (2016)
Elango, V.: Pase: parallelization strategies for efficient DNN training. In: 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 1025–1034 (2021)
Li, Y., Iwamoto, Y., Lin, L., Xu, R., Tong, R., Chen, Y.W.: VolumeNet: a lightweight parallel network for super-resolution of MR and CT volumetric data. IEEE Trans. Image Process. (2021)
Mahmud, A., Tanvin, A., Rahman, S.: Handwritten English character and digit recognition. In: 2021 International Conference on Electronics, Communications and Information Technology (ICECIT), Khulna, Bangladesh (2021)
Jain, M., Kaur, G., Quamar, M.P., Gupta, H.: Handwritten digit recognition using CNN. In: 2021 International Conference on Innovative Practices in Technology and Management (ICIPTM), Noida, India (2021)
Zaheer, R., Shaziya, H.: A study of the optimization algorithms in deep learning. In: 2019 Third International Conference on Inventive Systems and Control (ICISC), Coimbatore, India (2019)
Mukkamala, M.C., Hein, M.: Variants of RMSProp and adagrad with logarithmic regret bounds. arXiv preprint arXiv:1706.05507 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Badola, A., Nair, V.P., Lal, R.P.: An analysis of regularization methods in deep neural networks. In: 2020 IEEE 17th India Council International Conference (INDICON), New Delhi, India (2020)
Chen, Z., et al.: Handwritten digit recognition based on corner detection and convolutional neural network. J. Phys.: Conf. Ser. (2020)
Abu Ghosh, M.M., Maghari, A.Y.: A comparative study on handwriting digit recognition using neural networks. In: 2017 International Conference on Promising Electronic Technologies (ICPET), Deir El-Balah, Palestine (2017)
Younis, K.S., Alkhateeb, A.A.: A new implementation of deep neural networks for optical character recognition and face recognition. In: Proceedings of the New Trends in Information Technology, Jordan (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Amsaad, F. et al. (2023). Toward Secure and Efficient CNN Recognition with Different Activation and Optimization Functions. In: Daimi, K., Al Sadoon, A. (eds) Proceedings of the 2023 International Conference on Advances in Computing Research (ACR’23). ACR 2023. Lecture Notes in Networks and Systems, vol 700. Springer, Cham. https://doi.org/10.1007/978-3-031-33743-7_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-33743-7_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33742-0
Online ISBN: 978-3-031-33743-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)