Abstract
Over the past few decades, communication has become easier due to the rapid development in technology. Although several modes exist for communication, in this era of Internet, electronic mail or email, turn out to be very popular because of its effectiveness, inexpensive, and easy to use for personal communication as well as business purposes and sharing important information in the form of text, images, documents, etc. to others. This proficiency leads to email being exposed to numerous attacks together with spamming. At present, spam email is a major source of concern for email users where unbidden messages, used for business purposes, are directed extensively to several mailing lists, entities, or newsgroups. These spam emails are used for the purpose of advertising products, collecting personal information, sending destructive contents in the form of executable file to outbreak user systems or the link to malicious website to steal confidential data such as hacking bank accounts, passwords leading to reduction in efficiency, security threats, consume server storage space, and unessential consumption of network bandwidth. Currently, there are 47.3% spam emails out of all emails and henceforth it become necessary to build a competent spam filters to categorize and block spam email. In order to enhance the accuracy of the model, natural language processing is used. In the proposed framework, efficacy of word embedding is offered to categorize the spam emails. It fine-tunes the pretrained Bidirectional Encoder Representations from Transformers (BERT) model to classify the legitimate emails and spam emails. Attention layer is used by the BERT model to incorporate the context of the text into that perspective. The outcomes are compared with minimum Redundancy Maximum Relevance (mRMR), Artificial Neural Network, Recurrent Neural Network, and Long Short-Term Memory (LSTM) for Lingspam, Enron, and Spamassassin dataset. The proposed method accomplished the uppermost correctness of 97 and 98% F1-score.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hadeel, M.S.: An Efficient feature selection algorithm for the spam email classification. Period. Eng. Nat. Sci. 9(3), 520–531 (2021)
Thirumagal Dhivya, S., Nithya, S., Sangavi Priya, G., Pugazhendi, E.: Email spam detection and data optimization using NLP techniques. Int. J. Eng. Res. Technol. 10(8), 38–49 (2021)
Unnikrishnan, V., Kamath, P.: Analysis of email spam detection using machine learning. Int. Res. J. Modern. Eng. Technol. Sci. 3(9), 409–416 (2021)
Bansal, L., Tiwari, N.: Feature selection based classification of spams using fuzzy support vector machine. In: Proceedings of the 2020 International Conference on Smart Electronics and Communication (ICOSEC), pp. 258–263 (2020)
Lovelyn Rose, S., Ashok Kumar, L., Karthika Renuka, D.: Deep Learning using Python. Wiley, Hoboken (2019)
Sethi, M., Chandra, S., Chaudhary, V., Yash, S.: Email Spam detection using machine learning and neural networks. Int. Res. J. Eng. Technol. 8(4), 349–355 (2021)
Pashiri, R.T., Rostami, Y., Mahrami, M.: Spam detection through feature selection using artifcial neural network and sine–cosine algorithm. J. Math. Sci. 14(3), 193–199 (2020)
Sumathi, S., Pugalendhi, G.K.: Cognition based spam mail text analysis using combined approach of deep neural network classifer and random forest. J. Ambient Intell. Hum. Comput. 12(6), 5721–5731 (2020)
Basyar, I., Adiwijaya, S., Murdiansyah, D.T.: Email spam classification using gated recurrent unit and long short-term memory. J. Comput. Sci.Comput. Sci. 16(4), 559–567 (2020)
Sriram, S., Vinayakumar, R., Sowmya, V., Krichen, M., Noureddine, D.B., Anivilla, S., Soman, K.P.: Deep convolutional neural network based image spam classification. In: Proceedings of the 6th Conference on Data Science and Machine Learning Applications (CDMA), pp. 112–117, Riyadh, Saudi Arabia (2020)
Jitendra Kumar, A., Santhanavijayan, S., Rajendran, B., Bindhumadhava, B.S.: An adaptive neural network for email spam classification. In: Proceedings of the 2019 Fifteenth International Conference on Information Processing (ICINPRO), pp. 1–7, Bengaluru, India (2019)
Kulkarni, P., Jatinderkumar, S., Saini, R., Acharya, H.: Effect of header-based features on accuracy of classifiers for spam email classification. Int. J. Adv. Comput. Sci. Appl.Comput. Sci. Appl. 11(3), 396–401 (2020)
Sri Vinitha, V., Karthika Renuka, D.: MapReduce mRMR: random forests-based email spam classification in distributed environment. In: Data Management, Analytics and Innovation, pp. 241–253. Springer, Singapore (2020)
AbdulNabi, I., Yaseen, Q.: Spam email detection using deep learning techniques. Proced. Comput. Sci. 184, 853–858 (2021)
Isik, S., Kurt, Z., Anagun, Y., Ozkan, K.: Recurrent neural networks for spam e-mail classification on an agglutinative language. Int. J. Intell. Syst. Appl. Eng. 8(4), 221–227 (2020)
Liu, S., Tao, H., Feng, S.: Text classification research based on Bert model and Bayesian network. In: Proceedings of the 2019 Chinese Automation Congress (CAC), pp. 5842–5846, Hangzhou, China (2019)
Lagrari, F., Elkettani, Y.: Customized BERT with convolution model: a new heuristic enabled encoder for Twitter sentiment analysis. Int. J. Adv. Comput. Sci. Appl.Comput. Sci. Appl. 11(10), 423–431 (2020)
Si, S., Wang, R., Wosik, J., Zhang, H., Dov, D., Wang, G., Carin, L.: Students need more attention: BERT-based attention model for small data with application to automatic patient message triage. In: Machine Learning for Healthcare Conference; Virtual, pp. 436–456 (2020)
Ali, N., Fatima, A., Shahzadi, H., Ullah, A., Polat, K.: Feature extraction aligned email classification based on imperative sentence selection through deep learning. J. Artif. Intell. Syst. 3(1), 93–114 (2021)
Ablel-Rheem, D.M., Ibrahim, A.O., Kasim, S., Almazroi, A.A., Ismail, M.A.: Hybrid feature selection and ensemble learning method for spam email classification. Int. J. Adv. Trends Comput. Sci. Eng. 9(1), 217–223 (2020)
Bhattacharya, P., Singh, A.: E-mail spam filtering using genetic algorithm based on probabilistic weights and words count. Int. J. Integr. Eng. 12(1), 40–49 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Vinitha, V.S., Renuka, D.K., Kumar, L.A. (2023). Transformer-Based Attention Model for Email Spam Classification. In: Bhateja, V., Yang, XS., Ferreira, M.C., Sengar, S.S., Travieso-Gonzalez, C.M. (eds) Evolution in Computational Intelligence. FICTA 2023. Smart Innovation, Systems and Technologies, vol 370. Springer, Singapore. https://doi.org/10.1007/978-981-99-6702-5_18
Download citation
DOI: https://doi.org/10.1007/978-981-99-6702-5_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6701-8
Online ISBN: 978-981-99-6702-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)