Skip to main content
SpringerLink
Log in

Neural kernel mapping SVM model based on multi-head self-attention for classification of Chinese meteorological disaster warning texts

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Meteorological disaster warning information plays an important role in our life. However, if the meteorological department mistakenly sends incorrect meteorological disaster warning information, it will have catastrophic consequences. Meteorological disaster warning information is often sent in the form of text. Therefore, the study of meteorological disaster warning texts is very important. Different from many studies on English texts, we focus on Chinese meteorological disaster warning texts. This article proposes a new method combining neural kernel mapping support vector machine(SVM) and multi-head self-attention mechanism to improve the accuracy of predicting Chinese meteorological disaster warning texts. Our method takes multi-head self-attention mechanism as the neural kernel mapping of support vector machine. In addition, in order to solve the problem of training difficulties caused by insufficient Chinese meteorological disaster warning texts data, this paper develops an automatic semantic annotation system for Chinese meteorological disaster warning texts. Based on correct Chinese meteorological disaster warning texts, the system can automatically generate sample data of four types of errors(including wrong words, repetitions, missing words, and reverse order). In our experiments, we use 3 self-made Chinese meteorological disaster warning text datasets and 4 other types of Chinese text datasets. The experimental results show that compared with other methods, our method is not only effective for Chinese meteorological disaster warning texts, but also has certain advantages for other types of Chinese text data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Bansal T, Belanger D, McCallum A (2016) Ask the gru: Multi-task learning for deep text recommendations. In: proceedings of the 10th ACM Conference on Recommender Systems. pp 107–114

  2. Bayer M, Kaufhold M-A, Buchhold B et al (2022) Data augmentation in natural language processing: a novel text generation approach for long and short text classifiers. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-022-01553-3

    Article  Google Scholar 

  3. Bazi Y, Bashmal L, Rahhal MM Al et al (2021) Vision transformers for remote sensing image classification. Remote Sens 13(3):516. https://doi.org/10.3390/rs13030516

  4. Brown TB, Mann B, Ryder N et al (2020) Language models are few-shot learners. arXiv Prepr arXiv200514165

  5. Cao Z, Zhao J (2017) Research on early warning quality control technology. Manag Res Sci Technol Achiev 009:40–43

    Google Scholar 

  6. Chiu JPC, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist 4:357–370

    Article  Google Scholar 

  7. Cho K, Van Merriënboer B, Gulcehre C et al (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv Prepr arXiv14061078

  8. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1007/BF00994018

    Article  Google Scholar 

  9. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv Prepr arXiv181004805

  10. Forouzandeh S, Rostami M, Berahmand K (2021) Presentation a trust walker for rating prediction in recommender system with biased random walk: effects of h-index centrality, similarity in items and friends. Eng Appl Artif Intell 104:104325. https://doi.org/10.1016/j.engappai.2021.104325

    Article  Google Scholar 

  11. Forouzandeh S, Rostami M, Berahmand K (2022) A hybrid method for recommendation systems based on Tourism with an evolutionary algorithm and topsis model. Fuzzy Inf Eng 14:26–50. https://doi.org/10.1080/16168658.2021.2019430

    Article  Google Scholar 

  12. Garg S, Vu T, Moschitti A (2020) Tanda: transfer and adapt pre-trained transformer models for answer sentence selection. In: Proceedings of the AAAI Conference on Artificial Intelligence. AAAI, pp 7780–7788. https://doi.org/10.1609/aaai.v34i05.6282

  13. Garibaldi-Márquez F, Flores G, Mercado-Ravell DA, Ramírez-Pedraza A, Valentín-Coronado LM (2022) Weed classification from natural corn field-multi-plant images based on shallow and deep learning. Sensors 22(8):3021. https://doi.org/10.3390/s22083021

  14. Guo X, Li H, Jing L, Wang P (2022) Individual tree species classification based on convolutional neural networks and multitemporal high-resolution remote sensing images. Sensors 22(9):3157. https://doi.org/10.3390/s22093157

  15. Hermanto A, Adji TB, Setiawan NA (2015) Recurrent neural network language model for English-Indonesian Machine Translation: Experimental study. In: 2015 International conference on science in information technology (ICSITech). IEEE, pp 132–136

  16. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780

    Article  Google Scholar 

  17. Hu J, Guo T, Cao J, Zhang C (2017) End-to-end Chinese text recognition. In: 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, pp 1407–1411

  18. Huang XS, Perez F, Ba J, Volkovs M (2020) Improving transformer optimization through better initialization. In: International Conference on Machine Learning. PMLR, pp 4475–4483

  19. Johnson R, Zhang T (2014) Effective use of word order for text categorization with convolutional neural networks. arXiv Prepr arXiv14121058

  20. Johnson R, Zhang T (2015) Semi-supervised convolutional neural networks for text categorization via region embedding. Adv Neural Inf Process Syst 28:919

    Google Scholar 

  21. Li Y, Zhang T (2017) Deep neural mapping support vector machines. Neural Netw 93:185–194

    Article  Google Scholar 

  22. Liu Z, Kan H, Zhang T, Li Y (2020) DUKMSVM: A framework of deep uniform kernel mapping support vector machine for short text classification. Appl Sci 10(7):2348. https://doi.org/10.3390/app10072348

  23. Liu B, Zhou Y, Sun W (2020) Character-level text classification via convolutional neural network and gated recurrent unit. Int J Mach Learn Cybern 11:1939–1949. https://doi.org/10.1007/s13042-020-01084-9

    Article  Google Scholar 

  24. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv Prepr arXiv13013781

  25. Mikolov T, Zweig G (2012) Context dependent recurrent neural network language model. In: 2012 IEEE Spoken Language Technology Workshop (SLT). IEEE, pp 234–239

  26. Nasiri E, Berahmand K, Li Y (2023) Robust graph regularization nonnegative matrix factorization for link prediction in attributed networks. Multimed Tools Appl 82:3745–3768. https://doi.org/10.1007/s11042-022-12943-8

    Article  Google Scholar 

  27. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pp 1532–1543

  28. Peters ME, Neumann M, Iyyer M et al (2018) Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, New Orleans, Louisiana, pp 2227–2237

  29. Pham N-Q, Nguyen T-S, Niehues J et al (2019) Very deep self-attention networks for end-to-end speech recognition. arXiv Prepr arXiv190413377

  30. Qiao X, Peng C, Liu Z, Hu Y (2019) Word-character attention model for Chinese text classification. Int J Mach Learn Cybern 10:3521–3537

    Article  Google Scholar 

  31. Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding with unsupervised learning. Technical Report, OpenAI

  32. Radford A, Wu J, Child R et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1:9

    Google Scholar 

  33. Ren H, Yang L, Xun E (2018) A sequence to sequence learning for Chinese grammatical error correction. In: CCF International Conference on Natural Language Processing and Chinese Computing. Springer, pp 401–410

  34. Setyanto A, Laksito A, Alarfaj F, Alreshoodi M, Kusrini, Oyong I, Hayaty M, Alomair A, Almusallam N, Kurniasari L (2022) Arabic language opinion mining based on long short-term memory (LSTM). Appl Sci 12(9):4140. https://doi.org/10.3390/app12094140

  35. Sundermeyer M, Oparin I, Gauvain J-L et al (2013) Comparison of feedforward and recurrent neural network language models. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp 8430–8434

  36. Sundermeyer M, Schlüter R, Ney H (2012) LSTM neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association. ISCA. https://www.isca-speech.org/archive_v0/archive_papers/interspeech_2012/i12_0194.pdf

  37. Tao H, Tong S, Zhao H et al (2019) A radical-aware attention-based model for chinese text classification. Proc AAAI Conf Artif Intell 33(1):5125–5132. https://doi.org/10.1609/aaai.v33i01.33015125

    Article  Google Scholar 

  38. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

  39. Wang Q, Li B, Xiao T et al (2019) Learning deep transformer models for machine translation. arXiv Prepr arXiv190601787

  40. Wu Y-C, Yin F, Chen Z, Liu C-L (2017) Handwritten chinese text recognition using separable multi-dimensional recurrent neural network. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). IEEE, pp 79–84

  41. Yu J, Ji B, Li S, Ma J, Liu H, Xu H (2022) S-NER: A concise and efficient span-based model for named entity recognition. Sensors 22(8):2852. https://doi.org/10.3390/s22082852

  42. Zhang S, Miao K (2019) Implementation of quality control systems based on Bi-LSTM-CRF algorithm for meteorological warning information. Comput Mod 6:115–119. https://doi.org/10.3969/j.issn.1006-2475.2019.06.019

    Google Scholar 

  43. Zhang B, Titov I, Sennrich R (2019) Improving deep transformer with depth-scaled initialization and merged attention. arXiv Prepr arXiv190811365

  44. Zheng B, Che W, Guo J, Liu T (2016) Chinese grammatical error diagnosis with long short-term memory networks. In: Proceedings of the 3rd Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA2016). The COLING 2016 Organizing Committee, Osaka, Japan, pp 49–56. https://aclanthology.org/W16-4907

Download references

Funding

This research was funded by National key R & D projects, grant numbers 2018YFF0300105 and 2018YFC1507805.

Author information

Authors and Affiliations

  1. Public Meteorological Service Center of China Meteorological Administration, Beijing, China

    Muhua Wang, Wei Tang, Jianzhong Hui, Hanhua Qu, Yanpeng Li, Lei Cui & Tianyue Wang

  2. Beijing University of Technology, Beijing, China

    Jidong Han

Authors
  1. Muhua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianzhong Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hanhua Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanpeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lei Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tianyue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jidong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Muhua Wang or Jidong Han.

Ethics declarations

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Tang, W., Hui, J. et al. Neural kernel mapping SVM model based on multi-head self-attention for classification of Chinese meteorological disaster warning texts. Multimed Tools Appl 83, 16543–16561 (2024). https://doi.org/10.1007/s11042-023-16070-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16070-w

Keywords

Search

Navigation