Skip to main content
SpringerLink
Log in

Research on Non-pooling Convolutional Text Classification Technology Combined with Attention Mechanism

  • Conference paper
  • First Online:
Business Intelligence and Information Technology (BIIT 2021)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 107))

  • 1068 Accesses

Abstract

Text classification technology is a relatively basic and important application technology in natural language processing. Although the classification method based on traditional machine learning algorithms and manual annotation feature engineering has achieved certain results, the efficiency is not ideal. In recent years, the application of deep learning in natural language tasks processing has received extensive attention. In this paper, the structure of the traditional convolutional neural network model is modified, and an attention layer is introduced to complete the task of text classification. The feasibility of the model is analyzed, and a new research direction is provided for the application of neural networks in the field of natural language processing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Wan, J., Wu, Y.: Review of text classification research based on deep learning. J. TianJin Univ. Technol. 37(02), 41–47 (2021)

    Google Scholar 

  2. Antonellis, I., Bouras, C., Poulopoulos, V.: Personalized news categorization through scalable text classification. In: Zhou, X., Li, J., Shen, H.T., Kitsuregawa, M., Zhang, Y. (eds.) APWeb 2006. LNCS, vol. 3841, pp. 391–401. Springer, Heidelberg (2006). https://doi.org/10.1007/11610113_35

    Chapter  Google Scholar 

  3. Cai, X., Lou, J.: Sentiment analysis of telecom official micro-blog users based on LSTM deep learning model. Telecommun. Sci. 33(12), 136–141 (2017)

    Google Scholar 

  4. Li, D.: The study of Chinese text categorization based on Naive Bayes. Hebei University (2011)

    Google Scholar 

  5. Sun, Y., Li, Y., Bian, Y.: Application of random forest algorithm for book subject classification. Comput. Technol. Dev. 30(06), 65–70 (2020)

    Google Scholar 

  6. Yan, P., Tang, W.: Chinese text clasification based on improved BERT. 33(07), 108–110+112 (2020)

    Google Scholar 

  7. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, Doha, Qatar, pp. 1746–1751 (2014)

    Google Scholar 

  8. Ruderman, A., Rabinowitz, N.C., Morcos, A.S., et al.: Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs. arXiv (2018)

    Google Scholar 

  9. Tu, W., Yuan, Z., Yu, K.: Convolutional neural networks without pooling layer for Chinese word segmentation. Comput. Eng. Appl. 56(02), 120–126 (2020)

    Google Scholar 

  10. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)

    Google Scholar 

  11. Vaswani, A. Shazeer, N., Parmar, N., et al.: Attention is all your need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  12. Mikolov, T., Corrado, G., Chen, K., et al.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations, ICLR, Scottsdale, Arizona, USA, pp. 1–12 (2013)

    Google Scholar 

  13. Joulin, A., Grave, E., Bojanowski, P., et al.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers (2017)

    Google Scholar 

  14. Zhu, M.: Research and implementation of Chinese text classification algorithm based on machine learning. Beijing University of Posts and Telecommunications (2019)

    Google Scholar 

  15. He, K.: Reasearch and application of text classification based on natural language processing. Nanjing University of Posts and Telecommunications (2020)

    Google Scholar 

  16. Wu, M., Wen, L., Sun, M.: Attention Mechanism Image understanding algorithm of ocean scene. Computer Engineering and Applications, pp. 1–11. http://kns.cnki.net/kcms/detail/11.2127.TP.20210317.1008.002.html.2021/07/11

  17. Zhang, P., Xu, Z., Hu, P.: Segmentation of intracranial hemorrhage fusing dense connection and attention mechanism. J. Chin. Mini-Micro Comput. Syst. 42(07), 1458–1463 (2021)

    Google Scholar 

  18. Liu, T., Zhu, W., Liu, G.: Advances in deep learning based text classification. Electr. Power ICT 16(03), 1–7 (2018)

    Google Scholar 

Download references

Acknowledgments

This work is supported by the Natural Science Foundation of Heilongjiang Province of China (No. YQ2020G002), University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province (No. UNPYSCT-2020212).

Author information

Authors and Affiliations

  1. Harbin University of Commerce, Harbin, 150028, China

    Hui Li, Zeming Li, Wei Zhao & Xue Tan

  2. Heilongjiang Provincial Key Laboratory of Electronic Commerce and Information Processing, Harbin, 150028, China

    Hui Li

Authors
  1. Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zeming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xue Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Information Technology Department, Cairo University, Giza, Egypt

    Aboul Ella Hassanien

  2. School of Computer and Information Engineering, Harbin University of Commerce, Harbin, Heilongjiang, China

    Yaoqun Xu

  3. School of Computer and Information Engineering, Harbin University of Commerce, Harbin, Heilongjiang, China

    Zhijie Zhao

  4. Department of Computer Science, Lakehead University, Thunder Bay, ON, Canada

    Sabah Mohammed

  5. School of Computer and Information Engineering, Harbin University of Commerce, Harbin, Heilongjiang, China

    Zhipeng Fan

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, H., Li, Z., Zhao, W., Tan, X. (2022). Research on Non-pooling Convolutional Text Classification Technology Combined with Attention Mechanism. In: Hassanien, A.E., Xu, Y., Zhao, Z., Mohammed, S., Fan, Z. (eds) Business Intelligence and Information Technology. BIIT 2021. Lecture Notes on Data Engineering and Communications Technologies, vol 107. Springer, Cham. https://doi.org/10.1007/978-3-030-92632-8_23

Download citation

Publish with us

Policies and ethics

Search

Navigation