Advertisement

A New LSTM Network Model Combining TextCNN

  • Xiao Sun
  • Xiaohu Ma
  • Zhiwen Ni
  • Lina Bian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11301)

Abstract

The development of computer communication technology has brought massive amounts of spam texts. Spammers use a variety of textual means to avoid detection of spam texts, which has brought challenges to spam text filtering technology. Deep neural network has superior performance in feature representation and feature extraction. TextCNN based on convolutional neural network can extract the local feature representation of sentences, but ignore the successive relationship between words. The LSTM based on the recurrent neural network takes into account the sequential relationship between words, but it is not as good as TextCNN in representation of local features. We propose an algorithm that combines the TextCNN and LSTM network called TC-LSTM to implement spam text filtering, and compare the Precision, Recall and F-measure indicators with the traditional TextCNN and LSTM on two datasets. Experiments show that our TC-LSTM algorithm is superior to the traditional TextCNN and LSTM networks in spam text filtering.

Keywords

Deep neural network Network fusion Spam text filtering Deep learning 

Notes

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (61402310). Natural Science Foundation of Jiangsu Province of China (BK20141195).

References

  1. 1.
    Bengio, Y., Schwenk, H., Senécal, J.S., Morin, F., Gauvain, J.L.: Neural probabilistic language models. In: Holmes, D.E., Jain, L.C. (eds.) Innovations in Machine Learning. STUDFUZZ, vol. 194. Springer, Heidelberg (2006).  https://doi.org/10.1007/3-540-33486-6_6CrossRefGoogle Scholar
  2. 2.
    Cambria, E., White, B.: Jumping NLP curves: a review of natural language processing research. IEEE Comput. Intell. Mag. 9(2), 48–57 (2014)CrossRefGoogle Scholar
  3. 3.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)zbMATHGoogle Scholar
  4. 4.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  5. 5.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014)
  6. 6.
    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
  7. 7.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  8. 8.
    Liu, P., Qiu, X., Chen, J., Huang, X.: Deep fusion LSTMs for text semantic matching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1034–1043 (2016)Google Scholar
  9. 9.
    Mandic, D.P., Chambers, J.A., et al.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. Wiley Online Library, Hoboken (2001)CrossRefGoogle Scholar
  10. 10.
    Marz, N., Warren, J.: Big Data: Principles and Best Practices of Scalable Real-time Data Systems. Manning Publications Co, New York (2015)Google Scholar
  11. 11.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  12. 12.
    Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G.: Learning semantic representations using convolutional neural networks for web search. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 373–374. ACM (2014)Google Scholar
  13. 13.
    Wang, X., Liu, Y., Chengjie, S., Wang, B., Wang, X.: Predicting polarities of tweets by composing word embeddings with long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. 1, pp. 1343–1353 (2015)Google Scholar
  14. 14.
    Yih, W., He, X., Meek, C.: Semantic parsing for single-relation question answering. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 643–648 (2014)Google Scholar
  15. 15.
    Yu, K., Jia, L., Chen, Y., Xu, W.: Deep learning: yesterday, today, and tomorrow. J. Comput. Res. Dev. 20(6), 1349 (2013)Google Scholar
  16. 16.
    Zhou, X., Wan, X., Xiao, J.: Attention-based LSTM network for cross-lingual sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 247–256 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and TechnologySoochow UniversitySuzhouChina
  2. 2.Collaborative Innovation Center of Novel Software Technology and IndustrializationNanjingChina

Personalised recommendations