Advertisement

Convolutional Neural Networks for Unsupervised Anomaly Detection in Text Data

  • Oleg Gorokhov
  • Mikhail PetrovskiyEmail author
  • Igor Mashechkin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10585)

Abstract

In this paper, we discuss the problem of anomaly detection in text data using convolutional neural network (CNN). Recently CNNs have become one of the most popular and powerful tools for various machine learning tasks. CNN’s main advantage is an ability to extract complicated hidden features from high dimensional data with complex structure. Usually CNNs are applied in supervised learning mode. On the other hand, unsupervised anomaly detection is an important problem in many applications, including computer security, behavioral analytics, etc. Since there is no specified target in unsupervised mode, traditional CNN’s objective functions cannot be used. In this paper, we develop a specific CNN architecture. It consists of one convolutional layer and one subsampling layer, we use RBF activation function and logarithmic loss function on the final layer. Minimization of the corresponding objective function helps us to calculate the location parameter of the features’ weights discovered on the last network layer. We use \(l_2\)-regularization to avoid degenerate solution. Proposed CNN has been tested on anomalies discovering in a stream of text documents modeled with well-known Enron dataset, where proposed method demonstrates better results in comparison with the traditional outlier detection methods based on one-class SVM and NMF.

Keywords

Anomaly detection Text mining Convolutional neural network One-class classification Regularization SVM NMF 

Notes

Acknowledgment

This research is supported by the RFBR Grant No. 16-29-09555.

References

  1. 1.
    Britz, D.: Implementing a CNN for text classification in tensorflow (2015). http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/
  2. 2.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009)CrossRefGoogle Scholar
  3. 3.
    Clifton, L., Clifton, D.A., Zhang, Y., Watkinson, P., Tarassenko, L., Yin, H.: Probabilistic novelty detection with support vector machines. IEEE Trans. Reliab. 63(2), 455–467 (2014)CrossRefGoogle Scholar
  4. 4.
    Hawkins, S., He, H., Williams, G., Baxter, R.: Outlier detection using replicator neural networks. In: Kambayashi, Y., Winiwarter, W., Arikawa, M. (eds.) DaWaK 2002. LNCS, vol. 2454, pp. 170–180. Springer, Heidelberg (2002). doi: 10.1007/3-540-46145-0_17 CrossRefGoogle Scholar
  5. 5.
    Enron email dataset. www.cs.cmu.edu/./enron/
  6. 6.
    Kannan, R., Woo, H., Aggarwal, C.C., Park, H.: Outlier detection for text data: An extended version. CoRR abs/1701.01325 (2017)Google Scholar
  7. 7.
    Kim, Y.: Convolutional neural networks for sentence classification. CoRR abs/1408.5882 (2014)Google Scholar
  8. 8.
    Lee, J.Y., Dernoncourt, F.: Sequential short-text classification with recurrent and convolutional neural networks. CoRR abs/1603.03827 (2016). http://arxiv.org/abs/1603.03827
  9. 9.
    Manevitz, L.M., Yousef, M.: One-class SVMS for document classification. J. Mach. Learn. Res. 2, 139–154 (2001)zbMATHGoogle Scholar
  10. 10.
    Mashechkin, I.V., Petrovskii, M.I., Tsarev, D.V.: Machine learning methods for analyzing user behavior when accessing text data in information security problems. Mosc. Univ. Comput. Math. Cybern. 40(4), 179–184 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Mirzal, A.: Converged algorithms for orthogonal nonnegative matrix factorizations. CoRR abs/1010.5290 (2010)Google Scholar
  12. 12.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Tsarev, D.V., Petrovskiy, M.I., Mashechkin, I.V., Korchagin, A.Y., Korolev, V.Y.: Applying time series to the task of background user identification based on their text data analysis. Proc. Inst. Syst. Program. 27(1), 151–172 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Oleg Gorokhov
    • 1
  • Mikhail Petrovskiy
    • 1
    Email author
  • Igor Mashechkin
    • 1
  1. 1.Computer Science Department of Lomonosov Moscow State University, MSUMoscowRussia

Personalised recommendations