Convolutional Neural Networks for Unsupervised Anomaly Detection in Text Data
In this paper, we discuss the problem of anomaly detection in text data using convolutional neural network (CNN). Recently CNNs have become one of the most popular and powerful tools for various machine learning tasks. CNN’s main advantage is an ability to extract complicated hidden features from high dimensional data with complex structure. Usually CNNs are applied in supervised learning mode. On the other hand, unsupervised anomaly detection is an important problem in many applications, including computer security, behavioral analytics, etc. Since there is no specified target in unsupervised mode, traditional CNN’s objective functions cannot be used. In this paper, we develop a specific CNN architecture. It consists of one convolutional layer and one subsampling layer, we use RBF activation function and logarithmic loss function on the final layer. Minimization of the corresponding objective function helps us to calculate the location parameter of the features’ weights discovered on the last network layer. We use \(l_2\)-regularization to avoid degenerate solution. Proposed CNN has been tested on anomalies discovering in a stream of text documents modeled with well-known Enron dataset, where proposed method demonstrates better results in comparison with the traditional outlier detection methods based on one-class SVM and NMF.
KeywordsAnomaly detection Text mining Convolutional neural network One-class classification Regularization SVM NMF
This research is supported by the RFBR Grant No. 16-29-09555.
- 1.Britz, D.: Implementing a CNN for text classification in tensorflow (2015). http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/
- 5.Enron email dataset. www.cs.cmu.edu/./enron/
- 6.Kannan, R., Woo, H., Aggarwal, C.C., Park, H.: Outlier detection for text data: An extended version. CoRR abs/1701.01325 (2017)Google Scholar
- 7.Kim, Y.: Convolutional neural networks for sentence classification. CoRR abs/1408.5882 (2014)Google Scholar
- 8.Lee, J.Y., Dernoncourt, F.: Sequential short-text classification with recurrent and convolutional neural networks. CoRR abs/1603.03827 (2016). http://arxiv.org/abs/1603.03827
- 11.Mirzal, A.: Converged algorithms for orthogonal nonnegative matrix factorizations. CoRR abs/1010.5290 (2010)Google Scholar