Anomaly detecting and ranking of the cloud computing platform by multi-view learning

Abstract

Anomaly detecting as an important technical in cloud computing is applied to support smooth running of the cloud platform. Traditional detecting methods based on statistic, analysis, etc. lead to the high false-alarm rate due to non-adaptive and sensitive parameters setting. We presented an online model for anomaly detecting with the machine learning theory. However, most existing methods based on machine learning linked all features from difference sub-systems into a long feature vector directly, which is difficult to both exploit the complement information among sub-systems and ignore multi-view features enhancing the classification performance. Aiming to above problems, the proposed method automatic fuses multi-view features and optimizes the discriminative model to enhance the accuracy. This model takes advantage of extreme learning machine (ELM) to improve detection efficiency. ELM is the single hidden layer neural network, which is transforming iterative solution of the output weights to solution of linear equations and avoiding the local optimal solution. Moreover, we rank anomies according to the relationship between samples and the classification boundary, and then assigning weights for ranked anomalies, retraining the classification model finally. Our method exploits the complement information among sub-systems sufficiently, and avoids the influence from the imbalance distribution, therefore, deal with various challenges from the cloud computing platform. We deploy the privately cloud platform by Openstack, verifying the proposed model and comparing results to the state-of-the-art methods with better efficiency and simplicity.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. 1.

    Baek S, Kwon D, Kim J et al (2017) Unsupervised labeling for supervised anomaly detection in enterprise and cloud networks. In: 2017 IEEE 4th international conference on cyber security and cloud computing (CSC loud). IEEE, pp 205–210

  2. 2.

    Dean DJ, Nguyen H, Gu X (2012) Unsupervised behavior learning for predicting performance anomalies in virtualized cloud systems. In: Proceedings of the 9th ACM international conference on autonomic computing, pp 191–200

  3. 3.

    Du KL, Swamy MNS (2014) Independent component analysis. Springer, London

    Google Scholar 

  4. 4.

    Farshchi M, Schneider JG, Weber I et al (2017) Metric selection and anomaly detection for cloud operations using log and metric correlation analysis. J Syst Softw 137(5):531–549

    Google Scholar 

  5. 5.

    Fu S (2011) Performance metric selection for autonomic anomaly detection on cloud computing systems. In: Proceedings of the global telecommunications conference. IEEE, Washington, pp 1–5

  6. 6.

    Fu S (2011) Performance metric selection for autonomic anomaly detection on cloud computing systems. In: 2011 IEEE global telecommunications conference, pp 1–5

  7. 7.

    Fujimaki R, Yairi T, Machida K (2005) An approach to spacecraft anomaly detection problem using kernel feature space. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery in data mining. ACM, New York, pp 401–410

  8. 8.

    Guan Q, Fu S (2013) Adaptive Anomaly identification by exploring metric subspace in cloud computing infrastructures. In: Proceedings of the 2013 IEEE 32nd international symposium on reliable distributed systems. IEEE, Washington, pp 205–214

  9. 9.

    Lan ZL, Zheng ZM, Li YW (2010) Toward automated anomaly identification in large-scale systems. IEEE Trans Parallel Distrib Syst 21(2):174–187

    Article  Google Scholar 

  10. 10.

    Liu GC, Yan SC (2011) Latent low-rank representation for subspace segmentation and feature extraction. In: Proceedings of the IEEE international conference on computer vision. IEEE, Washington, pp 1615–1622

  11. 11.

    Liu J, Chen S, Zhou Z, et al. (2016) An anomaly detection algorithm of cloud platform based on self-organizing maps. Mathematical Problems in Engineering

  12. 12.

    Liu C, Jaja J, Pessoa L (2018) LEICA Laplacian Eigenmaps for group ICA decomposition of fMRI data. NeuroImage 169:363–373

    Article  Google Scholar 

  13. 13.

    Rittinghouse JW, Ransome JF (2016) Cloud computing: implementation, management, and security. CRC press, Boca Raton

    Google Scholar 

  14. 14.

    Rong HJ, Huang GB, Sundararajan N, et al. (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems[J]. IEEE Trans Syst Man Cybern Part B Cybern 39(4):1067–1072

    Article  Google Scholar 

  15. 15.

    Saleem M, Rajouri JK (2017) Cloud computing virtualization. International Journal of Computer Applications Technology and Research 6(7):290–292

    Article  Google Scholar 

  16. 16.

    Sauvanaud C, Silvestre G, Kaaniche M et al (2015) Data stream clustering for online anomaly detection in cloud applications. In: 2015 eleventh European dependable computing conference (EDCC). IEEE, pp 120–131

  17. 17.

    Tan Y, Nguyen H, Shen Z, Gu X, Venkatramani C, Rajan D. (2012) PREPARE predictive performance anomaly prevention for virtualized cloud systems. In: Proceedings of 32nd IEEE international conference on distributed computing systems, pp 285–294

  18. 18.

    Vidal R, Ma Y, Sastry SS (2016) Principal component analysis. Springer, New York

    Google Scholar 

  19. 19.

    Wang C, Talwar V, Schwan K, Ranganathan P (2010) Online detection of utility cloud anomalies using metric distributions. In: Proceedings of the 2010 IEEE/IFIP network operations and management symposium, pp 96–103

  20. 20.

    Wang DY (2012) Research and implementation of anomaly detection technology for cloud Computin. Shanghai Jiaotong University

  21. 21.

    Wang Y, Lin X, Wu L, Zhang W (2015) Effective multi-query expansions: robust landmark retrieval ACM multimedia

  22. 22.

    Wang Y, Lin X, Wu L, Zhang W, Zhang Q, Huang X (2015) Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans Image Process 24(11):3939–3949

    MathSciNet  Article  Google Scholar 

  23. 23.

    Wang Y, Zhang W, Wu L, et al. (2016) Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering[C]. In: IJCAI international joint conference on artificial intelligence, pp 2153–2159

  24. 24.

    Wang R, Nie F, Hong R et al (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030

    MathSciNet  Article  Google Scholar 

  25. 25.

    Wang Y, Lin X, Wu L, Zhang W (2017) Effective multi-query expansions: collaborative deep networks for robust landmark retrieval. IEEE Trans Image Processing 26(3):1393–1404

    MathSciNet  Article  Google Scholar 

  26. 26.

    Wang Y, Zhang W, Wu L, Lin X, Zhao X (2017) Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Transactions on Neural Networks and Learning Systems 28(1):57–70

    Article  Google Scholar 

  27. 27.

    Wang Y, Wu L (2018) Beyond low-rank representations: orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw 103:1–8

    Article  Google Scholar 

  28. 28.

    Wang Y, Wu L, Lin X, Gao J (2018) Multiview spectral clustering via structured low-rank matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 29(10):4833–4843

    Article  Google Scholar 

  29. 29.

    Ward JL, Lumsden SL (2016) Locally linear embedding: dimension reduction of massive protostellar spectra. Mon Not R Astron Soc 461(2):2250–2256

    Article  Google Scholar 

  30. 30.

    Wright J, Ganesh A, Rao S et al (2009) Robust principal component analysis: exact recovery of corrupted low-rank matrices. Advances in Neural Information Processing Systems 87(4):2080–2088

    Google Scholar 

  31. 31.

    Wu L, Shen C, Hengel A (2017) Deep linear discriminant analysis on fisher networks: a hybrid architecture for person re-identification. Pattern Recogn 65:238–250

    Article  Google Scholar 

  32. 32.

    Wu L, Wang Y, Gao J, Li X (2018) Where-and-when to look: deep siamese attention networks for video-based person re-identification. IEEE Trans. Multimedia:1–1

  33. 33.

    Wu L, Wang Y, Li X, Gao J (2018) Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Transactions on Cybernetics 2018 (99):1–12

    Google Scholar 

  34. 34.

    Wu L, Wang Y, Gao J, Li X (2018) Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recogn 73:275–288

    Article  Google Scholar 

  35. 35.

    Wu L, Wang Y, Li X, Gao J (2018) What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recogn 76:727–738

    Article  Google Scholar 

  36. 36.

    Wu L, Wang Y, Shao L (2019) Cycle-consistent Deep generative hashing for cross-modal retrieval. IEEE Trans Image Processing 28(4):1602–1612

    MathSciNet  Article  Google Scholar 

  37. 37.

    Wu L, Wang Y, Shao L, Wang M (2019) 3D PersonVLAD: learning deep global representations for video-based person re-identification. IEEE Trans. Neural Networks and Learning Systems

  38. 38.

    Yao X, Wand HM et al (2012) Fault detection mechanism based on adjoint state tracking in cloud computing system. J Comput Sci 35(5):856–870

    Google Scholar 

  39. 39.

    Zhen Z Research on anomaly detection strategy and algorithms aware of running environment for virtual machines in the cloud platform. Chongqing: Chongqing University. Department of Computer

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under grants 61373127, 61772252, the Young Scientists Fund of the National Natural Science Foundation of China under grants 61702242 and the Doctoral Scientific Research Foundation of Liaoning Province under grants 20170520207.

The authors would like to thank the anonymous reviewers for the valuable suggestions they provided.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jing Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, J. Anomaly detecting and ranking of the cloud computing platform by multi-view learning. Multimed Tools Appl 78, 30923–30942 (2019). https://doi.org/10.1007/s11042-019-7579-3

Download citation

Keywords

  • Anomaly detection
  • Cloud computing
  • Extreme learning machine
  • Multi-view fusing