Skip to main content
SpringerLink
Log in

Detect anomalies in cloud platforms by using network data: a review

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud computing is one of the utmost rapidly growing computing domains in today’s information technology ecosphere. Cloud computing links data and applications from various geographical locations over the internet. A large number of transactions and the secreted infrastructure in cloud computing systems have presented the research community with numerous challenges. Among these, maintaining cloud network security has emerged as a major challenge in the modern era. As well, detecting anomalous data has become a significant research area in the cloud computing domain. Anomaly detection (or outlier detection) is the identification of unusual or suspicious data that differs significantly from the majority of the data. Recently, machine learning methods have demonstrated their efficacy in anomaly detection approaches. The goal of this research study is to identify which machine learning algorithm is best suited for analyzing cloud network data on anomaly detection. This research study has led a systematic review by using scholarly articles which are published between 2017 and 2023. This review study has deliberated various techniques for anomaly detection on the cloud and different approaches for that.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Bo Wang, Q., Hua, H., Zhang, X., Tan: Yahui Nan,Rui Chen, Xinfeng Shu, Research on anomaly detection and real-time reliability evaluation with the log of cloud platform. Alexandria Eng. J. (2022). https://doi.org/10.1016/j.aej.2021.12.061. https://www.sciencedirect.com/science/article/pii/S1110016821008711

    Article  Google Scholar 

  2. Stallings, W., Brown, L.: Computer Security: Principles and Practice. Pearson, Harlow, United Kingdom (2019)

    Google Scholar 

  3. Alouffi, B., Hasnain, M., Alharbi, A., Alosaimi, W., Alyami, H., Ayaz, M.: “A Systematic Literature Review on Cloud Computing Security: Threats and Mitigation Strategies,“ in IEEE Access, vol. 9, pp. 57792–57807 (2021). https://doi.org/10.1109/ACCESS.2021.3073203

  4. Check Point Software Technology:. Cyber Security Report. Accessed: 2019. [Online]. Available: (2019). https://pages.checkpoint.com/cybersecurity-report-2019.html

  5. Elsayed, M.A., Zulkernine, M.: “PredictDeep: Security Analytics as a Service for Anomaly Detection and Prediction,“ in IEEE Access, vol. 8, pp. 45184–45197 (2020). https://doi.org/10.1109/ACCESS.2020.2977325

  6. Haji, S.H., Ameen, S.Y.: Attack and anomaly detection in IoT Networks using machine learning techniques: A review. Asian J. Res. Comput. Sci. 9(2), 30–46 (2021). https://doi.org/10.9734/ajrcos/2021/v9i230218

    Article  Google Scholar 

  7. Big Data Security and Privacy Handbook: : 100 Best Practices in Big Data Security and Privacy, Cloud Security Alliance, Seattle, WA, USA, (2016)

  8. Tanja Hagemann and Katerina Katsarou:. A Systematic Review on Anomaly Detection for Cloud Computing Environments (2020). https://doi.org/10.1145/3442536.3442550

  9. Vervaet, A.: “MoniLog: An Automated Log-Based Anomaly Detection System for Cloud Computing Infrastructures,” in 2021 IEEE 37th International Conference on Data Engineering (ICDE), Chania, Greece, 2021 pp. 2739–2743 https://doi.org/10.1109/ICDE51399.2021.00317

  10. Nedelkoski, S., Cardoso, J., Kao, O.: “Anomaly Detection from System Tracing Data Using Multimodal Deep Learning,“ 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), Milan, Italy, 2019, pp. 179–186 https://doi.org/10.1109/CLOUD.2019.00038

  11. Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: ‘‘Detecting large-scale system problems by mining console logs,’’ in Proc. ACM Symp. Operating Syst. Princ. (SOSP), pp. 117–132. (2009)

  12. Lou, J., Fu, Q., Yang, S., Xu, Y., Li, J.: ‘‘Mining invariants from console logs for system problem detection,’’ in Proc. USENIX Annu. Tech. Conf. (ATC), pp. 231–244 (2010)

  13. Lu, S., Wei, X., Li, Y., Wang, L.: ‘‘Detecting anomaly in big data system logs using convolutional neural network,’’ in Proc. IEEE 16th Int. Conf Dependable, Autonomic Secure Comput., 16th Int. Conf. Pervasive Intell. Comput., 4th Int. Conf Big Data Intell. Comput. Cyber Sci. Technol. Congr. (DASC/PiCom/DataCom/CyberSciTech), Aug. pp. 151–158 (2018)

  14. Nassif, A.B., Talib, M.A., Nasir, Q., Dakalbab, F.M.: Machine learning for Anomaly detection: A systematic review. in IEEE Access. 9, 78658–78700 (2021). https://doi.org/10.1109/ACCESS.2021.3083060

    Article  Google Scholar 

  15. Sureda Riera, T., Bermejo Higuera, J.-R., Bermejo Higuera, J., Mart´ınez Herraiz, J.-J., Sicilia Montalvo, J.-A.: Prevention and fighting against web attacks through Anomaly Detection Technology. Syst. Rev. Sustain. 12, 4945 (2020). https://doi.org/10.3390/su12124945

    Article  Google Scholar 

  16. Ozkan-Okay, M., Samet, R., Aslan, O., Gupta, D.: A comprehensive systematic literature review on Intrusion Detection Systems. in IEEE Access. 9, 157727–157760 (2021). https://doi.org/10.1109/ACCESS.2021.3129336

    Article  Google Scholar 

  17. Svacina, J., Raffety, J., Woodahl, C., Stone, B., Cerny, T., Bures, M., Shin, D., Frajtak, K., Pavel Tisnovsky: and. On Vulnerability and Security Log analysis: A Systematic Literature Review on Recent Trends (2020). https://doi.org/10.1145/3400286.3418261

  18. Jiang, D., Han, Y., Wang, X., Xu, Z., Xu, H., Chen, Z.: “A time-frequency detecting method for network traffic anomalies,” International Conference on Computational Problem- Solving, pp. 94–97. (2010)

  19. Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – A systematic literature review. ‎Inf. Softw. Technol. 51(1), 7–15 (2009)

    Article  Google Scholar 

  20. Kitchenham, B., Charters, S.M.: Guidelines for Performing Systematic Literature Reviews in Software Engineering. Keele University and Durham University (2007)

  21. Garg, S., Kaur, K., Kumar, N., Kaddoum, G., Zomaya, A.Y., Ranjan, R.: “A Hybrid Deep Learning-Based Model for Anomaly Detection in Cloud Datacenter Networks,“ in IEEE Transactions on Network and Service Management, vol. 16, no. 3, pp. 924–935, Sept (2019). https://doi.org/10.1109/TNSM.2019.2927886

  22. Alshammari, A., Aldribi, A.: Apply machine learning techniques to detect malicious network traffic in cloud computing. J. Big Data. 8, 90 (2021). https://doi.org/10.1186/s40537-021-00475-1

    Article  Google Scholar 

  23. Nedelkoski, S., Cardoso, J., Kao, O.: “Anomaly Detection from System Tracing Data Using Multimodal Deep Learning,” IEEE 12th International Conference on Cloud Computing (CLOUD), 2019, pp. 179–186 (2019). https://doi.org/10.1109/CLOUD.2019.00038

  24. Islam, M.S., Pourmajidi, W., Zhang, L., Steinbacher, J., Erwin, T., Miranskyy, A.: “Anomaly Detection in a Large-Scale Cloud Platform,” 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pp. 150–159 (2021). https://doi.org/10.1109/ICSE-SEIP52600.2021.00024

  25. Schmidt, F., Johannes, Kao, Odej: “Anomaly detection in cloud computing environments”, 2020, Technische Universit¨at Berlin https://doi.org/10.14279/depositonce-10393

  26. Salman, T., Bhamare, D., Erbad, A., Jain, R., Samaka, M.: “Machine Learning for Anomaly Detection and Categorization in Multi-Cloud Environments,” 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud), pp. 97–103 (2017). https://doi.org/10.1109/CSCloud.2017.15

  27. Hajjami, S.E., Malki, J., Berrada, M., Fourka, B.: “Machine Learning for anomaly detection. Performance study considering anomaly distribution in an imbalanced dataset,” 2020 5th International Conference on Cloud Computing and Artificial Intelligence: Technologies and Applications (CloudTech), pp. 1–8 (2020). https://doi.org/10.1109/CloudTech49835.2020.9365887

  28. Qiu, X., Dai, Y., Sun, P., Jin, X.: “PHM Technology for Memory Anomalies in Cloud Computing for IaaS,” 2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS), pp. 41–51 (2020). https://doi.org/10.1109/QRS51102.2020.00018

  29. Gerard, A., Latif, R., Latif, S., Iqbal, W., Saba, T., Gerard, N.: “MAD-Malicious Activity Detection Framework in Federated Cloud Computing,” 2020 13th International Conference on Developments in eSystems Engineering (DeSE), pp. 273–278 (2020). https://doi.org/10.1109/DeSE51703.2020.9450728

  30. Bogatinovski, J., Nedelkoski, S., Cardoso, J., Kao, O.: “Self-Supervised Anomaly Detection from Distributed Traces,” 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC), pp. 342–347 (2020). https://doi.org/10.1109/UCC48980.2020.00054

  31. Wang, W., Du, X., Shan, D., Qin, R., Wang, N.: “Cloud intrusion detection method based on stacked contractive auto-encoder and support vector machine” in IEEE Trans. Cloud Comput. https://doi.org/10.1109/TCC.2020.3001017

  32. Raj, C., Khular, L., Raj, G.: “Clustering Based Incident Handling For Anomaly Detection in Cloud Infrastructures,” 2020 10th International Conference on Cloud Computing, Data Science Engineering (Confluence), pp. 611–616 (2020). https://doi.org/10.1109/Confluence47617.2020.9058314

  33. Yuan, Y., Anu, H., Shi, W., Liang, B., Qin, B.: “Learning-Based Anomaly Cause Tracing with Synthetic Analysis of Logs from Multiple Cloud Service Components,” IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), 2019, pp. 66–71 (2019). https://doi.org/10.1109/COMPSAC.2019.00019

  34. Eltanbouly, S., Bashendy, M., AlNaimi, N., Chkirbene, Z., Erbad, A.: “Machine Learning Techniques for Network Anomaly Detection: A Survey,“ 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar, pp. 156–162 (2020). https://doi.org/10.1109/ICIoT48696.2020.9089465

  35. Sivapalan, G., Nundy, K.K., James, A., Cardiff, B., John, D.: “Interpretable rule mining for real-time ECG anomaly detection in IoT Edge Sensors,“ in IEEE Internet of Things Journal https://doi.org/10.1109/JIOT.2023.3260722

  36. He, Z., et al.: “A Spatiotemporal Deep Learning Approach for Unsupervised Anomaly Detection in Cloud Systems,“ in IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 4, pp. 1705–1719 (2023). https://doi.org/10.1109/TNNLS.2020.3027736

  37. Wang, Y., Du, X., Lu, Z., Duan, Q., Wu, J.: “Improved LSTM-Based Time-Series Anomaly Detection in Rail Transit Operation Environments,“ in IEEE Transactions on Industrial Informatics, vol. 18, no. 12, pp. 9027–9036 (2022). https://doi.org/10.1109/TII.2022.3164087

  38. Choi, K., Yi, J., Park, C., Yoon, S.: “Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines,“ in IEEE Access, vol. 9, pp. 120043–120065 (2021). https://doi.org/10.1109/ACCESS.2021.3107975

  39. Tang, S., Gu, Z., Yang, Q., Fu, S.: “Smart Home IoT Anomaly Detection based on Ensemble Model Learning From Heterogeneous Data,“ 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, pp. 4185–4190 (2019). https://doi.org/10.1109/BigData47090.2019.9006249

  40. Aljamal, I., Tekeoğlu, A., Bekiroglu, K., Sengupta, S.: “Hybrid Intrusion Detection System Using Machine Learning Techniques in Cloud Computing Environments,“ 2019 IEEE 17th International Conference on Software Engineering Research, Management and Applications (SERA), Honolulu, HI, USA, pp. 84–89 (2019). https://doi.org/10.1109/SERA.2019.8886794

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Computing and Information Systems, Faculty of Computing, Sabaragamuwa University of Sri Lanka, Belihuloya, Sri Lanka

    M. P. G. K. Jayaweera

  2. Faculty of Graduate Studies, Sabaragamuwa University of Sri Lanka, Belihuloya, Sri Lanka

    W. M. C. J. T. Kithulwatta

  3. Department of Information and Communication Technology, Faculty of Technological Studies, Uva Wellassa University of Sri Lanka, Badulla, Sri Lanka

    W. M. C. J. T. Kithulwatta

  4. Department of Physical Sciences and Technology, Faculty of Applied Sciences, Sabaragamuwa University of Sri Lanka, Belihuloya, Sri Lanka

    R. M. K. T. Rathnayaka

Authors
  1. M. P. G. K. Jayaweera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. M. C. J. T. Kithulwatta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. M. K. T. Rathnayaka
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author wrote the main manuscript text and prepared all figures. All authors reviewed and approved the manuscript.

Corresponding author

Correspondence to W. M. C. J. T. Kithulwatta.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jayaweera, M.P.G.K., Kithulwatta, W.M.C.J.T. & Rathnayaka, R.M.K.T. Detect anomalies in cloud platforms by using network data: a review. Cluster Comput 26, 3279–3289 (2023). https://doi.org/10.1007/s10586-023-04055-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-023-04055-1

Keywords

Search

Navigation