Abstract
Recently web applications have been widely used in enterprises to assist employees in providing effective and efficient business processes. Forecasting upcoming web events in enterprise web applications can be beneficial in many ways, such as efficient caching and recommendation. In this paper, we present a web event forecasting approach, DeepEvent, in enterprise web applications for better anomaly detection. DeepEvent includes three key features: web-specific neural networks to take into account the characteristics of sequential web events, self-supervised learning techniques to overcome the scarcity of labeled data, and sequence embedding techniques to integrate contextual events and capture dependencies among web events. We evaluate DeepEvent on web events collected from six real-world enterprise web applications. Our experimental results demonstrate that DeepEvent is effective in forecasting sequential web events and detecting web based anomalies. DeepEvent provides a context-based system for researchers and practitioners to better forecast web events with situational awareness.
Keywords
- Anomaly detection
- Event forecasting
- Self-supervised learning
- Neural networks
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptions
Notes
- 1.
- 2.
We detect randomness in URIs based on a gibberish detection tool (https://github.com/rrenaud/Gibberish-Detector).
References
Awad, M., Khan, L., Thuraisingham, B.: Predicting www surfing using multiple evidence combination. VLDB J. 17(3), 401–417 (2008)
Awad, M.A., Khalil, I.: Prediction of user’s web-browsing behavior: application of Markov model. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(4), 1131–1142 (2012)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS Deep Learning and Representation Learning Workshop (2014)
Da, N.T., Hanh, T., Duy, P.H.: A survey of webpage access prediction. In: 2018 International Conference on Advanced Technologies for Communications (ATC), pp. 315–320. IEEE (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Dong, Y., Zhang, Y.: Adaptively detecting malicious queries in web attacks. arXiv preprint arXiv:1701.07774 (2017)
Du, M., Li, F., Zheng, G., Srikumar, V.: DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS) (2017). https://doi.org/10.1145/3133956.3134015
El-Sayed, M., Ruiz, C., Rundensteiner, E.A.: Fs-Miner: efficient and incremental mining of frequent sequence patterns in web logs. In: Proceedings of the 6th Annual ACM International Workshop on Web Information and Data Management, pp. 128–135 (2004)
Estevez-Tapiador, J.M., Garcia-Teodoro, P., Diaz-Verdejo, J.: Measuring normality in http traffic for anomaly-based intrusion detection. Comput. Netw. 45, 175–193 (2004)
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J., et al.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies (2001)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6, 429–449 (2002)
Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://ieeexplore.ieee.org/abstract/document/9086055
Juvonen, A., Sipola, T.: Adaptive framework for network traffic classification using dimensionality reduction and clustering. In: 2012 IV International Congress on Ultra Modern Telecommunications and Control Systems, pp. 274–279. IEEE (2012)
Juvonen, A., Sipola, T., Häämäläinen, T.: Online anomaly detection using dimensionality reduction techniques for HTTP log analysis. Comput. Netw. 91, 46–56 (2015)
Kemeny, J.G., Snell, J.L.: Markov Chains. Springer, New York (1976)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations (2015)
Kruegel, C., Vigna, G.: Anomaly detection of web-based attacks. In: Proceedings of the 10th ACM Conference on Computer and Communications Security, pp. 251–261. ACM (2003)
Kruegel, C., Vigna, G., Robertson, W.: A multi-model approach to the detection of web-based attacks. Comput. Netw. 48(5), 717–738 (2005)
Liang, J., Zhao, W., Ye, W.: Anomaly-based web attack detection: a deep learning approach. In: Proceedings of the 2017 VI International Conference on Network, Communication and Computing, pp. 80–85. ACM (2017)
Liu, T., Qi, Y., Shi, L., Yan, J.: Locate-then-detect: real-time web attack detection via attention-based deep neural networks. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 4725–4731. AAAI Press (2019)
Oprea, A., Li, Z., Norris, R., Bowers, K.: MADE: security analytics for enterprise threat detection. In: Proceedings of the 34th Annual Computer Security Applications Conference, pp. 124–136. ACM (2018)
OWASP: Buffer overflow. https://www.owasp.org/index.php/Buffer_Overflow
OWASP: Crlf injection. https://www.owasp.org/index.php/CRLF_Injection
OWASP: Cross-site scripting (xss). https://www.owasp.org/index.php/Cross-site_Scripting_(XSS)
OWASP: Server-side includes (ssi) injection. https://www.owasp.org/index.php/Server-Side_Includes_(SSI)_Injection
OWASP: Sql injection. https://www.owasp.org/index.php/SQL_Injection
OWASP: Zed attack proxy project. https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
Pham, T.S., Hoang, T.H., Vu, V.C.: Machine learning techniques for web intrusion detection - a comparison. In: Proceedings of the 8th International Conference on Knowledge and Systems Engineering, pp. 291–297. IEEE (2016)
Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th International Conference on Machine Learning, pp. 759–766. ACM (2007)
Robertson, W., Vigna, G., Kruegel, C., Kemmerer, R.A., et al.: Using generalization and characterization techniques in the anomaly-based detection of web attacks. In: NDSS (2006)
Sharif, M., Urakawa, J., Christin, N., Kubota, A., Yamada, A.: Predicting impending exposure to malicious content from user behavior. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1487–1501. ACM (2018)
Shen, Y., Mariconti, E., Vervier, P.A., Stringhini, G.: Tiresias: predicting security events through deep learning. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS) (2018). https://doi.org/10.1145/3243734.3243811
Sipola, T., Juvonen, A., Lehtonen, J.: Anomaly detection from network logs using diffusion maps. In: Iliadis, L., Jayne, C. (eds.) AIAI/EANN -2011. IAICT, vol. 363, pp. 172–181. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23957-1_20
Su, Z., Yang, Q., Lu, Y., Zhang, H.: WhatNext: a prediction system for web requests using n-gram sequence models. In: Proceedings of the First International Conference on Web Information Systems Engineering, vol. 1, pp. 214–221. IEEE (2000)
Yang, Q., Zhang, H.H., Li, T.: Mining web logs for prediction models in www caching and prefetching. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 473–478 (2001)
Yu, Y., Yan, H., Guan, H., Zhou, H.: DeepHTTP: semantics-structure model with attention for anomalous HTTP traffic detection and pattern mining. arXiv preprint arXiv:1810.12751 (2018)
Zolotukhin, M., Hämäläinen, T., Kokkonen, T., Siltanen, J.: Analysis of http requests for anomaly detection of web attacks. In: Proceedings of the 12th International Conference on Dependable, Autonomic and Secure Computing. IEEE (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Yuan, X., Ding, L., Salem, M.B., Li, X., Wu, D. (2020). Connecting Web Event Forecasting with Anomaly Detection: A Case Study on Enterprise Web Applications Using Self-supervised Neural Networks. In: Park, N., Sun, K., Foresti, S., Butler, K., Saxena, N. (eds) Security and Privacy in Communication Networks. SecureComm 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 335. Springer, Cham. https://doi.org/10.1007/978-3-030-63086-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-63086-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63085-0
Online ISBN: 978-3-030-63086-7
eBook Packages: Computer ScienceComputer Science (R0)