Protecting a Moving Target: Addressing Web Application Concept Drift

  • Federico Maggi
  • William Robertson
  • Christopher Kruegel
  • Giovanni Vigna
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5758)


Because of the ad hoc nature of web applications, intrusion detection systems that leverage machine learning techniques are particularly well-suited for protecting websites. The reason is that these systems are able to characterize the applications’ normal behavior in an automated fashion. However, anomaly-based detectors for web applications suffer from false positives that are generated whenever the applications being protected change. These false positives need to be analyzed by the security officer who then has to interact with the web application developers to confirm that the reported alerts were indeed erroneous detections.

In this paper, we propose a novel technique for the automatic detection of changes in web applications, which allows for the selective retraining of the affected anomaly detection models. We demonstrate that, by correctly identifying legitimate changes in web applications, we can reduce false positives and allow for the automated retraining of the anomaly models.

We have evaluated our approach by analyzing a number of real-world applications. Our analysis shows that web applications indeed change substantially over time, and that our technique is able to effectively detect changes and automatically adapt the anomaly detection models to the new structure of the changed web applications.


Anomaly Detection Web Application Security Concept Drift Machine Learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Turner, D., Fossi, M., Johnson, E., Mark, T., Blackbird, J., Entwise, S., Low, M.K., McKinney, D., Wueest, C.: Symantec Global Internet Security Threat Report – Trends for July-December 2007. Technical Report XII, Symantec Corporation (April 2008)Google Scholar
  2. 2.
    Shezaf, O., Grossman, J., Auger, R.: Web Hacking Incidents Database (March 2009),
  3. 3.
    Open Security Foundation: DLDOS: Data Loss Database – Open Source (March 2009),
  4. 4.
    Cho, S., Cha, S.: SAD: web session anomaly detection based on parameter estimation. In: Computers & Security, vol. 23, pp. 312–319 (2004)Google Scholar
  5. 5.
    Kruegel, C., Robertson, W., Vigna, G.: A Multi-model Approach to the Detection of Web-based Attacks. Journal of Computer Networks 48(5), 717–738 (2005)CrossRefGoogle Scholar
  6. 6.
    Robertson, W., Vigna, G., Kruegel, C., Kemmerer, R.A.: Using Generalization and Characterization Techniques in the Anomaly-based Detection of Web Attacks. In: Proceedings of the Network and Distributed System Security Symposium (NDSS 2006), San Diego, CA, USA (February 2006)Google Scholar
  7. 7.
    Guangmin, L.: Modeling Unknown Web Attacks in Network Anomaly Detection. In: Proceedings of the 3rd International Conference on Convergence and Hybrid Information Technology (ICCIT 2008), Washington, DC, USA, pp. 112–116. IEEE Computer Society, Los Alamitos (2008)CrossRefGoogle Scholar
  8. 8.
    Zanero, S., Criscione, C.: Masibty: A Web Application Firewall based on Anomaly Detection. In: DeepSec - In-depth security conference (November 2008)Google Scholar
  9. 9.
    Citrix Systems, Inc.: Citrix Application Firewall (January 2009),
  10. 10.
    F5 Networks, Inc.: BIG-IP Application Security Manager (January 2009),
  11. 11.
    Breach Security, Inc.: Breach WebDefend (January 2009),
  12. 12.
    Axelsson, S.: The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection. In: Proceedings of the ACM Conference on Computer and Communications Security (CCS 1999), pp. 1–7. ACM, New York (1999)Google Scholar
  13. 13.
    Frias-Martinez, V., Stolfo, S.J., Keromytis, A.D.: Behavior-Profile Clustering for False Alert Reduction in Anomaly Detection Sensors. In: Proceedings of the Annual Computer Security Applications Conference (ACSAC 2008), Anaheim, CA, USA (December 2008)Google Scholar
  14. 14.
    Escalante, H.J., Fuentes, O.: Kernel Methods for Anomaly Detection and Noise Elimination. In: Proceedings of the International Conference on Computing (CORE 2006), Mexico City, Mexico, pp. 69–80 (2006)Google Scholar
  15. 15.
    Kim, S.i., Nwanze, N.: Noise-Resistant Payload Anomaly Detection for Network Intrusion Detection Systems. In: Proceedings of the Performance, Computing and Communications Conference (IPCCC 2008), Austin, TX, USA, pp. 517–523. IEEE Computer Society, Los Alamitos (2008)CrossRefGoogle Scholar
  16. 16.
    Cretu, G.F., Stavrou, A., Locasto, M.E., Stolfo, S.J., Keromytis, A.D.: Casting out Demons: Sanitizing Training Data for Anomaly Sensors. In: Proceedings of the 2008 IEEE Symposium on Security and Privacy (S&P 2008), Oakland, CA, USA, pp. 81–95. IEEE Computer Society, Los Alamitos (2008)CrossRefGoogle Scholar
  17. 17.
    Song, Y., Stolfo, S., Keromytis, A.: Spectrogram: A Mixture-of-Markov-Chains Model for Anomaly Detection in Web Traffic. In: Proc. of the 16th Annual Network and Distributed System Security Symposium, NDSS (2009)Google Scholar
  18. 18.
    Schlimmer, J., Granger, R.: Beyond incremental processing: Tracking concept drift. In: Proceedings of the Fifth National Conference on Artificial Intelligence, vol. 1, pp. 502–507 (1986)Google Scholar
  19. 19.
    Kolter, J., Maloof, M.: Dynamic weighted majority: An ensemble method for drifting concepts. The Journal of Machine Learning Research 8, 2755–2790 (2007)zbMATHGoogle Scholar
  20. 20.
    Hansen, R.: (RSnake): XSS (Cross Site Scripting) Cheat Sheet (June 2009),
  21. 21.
    Mavituna, F.: SQL Injection Cheat Sheet (June 2009),
  22. 22.
    Denning, D.E.: An Intrusion-Detection Model. IEEE Transactions on Software Engineering 13(2), 222–232 (1987)CrossRefGoogle Scholar
  23. 23.
    Lee, W., Stolfo, S.J.: A Framework for Constructing Features and Models for Intrusion Detection Systems. ACM Transactions on Information and System Security 3(4), 227–261 (2000)CrossRefGoogle Scholar
  24. 24.
    Mutz, D., Valeur, F., Vigna, G., Kruegel, C.: Anomaly system call detection. ACM Transactions on Information and System Security 9(1), 61–93 (2006)CrossRefGoogle Scholar
  25. 25.
    Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A Sense of Self for Unix Processes. In: Proceedings of the IEEE Symposium on Security and Privacy (S&P 1996), Oakland, CA, USA, pp. 120–128. IEEE Computer Society, Los Alamitos (1996)CrossRefGoogle Scholar
  26. 26.
    Wagner, D., Dean, D.: Intrusion Detection via Static Analysis. In: Proceedings of the IEEE Symposium on Security and Privacy (S&P 2001), Oakland, CA, USA, pp. 156–168. IEEE Computer Society, Los Alamitos (2001)Google Scholar
  27. 27.
    Maggi, F., Matteucci, M., Zanero, S.: Detecting intrusions through system call sequence and argument analysis. IEEE Transactions on Dependable and Secure Computing 99(1) (5555)Google Scholar
  28. 28.
    Wang, K., Stolfo, S.J.: Anomalous Payload-based Network Intrusion Detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 203–222. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  29. 29.
    Wang, K., Parekh, J.J., Stolfo, S.J.: Anagram: A Content Anomaly Detector Resistant to Mimicry Attack. In: Zamboni, D., Krügel, C. (eds.) RAID 2006. LNCS, vol. 4219, pp. 226–248. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  30. 30.
    Zanero, S.: Analyzing TCP traffic patterns using self organizing maps. In: Roli, F., Vitulano, S. (eds.) ICIAP 2005. LNCS, vol. 3617, pp. 83–90. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  31. 31.
    Kruegel, C., Mutz, D., Robertson, W., Valeur, F.: Bayesian Event Classification for Intrusion Detection. In: Proceedings of the Annual Computer Security Applications Conference (ACSAC 2003), Las Vegas, NV, USA. IEEE Computer Society, Los Alamitos (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Federico Maggi
    • 1
  • William Robertson
    • 1
  • Christopher Kruegel
    • 1
  • Giovanni Vigna
    • 1
  1. 1.Computer Security GroupUC Santa BarbaraUSA

Personalised recommendations