Evaluating Host-Based Anomaly Detection Systems: Application of the Frequency-Based Algorithms to ADFA-LD

  • Miao Xie
  • Jiankun Hu
  • Xinghuo Yu
  • Elizabeth Chang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8792)


ADFA Linux data set (ADFA-LD) is released recently for substituting the existing benchmark data sets in the area of host-based anomaly detection which have lost most of their relevance to modern computer systems. ADFA-LD is composed of thousands of system call traces collected from a contemporary Linux local server, with six types of up-to-date cyber attack involved. Previously, we have conducted a preliminary analysis of ADFA-LD, and shown that the frequency-based algorithms can be realised at a cheaper computational cost in contrast with the short sequence-based algorithms, while achieving an acceptable performance. In this paper, we further exploit the potential of the frequency-based algorithms, in attempts to reduce the dimension of the frequency vectors and identify the optimal distance functions. Two typical frequency-based algorithms, i.e., k-nearest neighbour (kNN) and k-means clustering (kMC), are applied to validate the effectiveness and efficiency.


host-based intrusion detection system (HIDS) Unix system call 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Stavroulakis, P., Stamp, M.: Handbook of information and communication security. Springer (2010)Google Scholar
  2. 2.
  3. 3.
  4. 4.
    Forrest, S., Hofmeyr, S., Somayaji, A., Longstaff, T.A.: A sense of self for Unix processes. In: Proceedings of the 1996 IEEE Symposium on Security and Privacy, pp. 120–128 (1996)Google Scholar
  5. 5.
    Kosoresow, A.P., Hofmeyer, S.A.: Intrusion detection via system call traces. IEEE Software 14, 35–42 (1997)CrossRefGoogle Scholar
  6. 6.
    Forrest, S., Hofmeyr, S., Somayaji, A.: The Evolution of System-Call Monitoring. In: Annual Computer Security Applications Conference, ACSAC 2008, pp. 418–430 (2008)Google Scholar
  7. 7.
    Eskin, E., Wenke, L., Stolfo, S.J.: Modeling system calls for intrusion detection with dynamic window sizes. In: Proceedings of the DARPA Information Survivability Conference Exposition II, DISCEX 2001, pp. 165–175 (2001)Google Scholar
  8. 8.
    Hoang, X.D., Hu, J.: An efficient hidden Markov model training scheme for anomaly intrusion detection of server applications based on system calls. In: Proceedings of the 12th IEEE International Conference on Networks (ICON 2004), pp. 470–474 (2004)Google Scholar
  9. 9.
    Hoang, X.D., Hu, J., Bertok, P.: A program-based anomaly intrusion detection scheme using multiple detection engines and fuzzy inference. Journal of Network and Computer Applications 32, 1219–1228 (2009)CrossRefGoogle Scholar
  10. 10.
    Creech, G., Hu, J.: Generation of a new IDS test dataset: Time to retire the KDD collection. In: 2013 IEEE Wireless Communications and Networking Conference (WCNC), pp. 4487–4492 (2013)Google Scholar
  11. 11.
    Creech, G., Hu, J.: A Semantic Approach to Host-Based Intrusion Detection Systems Using Contiguous and Discontiguous System Call Patterns. IEEE Transactions on Computers 63, 807–819 (2014)CrossRefGoogle Scholar
  12. 12.
    Liao, Y., Vemuri, V.R.: Use of K-nearest neighbor classifier for intrusion detection. Computers & Security 21, 439–448 (2002)CrossRefGoogle Scholar
  13. 13.
    Chen, W.-H., Hsu, S.-H., Shen, H.-P.: Application of SVM and ANN for intrusion detection. Computers & Operations Research 32, 2617–2634 (2005)CrossRefzbMATHGoogle Scholar
  14. 14.
    Sharma, A., Pujari, A.K., Paliwal, K.K.: Intrusion detection using text processing techniques with a kernel based similarity measure. Computers & Security 26, 488–495 (2007)CrossRefGoogle Scholar
  15. 15.
    Xie, M., Hu, J.: Evaluating host-based anomaly detection systems: A preliminary analysis of ADFA-LD. In: 2013 6th International Congress on Image and Signal Processing (CISP), pp. 1711–1716 (2013)Google Scholar
  16. 16.
  17. 17.
    Jolliffe, I.: Principal component analysis. Wiley Online Library (2005)Google Scholar
  18. 18.
    Xie, M., Han, S., Tian, B.: Highly Efficient Distance-Based Anomaly Detection through Univariate with PCA in Wireless Sensor Networks. In: 2011 IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pp. 564–571 (2011)Google Scholar
  19. 19.
    Xie, M., Hu, J., Tian, B.: Histogram-Based Online Anomaly Detection in Hierarchical Wireless Sensor Networks. In: 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pp. 751–759 (2012)Google Scholar
  20. 20.
    Xie, M., Hu, J., Han, S., Chen, H.-H.: Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks. IEEE Transactions on Parallel and Distributed Systems 24, 1661–1670 (2013)CrossRefGoogle Scholar
  21. 21.
    Hu, J., Gingrich, D., Sentosa, A.: A k-Nearest Neighbor Approach for User Authentication through Biometric Keystroke Dynamics. In: IEEE International Conference on Communications, ICC 2008, pp. 1556–1560 (2008)Google Scholar
  22. 22.
    Hartigan, J.A., Wong, M.A.: Algorithm AS 136: A k-means clustering algorithm. Applied Statistics, 100–108 (1979)Google Scholar
  23. 23.
    Mahmood, A.N., Hu, J., Tari, Z., Leckie, C.: Critical infrastructure protection: Resource efficient sampling to improve detection of less frequent patterns in network traffic. Journal of Network and Computer Applications 33, 491–502 (2010)CrossRefGoogle Scholar
  24. 24.
    Xi, K., Tang, Y., Hu, J.: Correlation keystroke verification scheme for user access control in cloud computing environment. The Computer Journal 54, 1632–1644 (2011)CrossRefGoogle Scholar
  25. 25.
    Lloyd, S.: Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 129–137 (1982)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Miao Xie
    • 1
  • Jiankun Hu
    • 1
  • Xinghuo Yu
    • 2
  • Elizabeth Chang
    • 1
  1. 1.UNSW CanberraCanberraAustralia
  2. 2.RMIT UniversityMelbourneAustralia

Personalised recommendations