Semi-Supervised Classification System for the Detection of Advanced Persistent Threats

  • Fàtima Barceló-Rico
  • Anna I. Esparcia-AlcázarEmail author
  • Antonio Villalón-Huerta
Part of the Studies in Computational Intelligence book series (SCI, volume 621)


Advanced Persistent Threats (APTs) are a highly sophisticated type of cyber attack usually aimed at large and powerful organisations. Human expert knowledge, coded as rules, can be used to detect these attacks when they attempt to extract information of their victim hidden within normal http traffic. Often, experts base their decisions on anomaly detection techniques, working under the hypothesis that APTs generate traffic that differs from normal traffic. In this work we aim at developing classifiers that can help human experts to find APTs. We first define an anomaly score metric to select the most anomalous subset of traffic data; then the human expert labels the instances within this set; finally we train a classifier using both labelled and unlabelled data. Three computational intelligence methods were employed to train classifiers, namely genetic programming, decision trees and support vector machines. The results show their potential in the fight against APTs.


Advanced Persistent Threat Anomaly Detection Semi-supervised classification Genetic programming Decision trees Support vector machines 


  1. 1.
    Virvilis, N., Gritzalis, D., Apostolopoulos, T.: Trusted computing versus advanced persistent threats: can a defender win this game?. In: IEEE 10th International Conference on Ubiquitous Intelligence and Computing and 10th International Conference on Autonomic and Trusted Computing (UIC/ATC) (2013)Google Scholar
  2. 2.
    Sullivan, D.: Beyond the hype: advanced persistent threats. Technical Report, TrendMICRO, 2011Google Scholar
  3. 3.
    Lemos, R.: Stuxnet attack more effective than bombs (2011).
  4. 4.
    Symantec, W32.duqu—the precursor to the next stuxnet. (2011).
  5. 5.
    Bencsath, B., Pek, G., Buttyan, L., Felegyhazi, M.: The cousins of stuxnet: Duqu, flame, and gauss. Future Internet 4(4), 971–1003 (2012)CrossRefGoogle Scholar
  6. 6.
    Labs, K.: “Red october” diplomatic cyber attacks investigation. (2013).
  7. 7.
    Tivadar, M., Balazs, B., Istrate, C.: A closer look at miniduke. (2013).
  8. 8.
    Binde, B., McRee, R., OConnor, T.: Assessing outbound traffic to uncover advanced persistent threads, Technical Report, SANS Technology Institute, 2011Google Scholar
  9. 9.
    Lee, M., Lewis, D.: Clustering disparate attacks: Mapping the activities of the advanced persistent threat. In: Virus Bulletin Conference (2011)Google Scholar
  10. 10.
    Cutler, T.: The anatomy of an advanced persistent threat (2010).
  11. 11.
    Molok, N., Chang, S., Ahmad, A.: Information leakage through online social networking: opening the doorway for advanced persistence threats. In: Australian Information Security Management Conference (2010)Google Scholar
  12. 12.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 15 (2009)CrossRefGoogle Scholar
  13. 13.
    Kumar, V.: Parallel and distributed computing for cybersecurity. IEEE Distrib. Syst. 6(10), 1–9 (2005)CrossRefGoogle Scholar
  14. 14.
    Spence, C., Parra, L., Sajda, P.: Detection, synthesis and compression in mammographic image analysis with a hierarchical image probability model. In: IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (2001)Google Scholar
  15. 15.
    Aleskerov, E., Freisleben, B., Rao, B.: Cardwatch: A neural network based database mining system for credit card fraud detection. In: IEEE Conference on Computational Intelligence for Financial Engineering (1997)Google Scholar
  16. 16.
    Fujimaki, R. Yairi, T., Machida, K.: An approach to spacecraft anomaly detection problem using kernel feature space. In: 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (2005)Google Scholar
  17. 17.
    Duda, R.O., Hart, P., Stork, D.: Pattern Classification, Wiley-Interscience (2001)Google Scholar
  18. 18.
    Stefano, C.D., Sansone, C., Vento, M.: To reject or not to reject: that is the question: an answer in the case of neural classifiers. IEEE Trans. Syst. Man Cybern. 30(1), 84–94 (2000)CrossRefGoogle Scholar
  19. 19.
    Barbara, D., Wu, N., Jajodia, S.: Detecting novel network intrusions using bayes estimators. In: 1st SIAM International Conference on Data Mining (2001)Google Scholar
  20. 20.
    Eskin, E., Arnold, A., Prerau, M., Portnoy, L., Stolfo, S.: Ageometric framework for unsupervised anomaly detection. In: Conference on Applications of Data Mining in Computer Security, Kluwer Academics (2002)Google Scholar
  21. 21.
    Tan, P., Steinbach, M.K.: Introduction to Data Mining, Addison-Wesley (2005)Google Scholar
  22. 22.
    Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets, In: CMSIGMOD International Conference on Management of Data (2000)Google Scholar
  23. 23.
    Breunig, M., Kriegel, H. Ng, R. Sander, J.: Lof: Identifying density-based local outliers. In: ACM SIGMOD International Conference on Management of Data (2000)Google Scholar
  24. 24.
    Guha, S., Rastogi, R., Shim, K.: Rock: A robust clustering algorithm for categorical attributes. In: IEEE 15th International Conference on Data Engineering. vol. 25 no. 5 (1999)Google Scholar
  25. 25.
    Eskin, E.: Anomaly detection over noisy data using learned probability distributions, In: 17th International Conference on Machine Learning (2000)Google Scholar
  26. 26.
    Desforges, M., Jacob, P., Cooper, J.: Applications of probability density estimation to the detection of abnormal conditions in engineering, institution of Mechanical Engineers. Part C: J. Mech. Eng. Sci. 212(8), 687–703 (1998)Google Scholar
  27. 27.
    Keogh, E., Lonardi, S., Ratanamahatana, C.: Towards parameter-free data mining. In: 10th ACMSIG-KDD International Conference on Knowledge Discovery and Data Mining (2004)Google Scholar
  28. 28.
    Agovic, A., Banerjee, A., Ganguly, A.: Ch6 Anomaly detection in transportation corridors using manifold embedding. Knowledge Discovery from Sensor Data (2007)Google Scholar
  29. 29.
    Ingham, K., Inoue, H.: Comparing anomaly detection techniques for http. Recent Advances in Intrusion Detection. Springer, Berlin (2007)Google Scholar
  30. 30.
    Kruegel, C., Vigna, G.: Anomaly detection of web-based attacks. In: 10th ACM Conference on Computer and Communications Security (2003)Google Scholar
  31. 31.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  32. 32.
    Espejo, P., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 40(2), 121–144 (2010)CrossRefGoogle Scholar
  33. 33.
    Lotz, M.: Modelling of process systems with genetic programming. Master’s thesis, University of Stellenbosch (2006)Google Scholar
  34. 34.
    Banzhaf, W., Nordin, P., Keller, R., Francone, F.: Genetic Programming: An Introduction, vol. 1. Morgan Kaufmann, San Francisco (1998)CrossRefzbMATHGoogle Scholar
  35. 35.
    Silva, S.: GPLAB A Genetic Programming Toolbox for MATLAB, ECOS - Evolutionary and Complex Systems Group University of Coimbra Portugal, version 3 ednGoogle Scholar
  36. 36.
    Safavian, S., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Breiman, L., Friedman, J., Stone, C., Olshen, R.: Classification and Regression Trees. CRC press, Boca Raton (1984)zbMATHGoogle Scholar
  38. 38.
    Timofeev, R.: Classification and regression trees (cart) theory and applications. Master’s thesis, Humboldt University, Berlin (2004)Google Scholar
  39. 39.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  40. 40.
    Hearst, M., Dumais, S., Osman, E., Platt, J., Scholkopf, B.: Support vector machines. Intell. Syst. Appl. IEEE 13(4), 18–28 (1998)CrossRefGoogle Scholar
  41. 41.
    Burges, C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)CrossRefGoogle Scholar
  42. 42.
    Alfaro-Cid, E., Sharman, K., Esparcia-Alcazar, A.: A genetic programming approach for bankruptcy prediction using a highly unbalanced database. Applications of Evolutionary Computing, pp. 169–178. Springer, Berlin (2007)Google Scholar
  43. 43.
    Thierens, D.: Scalability problems of simple genetic algorithms. Evol. Comput. 7(4), 331–352 (1999)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Fàtima Barceló-Rico
    • 1
  • Anna I. Esparcia-Alcázar
    • 1
    • 2
    Email author
  • Antonio Villalón-Huerta
    • 1
  1. 1.S2 GrupoValenciaSpain
  2. 2.Universitat Politècnica de ValènciaValenciaSpain

Personalised recommendations