Advertisement

A Survey on the Development of Self-Organizing Maps for Unsupervised Intrusion Detection

  • Xiaofei QuEmail author
  • Lin Yang
  • Kai Guo
  • Linru Ma
  • Meng Sun
  • Mingxing Ke
  • Mu Li
Article
  • 15 Downloads

Abstract

This paper describes a focused literature survey of self-organizing maps (SOM) in support of intrusion detection. Specifically, the SOM architecture can be divided into two categories, i.e., static-layered architectures and dynamic-layered architectures. The former one, Hierarchical Self-Organizing Maps (HSOM), can effectively reduce the computational overheads and efficiently represent the hierarchy of data. The latter one, Growing Hierarchical Self-Organizing Maps (GHSOM), is quite effective for online intrusion detection with low computing latency, dynamic self-adaptability, and self-learning. The ultimate goal of SOM architecture is to accurately represent the topological relationship of data to identify any anomalous attack. The overall goal of this survey is to comprehensively compare the primitive components and properties of SOM-based intrusion detection. By comparing with the two SOM-based intrusion detection systems, we can clearly understand the existing challenges of SOM-based intrusion detection systems and indicate the future research directions.

Keywords

Self organizing map (SOM) Hierarchical self-organizing map (HSOM) Growing hierarchical self-organizing map (GHSOM) Intrusion detection system (IDS) 

Notes

References

  1. 1.
    Denning DE (1987) An intrusion detection model. IEEE Trans Softw Eng (Special issue on Computer Security and Privacy) 13(2):222–232Google Scholar
  2. 2.
    Wu SX, Banzhaf W (2010) The use of computational intelligence in intrusion detection systems: a review. Appl Soft Comput 10:1–35Google Scholar
  3. 3.
    De la Hoz E, De la Hoz E, Ortiz A, Ortega J, Prieto B (2015) PCA filtering and probabilistic SOM for network intrusion detection. Advances in Computational Intelligence in Elsevier Neurocomputing 164:71–81Google Scholar
  4. 4.
    Zhisheng W, Xiaobing X (2013) An improved adaptive self-organizing map. Comput Eng Appl 49(17):112–115Google Scholar
  5. 5.
    Hoglund AJ, Hatonen K, Sorvari AS (2000) A computer host-based user anomaly detction system using the self-organizing map. Proceedings of the IEEEINNS-ENNS International Joint Conference on Neural Networks (IJCNN00) 5:24–27Google Scholar
  6. 6.
    Lichodzijewski P, Nur Zincir-Heywood A, Heywood MI (2002) Host-based intrusion detection using self-organizing maps. The IEEE World Congress on Computational Intelligence International Joint Conference on Neural Networks (IJCNN02)Google Scholar
  7. 7.
    Kayacik HG, Zincir-Heywood AN, Heywood MI (2003) On the capability of a SOM based intrusion detection system. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN03), vol 3, pp 20–24Google Scholar
  8. 8.
    Kayacik HG, Zincir-Heywood AN, Heywood MI (2007) A hierarchical SOM-based intrusion detection system. Eng Appl Artif Intell 20(4):439–451Google Scholar
  9. 9.
    Rauber A, Merkl D, Dittenbach M (2002) The growing hierarchical self-organizingmap:exploratory analysis of high-dimensional data. IEEE Trans Neural Networks 13:1331–1341zbMATHGoogle Scholar
  10. 10.
    dela Hoza E, dela Hoza E, Ortiz A, Ortega J, Martinez-Alvarez A (2014) Feature selection by multi-objective optimisation: Application to network anomaly detection by hierarchical self-organising maps. Knowl-Based Syst 71:322–338Google Scholar
  11. 11.
    Zanero S, Savaresi SM (2013) Unsupervised learning techniques for an intrusion detection system. Proceedings of the ACM symposium on applied computing 49(17):112–115Google Scholar
  12. 12.
    Zanero S (2004) Improving self organizing map performance for network intrusion detection. In: SDM 2005 Workshop on clustering high dimensional data and its applications, submitted for publicationGoogle Scholar
  13. 13.
    Zanero S (2005) Analyzing TCP traffic patterns using self organizing maps. In: Roli F., Vitulano S. (eds) International conference on image analysis and processing (ICIAP05), Cagliari, Italy, 6C8 September 2005, volume 3617 of Lecture Notes in Computer Science. Springer, Berlin, pp 83–90Google Scholar
  14. 14.
    Zanero S (May 2008) Unsupervised learning algorithms for intrusion detection. PhD dissertation, Politecnico di Milano T.U.Google Scholar
  15. 15.
    Palomo EJ, Domnguez E, Luque RM, Munoz J (2009) Network security using growing hierarchical self-organizing maps. In: Proceedings of the 9th international conference on adaptive and natural computing algorithms, ICANNGA09. Springer, Berlin, pp 130–139Google Scholar
  16. 16.
    Yang Y, Jiang D, Xia M (2010) Using improved GHSOM for intrusion detection. Journal of Information Assurance and Security 5:232–239Google Scholar
  17. 17.
    Ippoliti D, Zhou X (2012) A-GHSOM: an adaptive growing hierarchical self organizing map for network anomaly detection. J Parallel Distr Comput 72(12):1576–1590Google Scholar
  18. 18.
    Fox KL, Henning RR, Reed JH (1990) A neural network approach towards intrusion detection. In: Proceedings of the 13th national computer security conferenceGoogle Scholar
  19. 19.
    De La Hoz E, Ortiz A, Ortega J, De La Hoz E, Mendoza F (2015) Implementation of an intrusion detection system based on self-organizing map. J Theor Appl Inf Technol 71(3):324–334Google Scholar
  20. 20.
    McElwee S, Cannady J (2016) Improving the performance of self-organizing maps for intrusion detection. SoutheastconGoogle Scholar
  21. 21.
    Yin C, Zhang S, Kim K (2017) Mobile anomaly detection based on improved self-organizing maps. Mob Inf Syst 1:1–9Google Scholar
  22. 22.
    Shareef SM, Hashim SH (2017) An approach based on decision tree and self-organizing map for intrusion detection. Iraqi Journal of Science 58(3B):1503–1515Google Scholar
  23. 23.
    Vasighi M, Amini H (2017) A directed batch growing approach to enhance the topology preservation of self-organizing map[J]. Appl Soft Comput 55:424–435Google Scholar
  24. 24.
    Ullah I, Mahmoud QH (2017) A filter-based feature selection model for anomaly-based intrusion detection systems. IEEE international conference on big data (BIGDATA)Google Scholar
  25. 25.
    Ichimura T, Yamaguchi T (2011) A proposal of interactive growing hierarchical SOM. Proc. of 2011 IEEE SMC2011, pp 3149–3154Google Scholar
  26. 26.
    Zhu Y, Liang J, Chen J, Ming Z (2017) An improved NSGA-iii algorithm for feature selection used in intrusion detection. Knowl.-Based Syst 116:74–85Google Scholar
  27. 27.
    Yaping Z, Wenxiu B, Chang S, Luyao W, Han X (2016) Intrusion detection method based on improved growing hierarchical self-organizing map. Transactions of Tianjin University 22:334–338.  https://doi.org/10.1007/s12209-016-2737-4 Google Scholar
  28. 28.
    Landress AD (2016) A hybrid approach to reducing the false positive rate in unsupervised machine learning intrusion detection. In: Southeastcon, pp 1–6Google Scholar
  29. 29.
    Vesanto J, Alhoniemi E (2000) Clustering of the self-organizing map. IEEE Trans Neural Netw 3:11Google Scholar
  30. 30.
    Kohonen T (1998) The self-organizing map. Neurocomputing 21:1–6zbMATHGoogle Scholar
  31. 31.
    Kohonen T (1993) Things you haven’t heard about the self-organizing map. In: IEEE international conference on neural networks, 1993, pp 1147–1156Google Scholar
  32. 32.
    Tang A, Sethumadhavan S, Stolfo SJ (2014) Unsupervised anomaly-based malware detection using hardware features. In: 17th international symposium on research in attacks intrusions and defenses (RAID)Google Scholar
  33. 33.
    Alsulaiman MM, Alyahya AN, Alkharboush RA, Alghafis NS (2009) Intrusion detection system using self organizing maps. In: International conference on network & system securityGoogle Scholar
  34. 34.
    Wang C, Yu H, Wang H (2009) Grey self-organizing map based intrusion detection. Optoelectron Lett 5:64–68Google Scholar
  35. 35.
    Ryan W, Obimbo C (2011) Self-organizing feature maps for user-to-root and remote-to-local network intrusion detection on the KDD cup 1999 dataset. In: 2011 World congress on internet security (WorldCIS). IEEEGoogle Scholar
  36. 36.
    Xue B, Zhang M, Yao X, Browne WN A survey on evolutionary computation approaches to feature selection. IEEE transactions on evolutionary computation.  https://doi.org/10.1109/TEVC.2015.2504420 Google Scholar
  37. 37.
    Sarasamma ST, Zhu QA (2006) MinCMax hyperellipsoidal clustering for anomaly detection in network security. IEEE Transactions on Systems Man & Cybernetics Part B Cybernetics A Publication of the IEEE Systems Man & Cybernetics Society 36(4):887–901Google Scholar
  38. 38.
    Ramadas M, Ostermann S, Tjaden B (2003) Detecting anomalous network traffic with self-organizing maps. International Workshop on Recent Advances in Intr 2820(1):36–54Google Scholar
  39. 39.
    Kaski S (1997) Data exploration using self-organizing maps. Acta polytechnica scandinavica mathematics, computing and management in engineering series, no. 82Google Scholar
  40. 40.
    Sarasamma ST, Zhu QA, Julie H (2005) Hierarchical Kohonenen net for anomaly detection in network security. IEEE Transactions on Systems Man & Cybernetics Part B Cybernetics A Publication of the IEEE Systems Man & Cybernetics Society vol 35, no 2Google Scholar
  41. 41.
    Lichodzijewski P (2002) Network based anomaly detection using self organizing maps. Technical Report, Nova Scotia, Dalhousie University, HalifaxGoogle Scholar
  42. 42.
    Huai-bin W, Hong-liang Y, Zhi-jian X, Zheng Y (2010) A clustering algorithm use SOM and k-means in intrusion detection. In: Proceedings of 2010 international conference on E-business and Egovernment. IEEE, pp 1281–1284Google Scholar
  43. 43.
    Hoglund AJ, Hatonen K, Sorvari AS (2000) A computer host based user anomaly detection system using the self-organizing map. Proc Int Joint Conf Neural Netw 5:411–416Google Scholar
  44. 44.
    Depren O, Topallar M, Anarim E, Ciliz MK (2005) An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks. Expert Systems with Applications 29:713–722Google Scholar
  45. 45.
    Garcia-Teodoro P, Diaz-Verdejo J, Macia-Fernandez G, Vazquez E (2009) Anomaly-based network intrusion detection. Techniques, systems and challenges, Computers & Security 28(1):18–28Google Scholar
  46. 46.
    Patcha A, Park J-M (2007) An overview of anomaly detection techniques: Existing solutions and latest technological trends. Comput Netw 51(12):3448–3470Google Scholar
  47. 47.
    Lampinen J, Oja E (1992) Clustering properties of hierarchical self-organizing maps. J Math Imaging Vision 2:261–272zbMATHGoogle Scholar
  48. 48.
    Zhisheng W, Xiaobing X (2013) Improved SOM-based high-dimensional data visualization algorithm. Comput Eng Appl 49(17):112–115Google Scholar
  49. 49.
    Dittenbach M, Merkl D, Rauber A (2000) Growing hierarchical self-organizing map. Neural Netw 6 (2):15–19zbMATHGoogle Scholar
  50. 50.
    KDDCUP 99 Accessed 11 Nov (2011), [Online]. Available: https://blog.csdn.net/com_stu_zhang/article/details/6987632
  51. 51.
    Almi’ani M, Ghazleh AA, Al-Rahayfeh A, Razaque A (2018) Intelligent intrusion detection system using clustered self organized map. In: 2018 Fifth international conference on software defined systems (SDS), pp 138–144Google Scholar
  52. 52.
    Patcha A, Park JM (2007) An overview of anomaly detection techniques: Existing solutions and latest technological trends. Comput Netw 51(12):3448–3470Google Scholar
  53. 53.
    Brahmi I, Brahmi H, Ben Yahia S (2015) A multi-agents intrusion detection system using ontology and clustering techniques. IFIP Advances in Information and Communication Technology 456:381–393Google Scholar
  54. 54.
    Fung C, Zhang J, Aib I, Boutaba R (2011) Trust management and admission control for host-based collaborative intrusion detection. J Netw Syst Manag 19(2):257–277Google Scholar
  55. 55.
    Perez MG, Marmol FG, Perez GM (2015) Improving attack detection in self-organizing networks: a trust-based approach toward alert satisfaction. International Conference on Advances in Computing, pp 1945–1951Google Scholar
  56. 56.
    Bashir U, Chachoo M (2014) Intrusion detection and prevention system. In: International conference on computing for sustainable global development (INDIACom). IEEEGoogle Scholar
  57. 57.
    Anomaly Detection Accessed 17 Nov, 2015 [Online]. Available: https://iwringer.wordpress.com/2015/11/17/anomaly-detection-concepts-and-techniques/
  58. 58.
    Kumar G, Kumar K, Sachdeva M (2010) The use of artificial intelligence based techniques for intrusion detection: a review. Artif Intell Rev 34(4):369–387Google Scholar
  59. 59.
    Prez-Surez A, Martnez-Trinidad JF, Carrasco-Ochoa JA (2018) A review of conceptual clustering algorithms. Artif Intell Rev 6:1–30Google Scholar
  60. 60.
    Salem M, Buehler U (2013) An enhanced GHSOM for IDS. In: Proc iEEE SMC: cybernetic, OctoberGoogle Scholar
  61. 61.
    Li M, Tian X, Sun Y, Yang J (2015) Adaptive recognition method based on improved-GHSOM for motor imagery EEG. Chin J Sci Instrum 36(5):1064–1071Google Scholar
  62. 62.
    Kohonen T (2001) Self-organizing maps. Volume 30 of Springer series in information sciences, 3rd edn. Springer, BerlinGoogle Scholar
  63. 63.
    Hsu AL, Halgamuge SK (2003) Enhancement of topology preservation and hierarchical dynamic self organizing maps for data visualistion. Int J Approx Reason 32:259–279zbMATHGoogle Scholar
  64. 64.
    Roberto H, Victor L, Fernando B (2012) Spatial clustering using hierarchical SOM. Chapter 12: applications of self-organizing maps, pp 231–250Google Scholar
  65. 65.
    Kopylova Y, Buell DA, Huang CT, et al. (2008) Mutual information applied to anomaly detection[J]. J Commun Networks 10(1):89–97Google Scholar
  66. 66.
    Huai-bin W, Hong-liang Y, Zhi-jian X (2010) A clustering algorithm use SOM and k-means in intrusion detection[C]. In: International conference on E-business and E-government, pp 1281–1284Google Scholar
  67. 67.
    Saraswati A, Nguyen VT, Hagenbuchner M, Tsoi AC (2018) High-resolution self-organizing maps for advanced visualization and dimension reduction. Neural networks the official journal of the international neural network society 105–166Google Scholar
  68. 68.
    Kohonen T (1982) Self-organized formation of topologically correct feature maps. T Biol Cybern, pp 43–59. [Online] Available:  https://doi.org/10.1007/BF00337288 MathSciNetzbMATHGoogle Scholar
  69. 69.
    LippMann RP, Fried DJ, Graf I, Haines JW, Kendall KR, McClung D, Weber D, Webster SE, Wyschogrod D, Cunningham RK, Zissman MA (2000) Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation. Proc DARPA Information Survivability Conf and Expo 2:12–26Google Scholar
  70. 70.
    Guo M, Huafu D (2008) Clustering algorithm based on SOM network and K-means. Computer & Digital Engineering 36(9):22–36Google Scholar
  71. 71.
    Hou L, Wang W (2011) Improved K-Means clustering algorithm based on SOM. Journal of Inner Mongolia University (Natural Science Edition) 5:42Google Scholar
  72. 72.
    Almi’ani M, Ghazleh AA (2018) Intelligent intrusion detection system using clustered self organized map. In: Fifth international conference on software defined systems (SDS)Google Scholar
  73. 73.
    Amini M, Jalili R (2004) Network-based intrusion detection using unsupervised adaptive resonance theory (ART). In: Proceedings of the fourth conference on engineering of intelligent systems (EIS 2004), Madeira, PortugalGoogle Scholar
  74. 74.
    Choksi K, Shah B, Kale O (2004) Intrusion detection system using self organizing maps a survey. Int J Eng Res Appl 12:4Google Scholar
  75. 75.
    Buczak AL, Guven E (2017) A survey of data mining and machine learning methods for cyber security intrusion Detection[J]. IEEE Commun Surv Tutorials 18(2):1153–1176Google Scholar
  76. 76.
    Fernando ZT, Thaseen IS, Kumar CA (2014) Network attacks identification using consistency based feature selection and self-organizing maps. IEEE conference on N/ws & soft computingGoogle Scholar
  77. 77.
    Franco ED, Garcia AO, Lopera JO, Correa ED, Palechor FM (2015) Implementation of an intrusion detection system based on self organizing map. J Theor Appl Inf Technol 3:71Google Scholar
  78. 78.
    Koikkalainen P, Oja E (1990) Self-organizing hierarchical feature maps. IJCNN International Joint Conference on Neural Networks 2:279–284Google Scholar
  79. 79.
    Hu YC, Chen RS, Hsu YT, Tzeng GH (2002) Grey self-organizing feature maps 48(1-4):863–877Google Scholar
  80. 80.
    Forti A, Foresti GL (2006) Growing hierarchical tree SOM: an unsupervised neural network with dynamic topology. Neural Netw 19(10):1568–1580zbMATHGoogle Scholar
  81. 81.
    Wang CD, Yu HF, Wang HB (2009) Grey self-organizing map based intrusion detection[J]. Optoelectron Lett 5(1):64–68Google Scholar
  82. 82.
    Le DC, Nur Zincir-Heywood A, Malcolm I, Wang HB (2019) Unsupervised monitoring of network and service behaviour using self organizing maps. Journal of Cyber Security and Mobility 8(1):15–52Google Scholar
  83. 83.
    Jing X, Yan Z, Liang X, Pedrycz W (2018) Network traffic fusion and analysis against DDoS flooding attacks with a novel reversible sketch, information fusion(2018), 10(13).  https://doi.org/10.1016/j.inffus
  84. 84.
    CSE-CIC-IDS2018 [Online]. Available: https://www.unb.ca/cic/datasets/ids-2018.html
  85. 85.
    Alahakoon D, Halgamuge SK, Srinivasan B (2000) Dynamic self-organizing maps with controlled growth for knowledge discover. IEEE Trans Neural Netw 10:601–614Google Scholar
  86. 86.
    Hsu AL, Saeed I, Halgamuge SK (2009) Dynamic self-organizing maps: theory, methods and applications. In: Foundations of computational intelligence volume 1, vol 201. pp 363-379Google Scholar
  87. 87.
    Self-organizingmap [Online]. Available: https://en.wikipedia.org/wiki/Self-organizingmap
  88. 88.
    Fontugne R, Borgnat P, Abry P (2010) MAWILAb: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking. International conference, ACMGoogle Scholar
  89. 89.
    Self-organizingmap Accessed 9 June (2018), [Online]. Available: https://commons.wikimedia.org/wiki/Category:Self-organizing-map
  90. 90.
    Liukkonen M, Hiltunen Y (2018) Recognition of systematic spatial patterns in silicon wafers based on som and k-means. IFAC-PapersOnLineGoogle Scholar
  91. 91.
    Zhang M, Yang P, Tian C, Tang S, Gao X, Wang B, Xiao F (2016) Quality-aware sensing coverage in budget-constrained mobile crowdsensing networks. IEEE Trans Veh Technol 65(9):7698–7707Google Scholar
  92. 92.
    Wu X, Xiong Y, Yang P, Wan S, Huang W (2014) Sparsest random scheduling for compressive data gathering in wireless sensor networks. IEEE Trans Wirel Commun 13(10):5867–5877Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Xiaofei Qu
    • 1
    • 2
    Email author
  • Lin Yang
    • 2
  • Kai Guo
    • 2
  • Linru Ma
    • 2
  • Meng Sun
    • 1
  • Mingxing Ke
    • 1
  • Mu Li
    • 1
  1. 1.College of Command and Control EngineeringArmy Engineering UniversityNanjingChina
  2. 2.National Key Laboratory of Science and Technology on Information System SecurityInstitute of Systems Engineering, AMSBeijingChina

Personalised recommendations