A New Hybrid K-Means Evolving Spiking Neural Network Model Based on Differential Evolution

Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 5)


Clustering is one of the essential unsupervised learning techniques in Data Mining. In this paper, a new hybrid (K-DESNN) approach to combine differential evolution and K-means evolving spiking neural network model (K-means ESNN) for clustering problems has been proposed. The proposed model examines that ESNN improves by using K-DESNN model. This approach improves the flexibility of the ESNN algorithm in producing better solutions which is utilized to conquer the K-means disadvantages. Various UCI machine learning data sets have been utilized for evaluating the performance of this model. The results have shown that K-DESNN is much better than the original K-means ESNN in the number of pre-synaptic neurons measure and clustering accuracy performance.


Clustering K-means Differential evolution Spiking neural network Evolving spiking neural networks K-DESNN 


  1. 1.
    Schliebs, S., Defoin-Platel, M., Worner, S., Kasabov, N.: Integrated feature and parameter optimization for an evolving spiking neural network: exploring heterogeneous probabilistic models. Neural Networks 22, 623–632 (2009)CrossRefGoogle Scholar
  2. 2.
    Kasabov, N.K.: NeuCube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Networks 52, 62–76 (2014)CrossRefGoogle Scholar
  3. 3.
    Bohte, S.M., La Poutré, H., Kok, J.N.: Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks. IEEE Trans. Neural Networks 13, 426–435 (2002)CrossRefGoogle Scholar
  4. 4.
    Hamed, H.N.A., Saleh, A.Y., Shamsuddin, S.M.: A novel k-means evolving spiking neural network model for clustering problems. In: International Symposium on Neural Networks, pp. 382–389 (2015)Google Scholar
  5. 5.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)Google Scholar
  6. 6.
    Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., et al.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14, 1–37 (2008)CrossRefGoogle Scholar
  7. 7.
    Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R.: Advances in knowledge discovery and data mining (1996)Google Scholar
  8. 8.
    Firouzi, B., Niknam, T., Nayeripour, M.: A new evolutionary algorithm for cluster analysis. World Acad. Sci. Eng. Technol. 36, 605–609 (2008)Google Scholar
  9. 9.
    Schenker, A., Last, M., Bunke, H., Kandel, A.: A comparison of two novel algorithms for clustering web documents. In: Proceedings of the 2nd International Workshop on Web Document Analysis (WDA 2003), pp. 71–74 (2003)Google Scholar
  10. 10.
    Pham, D.T., Dimov, S.S., Nguyen, C.: Selection of K in K-means clustering. Proc. Inst. Mech. Eng. Part C J. Mech. Eng. Sci. 219, 103–119 (2005)CrossRefGoogle Scholar
  11. 11.
    Wu, J.: Advances in K-means clustering: a data mining thinking. Springer Science & Business Media, Berlin Heidelberg (2012)CrossRefzbMATHGoogle Scholar
  12. 12.
    Alsabti, K., Ranka, S., Singh, V.: An efficient k-means clustering algorithm (1997)Google Scholar
  13. 13.
    Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31, 651–666 (2010)CrossRefGoogle Scholar
  14. 14.
    Berkhin, P.: A survey of clustering data mining techniques. In: Grouping Multidimensional Data, pp. 25–71. Springer (2006)Google Scholar
  15. 15.
    Bock, H.-H.: Clustering methods: a history of k-means algorithms. In: Selected Contributions in Data Analysis and Classification, pp. 161–172. Springer (2007)Google Scholar
  16. 16.
    Patel, V.R., Mehta, R.G.: Modified k-means clustering algorithm. In: Computational Intelligence and Information Technology, pp. 307–312. Springer (2011)Google Scholar
  17. 17.
    Thakare, Y., Bagal, S.: Performance evaluation of K-means clustering algorithm with various distance metrics. Int. J. Comput. Appl. 110 (2015)Google Scholar
  18. 18.
    Singh, A., Yadav, A., Rana, A.: K-means with three different distance metrics. Int. J. Comput. Appl. 67 (2013)Google Scholar
  19. 19.
    Mandloi, M.: A survey on clustering algorithms and k-means, July 2014Google Scholar
  20. 20.
    Kotsiantis, S., Pintelas, P.: Recent advances in clustering: a brief survey. WSEAS Trans. Inf. Sci. Appl. 1, 73–81 (2004)Google Scholar
  21. 21.
    Schliebs, S., Kasabov, N.: Evolving spiking neural network—a survey. Evolving Syst. 4, 87–98 (2013)CrossRefGoogle Scholar
  22. 22.
    Hamed, H.N., Kasabov, N., Michlovský, Z., Shamsuddin, S.: String pattern recognition using evolving spiking neural networks and quantum inspired particle swarm optimization. In: Leung, C., Lee, M., Chan, J. (eds.) Neural Information Processing, vol. 5864, pp. 611–619. Springer, Berlin Heidelberg (2009)Google Scholar
  23. 23.
    Hamed, H.N., Kasabov, N.: Quantum-inspired particle swarm optimisation for integrated feature and parameter optimisation of evolving spiking neural networks. Int. J. Artif. Intell. 7, 114–124 (2011)Google Scholar
  24. 24.
    Hamed, H.N., Kasabov, N., Shamsuddin, S.M., Widiputra, H., Dhoble, K.: An extended evolving spiking neural network model for spatio-temporal pattern classification. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 2653–2656 (2011)Google Scholar
  25. 25.
    Kasabov, N., Dhoble, K., Nuntalid, N., Indiveri, G.: Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition. Neural Networks 41, 188–201 (2013)CrossRefGoogle Scholar
  26. 26.
    Kasabov, N., Filev, D.: Evolving intelligent systems: methods, learning, & applications. In: 2006 International Symposium on Evolving Fuzzy Systems, pp. 8–18 (2006)Google Scholar
  27. 27.
    Wysoski, S., Benuskova, L., Kasabov, N.: Adaptive learning procedure for a network of spiking neurons and visual pattern recognition, pp. 1133–1142 (2006)Google Scholar
  28. 28.
    Kasabov, N.: Evolving spiking neural networks and neurogenetic systems for spatio-and spectro-temporal data modelling and pattern recognition. In: Advances in Computational Intelligence, pp. 234–260. Springer (2012)Google Scholar
  29. 29.
    Kasabov, N., Feigin, V., Hou, Z.-G., Chen, Y., Liang, L., Krishnamurthi, R., et al.: Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke. Neurocomputing 134, 269–279 (2014)CrossRefGoogle Scholar
  30. 30.
    Wysoski, S., Benuskova, L., Kasabov, N.: On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition. In: Artificial Neural Networks–ICANN 2006, pp. 61–70 (2006)Google Scholar
  31. 31.
    Soltic, S., Wysoski, S.G., Kasabov, N.K.: Evolving spiking neural networks for taste recognition. In: IEEE International Joint Conference on Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence), pp. 2091–2097 (2008)Google Scholar
  32. 32.
    Schliebs, S., Defoin-Platel, M., Kasabov, N.: Integrated feature and parameter optimization for an evolving spiking neural network. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) Advances in Neuro-Information Processing, vol. 5506, pp. 1229–1236. Springer, Berlin Heidelberg (2009)Google Scholar
  33. 33.
    Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Networks 10, 1659–1671 (1997)CrossRefGoogle Scholar
  34. 34.
    Thorpe, S., Delorme, A., Van Rullen, R.: Spike-based strategies for rapid processing. Neural Networks 14, 715–725 (2001)CrossRefGoogle Scholar
  35. 35.
    Bohte, S.M., Kok, J.N.: Applications of spiking neural networks. Inf. Proc. Lett. 95, 519–520 (2005)CrossRefzbMATHGoogle Scholar
  36. 36.
    Belatreche, A., Maguire, L.P., Mcginnity, M., Wu, Q.X.: Evolutionary design of spiking neural networks. New Math. Natural Comput. 2, 237–253 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J.M., et al.: Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci. 23, 349–398 (2007)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Hamed, Kasabov, N., Michlovský, Z., Shamsuddin, S.: String pattern recognition using evolving spiking neural networks and quantum inspired particle swarm optimization, pp. 611–619 (2009)Google Scholar
  39. 39.
    Abbass, H.A., Sarker, R., Newton, C.: PDE: a Pareto-frontier differential evolution approach for multi-objective optimization problems. In: Proceedings of the 2001 Congress on Evolutionary Computation, pp. 971–978, (2001)Google Scholar
  40. 40.
    Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15, 4–31 (2011)CrossRefGoogle Scholar
  41. 41.
    Noman, N., Iba, H.: Accelerating differential evolution using an adaptive local search. IEEE Trans. Evol. Comput. 12, 107–125 (2008)CrossRefGoogle Scholar
  42. 42.
    Gong, W., Cai, Z., Ling, C.X.: DE/BBO: a hybrid differential evolution with biogeography-based optimization for global numerical optimization. Soft. Comput. 15, 645–665 (2010)CrossRefGoogle Scholar
  43. 43.
    Ilonen, J., Kamarainen, J.-K., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Process. Lett. 17, 93–105 (2003)CrossRefGoogle Scholar
  44. 44.
    Storn, R., Price, K.: Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11, 341–359 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Saleh, A.Y., Hameed, H.N.B.A., Najib, M., Salleh, M.: A novel hybrid algorithm of differential evolution with evolving spiking neural network for pre-synaptic neurons optimization. Int. J. Adv. Soft Comput. Appl. 6 (2014)Google Scholar
  46. 46.
    Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.B.A.: Parameter tuning of evolving spiking neural network with differen-tial evolution algorithm. In: International Conference of Recent Trends in Information and Communication Technologies, p. 13 (2014)Google Scholar
  47. 47.
    Bohte, S.M., Kok, J.N., La Poutre, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48, 17–37 (2002)CrossRefzbMATHGoogle Scholar
  48. 48.
    Thorpe, S.: How can the human visual system process a natural scene in under 150 ms? Experiments and neural network models, p. 2-9600049 (1997)Google Scholar
  49. 49.
    Zaki, M.J., Meira Jr., W., Meira, W.: Data Mining and Analysis: Fundamental Concepts and Algorithms. Cambridge University Press, Cambridge (2014)zbMATHGoogle Scholar
  50. 50.
    Sun, Y., Zhu, Q., Chen, Z.: An iterative initial-points refinement algorithm for categorical data clustering. Pattern Recogn. Lett. 23, 875–884 (2002)CrossRefzbMATHGoogle Scholar
  51. 51.
    Huang, Z.: Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Mining Knowl. Discov. 2, 283–304 (1998)CrossRefGoogle Scholar
  52. 52.
    Huang, Z., Ng, M.K.: A fuzzy k-modes algorithm for clustering categorical data. IEEE Trans. Fuzzy Syst. 7, 446–452 (1999)CrossRefGoogle Scholar
  53. 53.
    He, Z., Deng, S., Xu, X.: Improving k-modes algorithm considering frequencies of attribute values in mode. In: International Conference on Computational and Information Science, pp. 157–162 (2005)Google Scholar
  54. 54.
    Godara, S., Singh, R., Kumar, S.: A novel weighted class based clustering for medical diagnostic interface. Indian J. Sci. Technol. 9 (2016)Google Scholar
  55. 55.
    Nazeer, K.A., Sebastian, M.: Clustering biological data using enhanced k-means algorithm. In: Electronic Engineering and Computing Technology, pp. 433–442. Springer (2010)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.FSKPM FacultyUniversity Malaysia Sarawak (UNIMAS)Kota SamarahanMalaysia
  2. 2.Soft Computing Research Group, Faculty of ComputingUniversiti Teknologi Malaysia (UTM)SkudaiMalaysia
  3. 3.UTM Big Data CentreUniversiti Teknologi MalaysiaSkudaiMalaysia
  4. 4.Faculty of Computer and TechnologyAlzaiem Alazhari UniversityKhartoumSudan
  5. 5.Nile CollegeKhartoum NorthSudan

Personalised recommendations