Evolving Systems

, Volume 8, Issue 3, pp 179–191 | Cite as

A mutual information based online evolving clustering approach and its applications

  • Fling Tseng
  • Dimitar Filev
  • Ratna Babu Chinnam
Original Paper


In this article, a new recursive evolving clustering method is proposed based on the well-known Gustafson–Kessel algorithm. The novelty of the proposed method involves the adaptation and integration of the mutual information based formulation to accommodate the Mahalanobis distance, which functions as the similarity measure and the unification of the clustering generation and pruning mechanisms. Example applications of the method are also discussed in the areas of data compression and knowledge extraction.


Evolving clustering Recursive clustering Evolving system 


  1. Abdullah O, Abdel-Qader I, Bazuin B (2016) K-means-Jensen–Shannon divergence for a WLAN indoor positioning system. In: Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), IEEE Annual (pp. 1–5). IEEEGoogle Scholar
  2. Ahmed HA, Mahanta P, Bhattacharyya DK, Kalita JK (2011) Gerc: tree based clustering for gene expression data. In: Bioinformatics and bioengineering (BIBE). In Bioinformatics and bioengineering (BIBE), 2011 IEEE 11th international conference on, pp. 299–302.Google Scholar
  3. Angelov P (2013) Evolving rule-based models: a tool for design of flexible adaptive systems (vol. 92). PhysicaGoogle Scholar
  4. Angelov P, Filev DP, Kasabov N (eds) (2010) Evolving intelligent systems: methodology and applications (vol. 12). Wiley, New YorkGoogle Scholar
  5. Banerjee A, Guo X, Wang H (2005a) On the optimality of conditional expectation as a Bregman predictor. IEEE Trans Inf Theory 51(7):2664–2669MathSciNetCrossRefzbMATHGoogle Scholar
  6. Banerjee A, Merugu S, Dhilon IS, Ghosh J (2005b) Clustering with Bregman divergences. J Mach Learn Res 6:1705–1749MathSciNetzbMATHGoogle Scholar
  7. Bezdek JC, Ehrlich R, Full W (1984) FCM: The fuzzy c-means clustering algorithm. Comput Geosci 10(23):191–203CrossRefGoogle Scholar
  8. Boubacar HA, Lecoeuche S (2005). A new kernel-based algorithm for online clustering. In: International conference on artificial neural networks, pp. 583–588. Springer Berlin HeidelbergGoogle Scholar
  9. Bouguila N, Ziou D (2006) Online clustering via finite mixtures of Dirichlet and minimum message length. Eng Appl Artif Intell 19(4):371–379CrossRefGoogle Scholar
  10. Bregman LM (1967) The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput Math Math Phys 7(3):200–217MathSciNetCrossRefzbMATHGoogle Scholar
  11. Davies DL, Bouldin DW (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 2:224–227CrossRefGoogle Scholar
  12. Dovžan D, Škrjanc I (2011) Recursive clustering based on a Gustafson–Kessel algorithm. Evolving Syst 2(1):15–24CrossRefGoogle Scholar
  13. Driver HE, Kroeber AL (1932) Quantitative expression of cultural relationships. University of California Press, BerkeleyGoogle Scholar
  14. Duchi J (2007) Derivations for linear algebra and optimization. Berkeley, California.
  15. Duda RO, Hart PE, Stork DG (1973) Pattern classification. Wiley, New York, pp 526–528zbMATHGoogle Scholar
  16. Filev DP, Georgieva O (2010) An extended version of the Gustafson–Kessel algorithm for evolving data stream clustering. Evolving intelligent systems: methodology and applications. IEEE Press Series on Computational Intelligence. Wiley, New York, pp 273–300Google Scholar
  17. Filev D, Tseng F (2006a) Novelty detection based machine health prognostics. In: 2006 International symposium on evolving fuzzy systems, IEEE, pp 193–199Google Scholar
  18. Filev D, Tseng F (2006b) Real time novelty detection modeling for machine health prognostics. In: NAFIPS 2006–2006 annual meeting of the North American fuzzy information processing societyGoogle Scholar
  19. Filev D, Tseng FF, Kristinsson J, McGee R (2011) Contextual on-board learning and prediction of vehicle destinations. IEEE Computational Intelligence in Vehicles and Transportation Systems (CIVTS), pp 87–91Google Scholar
  20. Georgieva O, Filev DP (2009). Gustafson–kessel algorithm for evolving data stream clustering. In: Proceedings of the international conference on computer systems and technologies and workshop for PhD students in computing p 62, ACMGoogle Scholar
  21. Gustafson DE, Kessel WC (1978) Fuzzy clustering with a fuzzy covariance matrix. Scientific Systems. Inc., CambridgeCrossRefzbMATHGoogle Scholar
  22. Hai Y, Tseng F, Wang Q (2015) Patent No. 9121722 USGoogle Scholar
  23. Hartigan JA, Wong MA (1979) Algorithm AS 136: a k-means clustering algorithm. J R Stat Soc Series C (Applied Statistics) 28(1):100–108zbMATHGoogle Scholar
  24. Iglesias JA, Angelov P, Ledezma A, Sanchis A (2012) Creating evolving user behavior profiles automatically. IEEE Trans Knowl Data Eng 24(5):854–867CrossRefGoogle Scholar
  25. Jain AK, Murty NM, Flynn PJ (1999) Data clustering: a review. ACM Comput surv (CSUR) 31(3):264–323CrossRefGoogle Scholar
  26. Kasabov NK, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154CrossRefGoogle Scholar
  27. Kohonen T (2012) Self-organization and associative memory. Springer, New YorkzbMATHGoogle Scholar
  28. Kuo CFJ, Jian BL, Wu HC, Peng KC (2012) Automatic machine embroidery image color analysis system. Part I: using Gustafson–Kessel clustering algorithm in embroidery fabric color separation. Textile Res J 82(6):571–583CrossRefGoogle Scholar
  29. Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37(1):145–151MathSciNetCrossRefzbMATHGoogle Scholar
  30. Lughofer ED (2008) FLEXFIS: a robust incremental learning approach for evolving Takagi–Sugeno fuzzy models. IEEE Trans Fuzzy Syst 16(6):1394–1410CrossRefGoogle Scholar
  31. Mahalanobis PC (1936) On the generalized distance in statistics. In: Proceedings of the National Institute of Sciences (Calcutta) vol 2, pp 49–55Google Scholar
  32. Masson MH, Denoeux T (2008) An evidential version of the fuzzy c-means algorithm. Pattern Recognit 41(4):1384–1397CrossRefzbMATHGoogle Scholar
  33. McGee RA, Filev DP, Kristinsson JG, Tseng F, Philips AM (2012) Patent No. 13/714919 USGoogle Scholar
  34. Mousselly-Sergieh H, Döller M, Egyed-Zsigmond E, Gianini G, Kosch H, Pinon JM (2014) Tag relatedness using laplacian score feature selection and adapted Jensen–Shannon divergence. In: International Conference on multimedia modeling pp 159–171. Springer International PublishingGoogle Scholar
  35. Qiao J, Wang H (2008) A self-organizing fuzzy neural network and its applications to function approximation and forecast modeling. Neurocomputing 71(4):564–569CrossRefGoogle Scholar
  36. Serir L, Ramasso E, Zerhouni N (2012) Evidential evolving Gustafson–Kessel algorithm for online data streams partitioning using belief function theory. Int J Approx Reason 53(5):747–768MathSciNetCrossRefGoogle Scholar
  37. Shankar AS, Asokan A, Sivakumar D (2016) Brain tumor classification using Gustafson–Kessel (GK) fuzzy clustering algorithm. Brain 1(05):68–72Google Scholar
  38. Tefaj E, Kasneci G, Rosenstiel W, Bogdan M (2012) Bayesian online clustering of eye movement data. In: Proceedings of the symposium on eye tracking research and applications, pp. 285–288. ACMGoogle Scholar
  39. Tseng F, Filev DP, Makki IH, Michelini JO (2016) Patent No. 20160109243 USGoogle Scholar
  40. Tyron RC (1939) Cluster analysis: correlation profile and orthometric (factor) analysis for the isolation of unities in mind and personality. Edwards brother, Incorporated, lithoprinters and publishersGoogle Scholar
  41. Wagstaff K, Cardie C, Rogers S, Schrödl S (2001) Constrained k-means clustering with background knowledge. In ICML 1:577–584Google Scholar
  42. Wood AR, Esko T, Yang J, Vedantam S, Pers TH, Gustafsson S et al (2014) Defining the role of common variation in the genomic and biological architecture of adult human height. Nat Genet 46(11):1173–1186CrossRefGoogle Scholar
  43. Woodbury MA (1950) Inverting modified matrices, memorandum Rept. 42. Statistical Research Group, 316. Princeton University, Princeton, p 316Google Scholar
  44. Xu R, Wunsch D (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678CrossRefGoogle Scholar
  45. Yager RR, Filev DP (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284CrossRefGoogle Scholar
  46. Yu J, Yang MS (2015) Analysis of parameter selection for Gustafson–Kessel fuzzy clustering using Jacobian matrix. IEEE Trans Fuzzy Syst 23(6):2329–2342CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Fling Tseng
    • 1
    • 2
  • Dimitar Filev
    • 2
  • Ratna Babu Chinnam
    • 1
  1. 1.Department of Industrial and Systems EngineeringWayne State UniversityDetroitUSA
  2. 2.Modern Control Methods and Computational Intelligence, Research and Advanced EngineeringFord Motor CompanyDearbornUSA

Personalised recommendations