Soft Computing

, Volume 21, Issue 5, pp 1347–1369 | Cite as

An agent-based algorithm exploiting multiple local dissimilarities for clusters mining and knowledge discovery

  • Filippo Maria Bianchi
  • Enrico Maiorino
  • Lorenzo Livi
  • Antonello Rizzi
  • Alireza Sadeghian
Methodologies and Application

Abstract

We propose a multi-agent algorithm able to automatically discover relevant regularities in a given dataset, determining at the same time the set of configurations of the adopted parametric dissimilarity measure that yield compact and separated clusters. Each agent operates independently by performing a Markovian random walk on a weighted graph representation of the input dataset. Such a weighted graph representation is induced by a specific parameter configuration of the dissimilarity measure adopted by an agent for the search. During its lifetime, each agent evaluates different parameter configurations and takes decisions autonomously for one cluster at a time. Results show that the algorithm is able to discover parameter configurations that yield a consistent and interpretable collection of clusters. Moreover, we demonstrate that our algorithm shows comparable performances with other similar state-of-the-art algorithms when facing specific clustering problems. Notably, we compare our method with respect to several graph-based clustering algorithms and a well-known subspace search method.

Keywords

Agent-based algorithms Data mining Knowledge discovery Clustering Local dissimilarity measure Graph conductance Random walk 

References

  1. Aggarwal CC, Wolf JL, Yu PS, Procopiuc C, Park JS (1999) Fast algorithms for projected clustering. SIGMOD Rec 28(2):61–72. doi:10.1145/304181.304188 CrossRefGoogle Scholar
  2. Agogino A, Tumer K (2006) Efficient agent-based cluster ensembles. In: Proceedings of the fifth international joint conference on autonomous agents and multiagent systems, ACM, pp 1079–1086Google Scholar
  3. Alamgir M, von Luxburg U (2010) Multi-agent random walks for local clustering on graphs. In: Proceedings of the IEEE 10th international conference on data mining, pp 18–27. doi:10.1109/ICDM.2010.87
  4. Andersen R, Chung F, Lang K (2006) Local graph partitioning using pagerank vectors. In: Proceedings of the 47th annual IEEE symposium on foundations of computer science. IEEE Computer Society, Washington, DC, USA, pp 475–486. doi:10.1109/FOCS.2006.44
  5. Arora S, Rao S, Vazirani U (2008) Geometry, flows, and graph-partitioning algorithms. Commun ACM 51(10):96–105CrossRefGoogle Scholar
  6. Arora S, Rao S, Vazirani U (2009) Expander flows, geometric embeddings and graph partitioning. J ACM 56(2):5MathSciNetCrossRefMATHGoogle Scholar
  7. Azran A, Ghahramani Z (2006) Spectral methods for automatic multiscale data clustering. In: Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition. IEEE Computer Society, Washington, DC, USA, pp 190–197. doi:10.1109/CVPR.2006.289
  8. Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 18 Feb 2015
  9. Bereta M, Pedrycz W, Reformat M (2013) Local descriptors and similarity measures for frontal face recognition: a comparative analysis. J Vis Commun Image Represent 24(8):1213–1231. doi:10.1016/j.jvcir.2013.08.004 CrossRefGoogle Scholar
  10. Bianchi FM, Livi L, Rizzi A (2015) Two density-based k-means initialization algorithms for non-metric data clustering. Pattern Anal Appl. doi:10.1007/s10044-014-0440-4 Google Scholar
  11. Bulò SR, Pelillo M (2013) A game-theoretic approach to hypergraph clustering. IEEE Trans Pattern Anal Mach Intell 35(6):1312–1327CrossRefGoogle Scholar
  12. Cao L (2009) Data mining and multi-agent integration. Springer, BerlinCrossRefMATHGoogle Scholar
  13. Cao J, Wu Z, Wu J, Liu W (2013) Towards information-theoretic K-means clustering for image indexing. Signal Process 93(7):2026–2037. doi:10.1016/j.sigpro.2012.07.030 CrossRefGoogle Scholar
  14. Chaimontree S, Atkinson K, Coenen F (2012) A framework for multi-agent based clustering. Auton Agents Multi-Agent Syst 25(3):425–446. doi:10.1007/s10458-011-9187-0 CrossRefGoogle Scholar
  15. Chandrasekhar U, Naga P (2011) Recent trends in ant colony optimization and data clustering: a brief survey. In: 2nd international conference on intelligent agent and multi-agent systems (IAMA), pp 32–36. doi:10.1109/IAMA.2011.6048999
  16. Chang CC (2012) A boosting approach for supervised mahalanobis distance metric learning. Pattern Recogn 45(2):844–862CrossRefMATHGoogle Scholar
  17. Chung F (1994) Spectral graph theory. AMS, ProvidenceGoogle Scholar
  18. De Smet F, Aeyels D (2009) Cluster transitions in a multi-agent clustering model. In: Proceedings of the 48th IEEE conference on decision and control, 2009 held jointly with the 2009 28th Chinese control conference. CDC/CCC 2009, pp 4778–4784. doi:10.1109/CDC.2009.5400314
  19. Delvenne JC, Yaliraki SN, Barahona M (2010) Stability of graph communities across time scales. Proc Natl Acad Sci 107(29):12,755–12,760. doi:10.1073/pnas.0903215107 CrossRefGoogle Scholar
  20. Ditterrich TG (1997) Machine learning research: four current direction. Artif Intell Magz 4:97–136Google Scholar
  21. Duin RPW, Pȩkalska E (2012) The dissimilarity space: bridging structural and statistical pattern recognition. Pattern Recogn Lett 33(7):826–832. doi:10.1016/j.patrec.2011.04.019 CrossRefGoogle Scholar
  22. Ferrer M, Valveny E, Serratosa F, Bardají I, Bunke H (2009) Graph-based k-means clustering: a comparison of the set median versus the generalized median graph. In: Proceedings of the 13th international conference on computer analysis of images and patterns, CAIP ’09. Springer, Berlin, pp 342–350. doi:10.1007/978-3-642-03767-2_42
  23. Forestier G, Gançarski P, Wemmert C (2010) Collaborative clustering with background knowledge. Data Knowl Eng 69(2):211–228CrossRefMATHGoogle Scholar
  24. Gallesco C, Mueller S, Popov S (2011) A note on spider walks. ESAIM: Probab Stat 15:390–401MathSciNetCrossRefMATHGoogle Scholar
  25. Galluccio L, Michel O, Comon P, Hero AO III (2012) Graph based k-means clustering. Signal Process 92(9):1970–1984. doi:10.1016/j.sigpro.2011.12.009 CrossRefGoogle Scholar
  26. Galluccio L, Michel O, Comon P, Kliger M, Hero AO III (2013) Clustering with a new distance measure based on a dual-rooted tree. Inf Sci 251:96–113. doi:10.1016/j.ins.2013.05.040 MathSciNetCrossRefGoogle Scholar
  27. Giannella C, Bhargava R, Kargupta H (2004) Multi-agent systems and distributed data mining. In: Cooperative information agents VIII. Springer, Berlin, pp 1–15Google Scholar
  28. Gisbrecht A, Schleif FM (2015) Metric and non-metric proximity transformations at linear costs. Neurocomputing 167:643–657. doi:10.1016/j.neucom.2015.04.017 CrossRefGoogle Scholar
  29. Gkantsidis C, Mihail M, Saberi A (2003) Conductance and congestion in power law graphs. ACM SIGMETRICS Perform Eval Rev ACM 31:148–159CrossRefGoogle Scholar
  30. Gönen M, Alpaydın E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268MathSciNetMATHGoogle Scholar
  31. Gorodetsky V, Karsaeyv O, Samoilov V (2003) Multi-agent technology for distributed data mining and classification. In: Proceedings of the IEEE/WIC international conference on intelligent agent technology, IEEE, pp 438–441Google Scholar
  32. Han J, Kamber M, Pei J (2011) Data mining: concepts and techniques: concepts and techniques. Elsevier, New YorkMATHGoogle Scholar
  33. Hoory S, Linial N, Wigderson A (2006) Expander graphs and their applications. Bull Am Math Soc 43(4):439–561MathSciNetCrossRefMATHGoogle Scholar
  34. Izakian H, Pedrycz W, Jamal I (2013) Clustering spatio-temporal data: an augmented fuzzy C-means. IEEE Trans Fuzzy Syst 21(5):855–868. doi:10.1109/TFUZZ.2012.2233479 CrossRefGoogle Scholar
  35. Kannan R, Vempala S, Vetta A (2004) On clusterings: good, bad and spectral. J ACM 51(3):497–515MathSciNetCrossRefMATHGoogle Scholar
  36. Kim SW, Duin RPW (2009) A combine-correct-combine scheme for optimizing dissimilarity-based classifiers. In: Bayro-Corrochano E, Eklundh JO (eds) Progress in pattern recognition, image analysis, computer vision, and applications, LNCS, vol 5856. Springer, Berlin, pp 425–432. doi:10.1007/978-3-642-10268-4_49
  37. Klusch M, Lodi S, Moro G (2003) Agent-based distributed data mining: the KDEC scheme. In: Klusch M, Bergamaschi S, Edwards P, Petta P (eds) Intelligent information agents, vol 2586. Springer, Berlin, pp 104–122. doi:10.1007/3-540-36561-3_5
  38. Komorowski J, Zytkow J (1997) Principles of data mining and knowledge discovery. Springer, BerlinCrossRefGoogle Scholar
  39. Kriegel HP, Kröger P, Zimek A (2009) Clustering high-dimensional data: a survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Trans Knowl Discov Data 3(1):1:1–1:58. doi:10.1145/1497577.1497578 CrossRefGoogle Scholar
  40. Leighton T, Rao S (1999) Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. J ACM 46:787–832. doi:10.1145/331524.331526 MathSciNetCrossRefMATHGoogle Scholar
  41. Livi L, Rizzi A, Sadeghian A (2014) Optimized dissimilarity space embedding for labeled graphs. Inf Sci 266:47–64. doi:10.1016/j.ins.2014.01.005 MathSciNetCrossRefGoogle Scholar
  42. Livi L, Rizzi A, Sadeghian A (2015) Granular modeling and computing approaches for intelligent analysis of non-geometric data. Appl Soft Comput 27:567–574. doi:10.1016/j.asoc.2014.08.072 CrossRefGoogle Scholar
  43. Lovász L (1996) Random walks on graphs: a survey. In: Miklós D, Sós VT, Szőnyi T (eds) Combinatorics, Paul Erdős is eighty, vol 2. János Bolyai Mathematical Society, Budapest, pp 353–398Google Scholar
  44. Madry A (2010) Fast approximation algorithms for cut-based problems in undirected graphs. In: Proceedings of the 51st annual IEEE symposium on foundations of computer science, pp 245–254. doi:10.1109/FOCS.2010.30
  45. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092. doi:10.1063/1.1699114 CrossRefGoogle Scholar
  46. Mitra S, Banka H, Pedrycz W (2006) Rough-fuzzy collaborative clustering. IEEE Trans Syst Man Cybern Part B: Cybern 36(4):795–805CrossRefGoogle Scholar
  47. Mu Y, Ding W, Tao D (2013) Local discriminative distance metrics ensemble learning. Pattern Recogn 46(8):2337–2349. doi:10.1016/j.patcog.2013.01.010 CrossRefMATHGoogle Scholar
  48. Negenborn RR, Hug-Glanzmann G, De Schutter B, Andersson G (2010) A novel coordination strategy for multi-agent control using overlapping subnetworks with application to power systems. In: Mohammadpour J, Grigoriadis KM (eds) Efficient modeling and control of large-scale systems. Springer, Norwell, pp 251–278Google Scholar
  49. Nguyen TM, Wu QMJ (2013) Dynamic fuzzy clustering and its application in motion segmentation. IEEE Trans Fuzzy Syst 21(6):1019–1031. doi:10.1109/TFUZZ.2013.2240689 CrossRefGoogle Scholar
  50. North MJ (2014) A theoretical formalism for analyzing agent-based models. Complex Adapt Syst Model 2(1):3CrossRefGoogle Scholar
  51. Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. ACM SIGKDD Explor Newslett 6(1):90–105CrossRefGoogle Scholar
  52. Pedrycz W (2002) Collaborative fuzzy clustering. Pattern Recogn Lett 23(14):1675–1686CrossRefMATHGoogle Scholar
  53. Pedrycz W (2005) Knowledge-based clustering: from data to information granules. Wiley, New YorkCrossRefMATHGoogle Scholar
  54. Pedrycz W (2013) Proximity-based clustering: a search for structural consistency in data with semantic blocks of features. IEEE Trans Fuzzy Syst 21(5):978–982. doi:10.1109/TFUZZ.2012.2236842 CrossRefGoogle Scholar
  55. Prodromidis A, Chan P, Stolfo S (2000) Meta-learning in distributed data mining systems: issues and approaches. Adv Distrib Parallel Knowl Discov 3:81–114Google Scholar
  56. Provost FJ, Hennessy DN (1996) Scaling up: distributed machine learning with cooperation. In: Proceedings of the thirteenth national conference on artificial intelligence, vol 1, pp 74–79Google Scholar
  57. Queiroz S, de Carvalho FDAT, Lechevallier Y (2013) Nonlinear multicriteria clustering based on multiple dissimilarity matrices. Pattern Recogn 46(12):3383–3394. doi:10.1016/j.patcog.2013.06.008 CrossRefMATHGoogle Scholar
  58. Sarma AD, Gollapudi S, Panigrahy R (2011) Estimating pagerank on graph streams. J ACM 58(3):13MathSciNetCrossRefMATHGoogle Scholar
  59. Shen C, Kim J, Liu F, Wang L, van den Hengel A (2014) Efficient dual approach to distance metric learning. IEEE Trans Neural Netw Learn Syst 25(2):394–406. doi:10.1109/TNNLS.2013.2275170 CrossRefGoogle Scholar
  60. Spielman DA, Teng SH (2013) A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM J Comput 42(1):1–26MathSciNetCrossRefMATHGoogle Scholar
  61. Tabrizi SA, Shakery A, Asadpour M, Abbasi M, Tavallaie MA (2013) Personalized pagerank clustering: a graph clustering algorithm based on random walks. Phys A: Stat Mech Appl 392(22):5772–5785. doi:10.1016/j.physa.2013.07.021 MathSciNetCrossRefGoogle Scholar
  62. Trefethen LN, Bau D III (1997) Numerical linear algebra, vol 50. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
  63. Vidal R (2010) A tutorial on subspace clustering. IEEE Signal Process Maga 28(2):52–68CrossRefGoogle Scholar
  64. Yang L, Jin R, Mummert L, Sukthankar R, Goode A, Zheng B, Hoi SCH, Satyanarayanan M (2010) A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval. IEEE Trans Pattern Anal Mach Intell 32(1):30–44. doi:10.1109/TPAMI.2008.273
  65. Yin X, Shu T, Huang Q (2012) Semi-supervised fuzzy clustering with metric learning and entropy regularization. Knowl-Based Syst 35:304–311CrossRefGoogle Scholar
  66. Zhang H, Yu J, Wang M, Liu Y (2012) Semi-supervised distance metric learning based on local linear regression for data clustering. Neurocomputing 93:100–105Google Scholar
  67. Zhang L, Pedrycz W, Lu W, Liu X, Zhang L (2014) An interval weighed fuzzy c-means clustering by genetically guided alternating optimization. Expert Syst Appl 41(13):5960–5971CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Information Engineering, Electronics and Telecommunications (DIET)“Sapienza” University of RomeRomeItaly
  2. 2.Department of Computer ScienceRyerson UniversityTorontoCanada

Personalised recommendations