An Integration-Based Approach to Pattern Clustering and Classification

  • Laura Sani
  • Gianluca D’Addese
  • Riccardo PecoriEmail author
  • Monica Mordonini
  • Marco Villani
  • Stefano Cagnoni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11298)


Methods based on information theory, such as the Relevance Index (RI), have been employed to study complex systems for their ability to detect significant groups of variables, well integrated among one another and well separated from the others, which provide a functional block description of the system under analysis. The integration (or zI in its standardized form) is a metric that can express the significance of a group of variables for the system under consideration: the higher the zI, the more significant the group. In this paper, we use this metric for an unusual application to a pattern clustering and classification problem. The results show that the centroids of the clusters of patterns identified by the method are effective for distance-based classification algorithms. We compare such a method with other conventional classification approaches to highlight its main features and to address future research towards the refinement of its accuracy and computational efficiency.


Classification Clustering Complex systems RI metrics 



The authors would like to thank Chiara Lasagni for the many tests and for helping us reach full awareness of some of the finer details of the method.


  1. 1.
    Aldana-Bobadilla, E., Kuri-Morales, A.: A clustering method based on the maximum entropy principle. Entropy 17(1), 151–180 (2015)CrossRefGoogle Scholar
  2. 2.
    Bouckaert, R.R., et al.: WEKA manual for version 3-7-8. University of Waikato, NZ (2013)Google Scholar
  3. 3.
    Cagnoni, S., Valli, G.: OSLVQ: a training strategy for optimum-size learning vector quantization classifiers. In: IEEE International Conference on Neural Networks, IEEE WCCI 1994, vol. 2, pp. 762–765 (1994)Google Scholar
  4. 4.
    D’Addese, G.: Individuazione di Sottoinsiemi Rilevanti in Sistemi Dinamici. Bachelor thesis, University of Modena and Reggio Emilia, Italy (2017)Google Scholar
  5. 5.
    Faivishevsky, L., Goldberger, J.: A nonparametric information theoretic clustering algorithm. In: Proceedings of the 27th International Conference on Machine Learning ICML 2010, pp. 351–358 (2010)Google Scholar
  6. 6.
    Filisetti, A., Villani, M., Roli, A., Fiorucci, M., Serra, R.: Exploring the organisation of complex systems through the dynamical interactions among their relevant subsets. In: Andrews, P., et al. (eds.) ECAL 2015, pp. 286–293. The MIT Press, Cambridge (2015)Google Scholar
  7. 7.
    Kohonen, T.: Learning vector quantization. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 537–540. MIT Press, Cambridge (1998)Google Scholar
  8. 8.
    Müller, A.C., Nowozin, S., Lampert, C.H.: Information theoretic clustering using minimum spanning trees. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 205–215. Springer, Heidelberg (2012). Scholar
  9. 9.
    Sani, L., et al.: Efficient search of relevant structures in complex systems. In: Adorni, G., Cagnoni, S., Gori, M., Maratea, M. (eds.) AI*IA 2016. LNCS (LNAI), vol. 10037, pp. 35–48. Springer, Cham (2016). Scholar
  10. 10.
    Silvestri, G., et al.: Searching relevant variable subsets in complex systems using k-means PSO. In: Pelillo, M., Poli, I., Roli, A., Serra, R., Slanzi, D., Villani, M. (eds.) WIVACE 2017. CCIS, vol. 830, pp. 308–321. Springer, Cham (2018). Scholar
  11. 11.
    Tononi, G., McIntosh, A., Russel, D., Edelman, G.: Functional clustering: Identifying strongly interactive brain regions in neuroimaging data. Neuroimage 7, 133–149 (1998)CrossRefGoogle Scholar
  12. 12.
    Tononi, G., Sporns, O., Edelman, G.M.: A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc. Nat. Acad. Sci. 91(11), 5033–5037 (1994)CrossRefGoogle Scholar
  13. 13.
    Ver Steeg, G., Galstyan, A., Sha, F., DeDeo, S.: Demystifying information-theoretic clustering. In: Proceedings of the 31st International Conference on International Conference on Machine Learning ICML 2014, pp. I-19–I-27 (2014)Google Scholar
  14. 14.
    Vicari, E., et al.: GPU-based parallel search of relevant variable sets in complex systems. In: Rossi, F., Piotto, S., Concilio, S. (eds.) WIVACE 2016. CCIS, vol. 708, pp. 14–25. Springer, Cham (2017). Scholar
  15. 15.
    Villani, M., Roli, A., Filisetti, A., Fiorucci, M., Poli, I., Serra, R.: The search for candidate relevant subsets of variables in complex systems. Artif. Life 21(4), 412–431 (2015)CrossRefGoogle Scholar
  16. 16.
    Villani, M., et al.: A relevance index method to infer global properties of biological networks. In: Pelillo, M., Poli, I., Roli, A., Serra, R., Slanzi, D., Villani, M. (eds.) WIVACE 2017. CCIS, vol. 830, pp. 129–141. Springer, Cham (2018). Scholar
  17. 17.
    Villani, M., et al.: An iterative information-theoretic approach to the detection of structures in complex systems. Complexity (2018, in press)Google Scholar
  18. 18.
    Wang, M., Sha, F.: Information theoretical clustering via semidefinite programming. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 761–769 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Laura Sani
    • 1
  • Gianluca D’Addese
    • 2
  • Riccardo Pecori
    • 1
    • 3
    Email author
  • Monica Mordonini
    • 1
  • Marco Villani
    • 2
  • Stefano Cagnoni
    • 1
  1. 1.Department of Engineering and ArchitectureUniversity of ParmaParmaItaly
  2. 2.FIM DepartmentUniversity of Modena and Reggio EmiliaModenaItaly
  3. 3.SMARTEST Research CentreeCAMPUS UniversityNovedrateItaly

Personalised recommendations