Abstract
With the exponential growth of collected data in different fields like recommender system (user, items), text mining (document, term), bioinformatics (individual, gene), co-clustering, which is a simultaneous clustering of both dimensions of a data matrix, has become a popular technique. Co-clustering aims to obtain homogeneous blocks leading to a straightforward simultaneous interpretation of row clusters and column clusters. Many approaches exist; in this paper, we rely on the latent block model (LBM), which is flexible, allowing to model different types of data matrices. We extend its use to the case of a tensor (3D matrix) data in proposing a Tensor LBM (TLBM), allowing different relations between entities. To show the interest of TLBM, we consider continuous, binary, and contingency tables datasets. To estimate the parameters, a variational EM algorithm is developed. Its performances are evaluated on synthetic and real datasets to highlight different possible applications.
Similar content being viewed by others
References
Ailem, M., Role, F., Nadif, M.: Model-based co-clustering for the effective handling of sparse data. Pattern Recognit. 72, 108–122 (2017)
Ailem, M., Role, F., Nadif, M.: Sparse poisson latent block model for document clustering. IEEE Trans. Knowl. Data Eng. 29(7), 1563–1576 (2017)
Banerjee, A., Krumpelman, C., Ghosh, J., Basu, S., Mooney, R.J.: Model-based overlapping clustering. In: Proceedings of the Eleventh ACM SIGKDD, pp. 532–537 (2005)
Bouchareb, A., Boullé, M., Clérot, F., Rossi, F.: Co-clustering based exploratory analysis of mixed-type data tables. In: Advances in Knowledge Discovery and Management, pp. 23–41. Springer (2019)
Bourgeois, F., Lassalle, J.C.: An extension of the Munkres algorithm for the assignment problem to rectangular matrices. Commun. ACM 14(12), 802–804 (1971)
Boutalbi, R., Labiod, L., Nadif, M.: Co-clustering from tensor data. In: Yang, Q., Zhou, Z.H., Gong, Z., Zhang, M.L., Huang, S.J. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 370–383 (2019)
Briand, A.S., Côme, E., El Mahrsi, M.K., Oukhellou, L.: A mixture model clustering approach for temporal passenger pattern characterization in public transport. Int. J. Data Sci. Anal. 1(1), 37–50 (2016)
Celeux, G., Govaert, G.: A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14(3), 315–332 (1992)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977)
Deodhar, M., Ghosh, J.: SCOAL: a framework for simultaneous co-clustering and learning from complex data. ACM Trans. Knowl. Discov. Data 4, 1–31 (2010)
Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. In: Proceedings of the Ninth ACM SIGKDD, pp. 89–98 (2003)
Feizi, S., Javadi, H., Tse, D.: Tensor biclustering. In: Advances in Neural Information Processing Systems, vol. 30, pp. 1311–1320. Curran Associates, Inc. (2017)
Fraley, C., Raftery, A.E.: How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput. J. 41(8), 578–588 (1998)
Govaert, G., Nadif, M.: Comparison of the mixture and the classification maximum likelihood in cluster analysis with binary data. Comput. Stat. Data Anal. 23(1), 65–81 (1996)
Govaert, G., Nadif, M.: Clustering with block mixture models. Pattern Recognit. 36, 463–473 (2003)
Govaert, G., Nadif, M.: An EM algorithm for the block mixture model. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 643–647 (2005)
Govaert, G., Nadif, M.: Fuzzy clustering to estimate the parameters of block mixture models. Soft. Comput. 10(5), 415–422 (2006)
Govaert, G., Nadif, M.: Block clustering with bernoulli mixture models: comparison of different approaches. Comput. Stat. Data Anal. 52(6), 3233–3245 (2008)
Govaert, G., Nadif, M.: Co-clustering. Wiley-IEEE Press, Hoboken (2013)
Govaert, G., Nadif, M.: Mutual information, phi-squared and model-based co-clustering for contingency tables. Adv. Data Anal. Classif. 12(3), 455–488 (2018)
Haralick, R., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. 3(6), 610–621 (1973)
Kossaifi, J., Panagakis, Y., Anandkumar, A., Pantic, M.: Tensorly: tensor learning in python (2018). CoRR arXiv:1610.09555
Kumar, R.M., Sreekumar, K.: A survey on image feature descriptors. Int. J. Comput. Sci. Inf. Technol. (IJCSIT) 5(1), 7668–7673 (2014)
Kurban, H., Jenne, M., Dalkilic, M.M.: Using data to build a better em: Em* for big data. Int. J. Data Sci. Anal. 4(2), 83–97 (2017)
Labiod, L., Nadif, M.: Co-clustering under nonnegative matrix tri-factorization. In: International Conference on Neural Information Processing, pp. 709–717. Springer (2011)
Labiod, L., Nadif, M.: A unified framework for data visualization and coclustering. IEEE Trans. Neural Netw. Learn. Syst. 26(9), 2194–2199 (2014)
Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)
Pagès, J.: Multiple Factor Analysis by Example Using R. Chapman and Hall, London (2014)
Role, F., Morbieu, S., Nadif, M.: Coclust: a python package for co-clustering. J. Stat. Softw. 88, 1–29 (2019)
Salah, A., Nadif, M.: Model-based von Mises-Fisher co-clustering with a conscience. In: Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 246–254. SIAM (2017)
Salah, A., Nadif, M.: Directional co-clustering. In: Advances in Data Analysis and Classification, pp. 1–30 (2018)
Steinley, D.: Properties of the Hubert-Arabie adjusted rand index. Psychol. Methods 9(3), 386 (2004)
Strehl, A., Ghosh, J.: Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)
Vu, D., Aitkin, M.: Variational algorithms for biclustering models. In: Computational Statistics and Data Analysis, pp. 12–24 (2015)
Wu, T., Benson, A.R., Gleich, D.F.: General tensor spectral co-clustering for higher-order data. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 2559–2567. Curran Associates, Inc., Red Hook (2016)
Zhang, H., Yang, M., Yang, W., Lv, J.: Spatial-aware hyperspectral image classification via multifeature kernel dictionary learning. Int. J. Data Sci. Anal. 7(2), 115–129 (2019)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This submission is an extension version of the PAKDD 2019 paper ’Co-clustering from Tensor Data’.
Appendices
Appendix: Update \(\tilde{z}_{ik}\) and \(\tilde{w}_{j\ell }\) \(\forall i,k,j,\ell \)
To obtain the expression of \(\tilde{z}_{ik}\), we maximize the above soft criterion \(F_C({\tilde{{\mathbf {z}}}},{\tilde{{\mathbf {w}}}};\Omega )\) with respect to \(\tilde{z}_{ik}\), subject to the constraint \(\sum _k \tilde{z}_{ik}=1\). The corresponding Lagrangian, up to terms which are not function of \(\tilde{z}_{ik}\), is given by :
Taking derivatives with respect to \(\tilde{z}_{ik}\), we obtain:
Setting this derivative to zero yields:
Summing both sides over all \(k'\) yields
Plugging \(\exp (\beta )\) in \(\tilde{z}_{ik}\) leads to:
In the same way, we can estimate \(\tilde{w}_{jk}\) maximizing \(F_C({\tilde{{\mathbf {z}}}},{\tilde{{\mathbf {w}}}};\Omega )\) with respect to \(\tilde{w}_{j\ell }\), subject to the constraint \(\sum _\ell \tilde{w}_{j\ell }=1\); we obtain
Appendix: Estimation of the \(\mu _{k\ell }\)’s and \(\varSigma _{k\ell }\)’s of Gaussian TLBM
The \(\mu _{k\ell }\)’s and \(\varSigma _{k\ell }\)’s can be obtained from the following derivatives:
where
with \( \tilde{z}_{.k}=\sum _i{\tilde{z}_{ik}}\) and \(\tilde{w}_{.\ell }=\sum _j \tilde{w}_{j\ell }.\) The following formulas involving the vector-by-vector (\({\mathbf {x}}\)) and matrix-by-matrix (\({\mathbf {M}}\)) derivatives
lead to
and
The two partial derivatives set to 0 lead to
and
Rights and permissions
About this article
Cite this article
Boutalbi, R., Labiod, L. & Nadif, M. Tensor latent block model for co-clustering. Int J Data Sci Anal 10, 161–175 (2020). https://doi.org/10.1007/s41060-020-00205-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41060-020-00205-5