Neural Processing Letters

, Volume 47, Issue 3, pp 1011–1025 | Cite as

Hierarchical Tensor SOM Network for Multilevel–Multigroup Analysis



The aim of this work is to develop a visualization method for multilevel–multigroup analysis based on a multiway nonlinear dimensionality reduction. The task of the method is to visualize what kinds of members each group is composed and to visualize the similarity between the groups in terms of probability distribution of constituent members. To achieve the task, the proposed method consists of hierarchically coupled tensor self-organizing maps, corresponding to the member/group level. This architecture enables more flexible analysis than ordinary parametric multilevel analysis, as it retains a high level of interpretability supported by strong visualization. We applied the proposed method to one benchmark dataset and two practical datasets: one is the survey data on the football players belonging to different teams and the other is the employee survey data belonging to different departments in a company. Our method successfully visualizes the types of the members that constitute each group as well as visualizes the differences or similarities between the groups.


Multilevel analysis Multigroup analysis Tensor decomposition Self-organizing map SOM 



We would like to thank Prof. Isogai and Dr. Shinriki, Kyushu Institute of Technology, for allowing us to use football player data. We also would like to thank Dr. Iwasaki for much assistance in developing analysis tools and useful discussion.


  1. 1.
    Argyriou A, Evgeniou T, Pontil M (2007) Multi-task feature learning. In: Proceedings of the 20th annual conference on neural information processing systems, pp 41–48Google Scholar
  2. 2.
    Bishop CM, Svensen M, Williams CKI (1998) GTM: the generative topographic mapping. Neural Comput 10:215–234CrossRefMATHGoogle Scholar
  3. 3.
    Cao N, Cui W (2016) Introduction to text visualization techniques, vol 1. Atlantis Briefs in Artificial Intelligence, pp 11–40. doi: 10.2991/978-94-6239-186-4
  4. 4.
    Caruana R (1997) Multitask learning. Mach Learn 28:41–75CrossRefGoogle Scholar
  5. 5.
    Ceulemans E, Wilderjans TF, Kiers HAL, Timmerman ME (2016) Multilevel simultaneous component analysis: a computational shortcut and software package. Behav Res Methods 48:1008–1020. doi: 10.3758/s13428-015-0626-8 CrossRefGoogle Scholar
  6. 6.
    Csurka G, Bray C, Dance C, Fan L (2004) Visual categorization with bags of keypoints. In: ECCV international workshop on statistical learning in computer vision, pp 1–22Google Scholar
  7. 7.
    de Leeuw J, Meijer E (2008) Handbook of multilevel analysis. Springer, New YorkCrossRefMATHGoogle Scholar
  8. 8.
    De Roover K, Timmerman ME, Mesquita B, Ceulemans E (2013) Common and cluster-specific simultaneous component analysis. PLoS ONE 8:e62280. doi: 10.1371/journal.pone.0062280 CrossRefGoogle Scholar
  9. 9.
    De Roover K, Ceulemans E, Giordani P (2016) Overlapping clusterwise simultaneous component analysis. Chemom Intell Lab Syst 156:249–259CrossRefGoogle Scholar
  10. 10.
    Eslami A, Qannari EM, Kohler A, Bougeard S (2013) General overview of methods of analysis of multi-group datasets. Revue des Nouvelles Technologies de l fInformation 25:108–123MATHGoogle Scholar
  11. 11.
    Eslamia A, Qannarib EM, Kohlerd A, Bougeard S (2014) Algorithms for multi-group PLS. J Chemom 28:192–201. doi: 10.1002/cem.2593 Google Scholar
  12. 12.
    Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 109–117Google Scholar
  13. 13.
    French BF, Finch WH (2008) Multigroup confirmatory factor analysis: locating the invariant referent sets. Struct Equ Model 15:96–113MathSciNetCrossRefGoogle Scholar
  14. 14.
    Friedman JH (1987) Exploratory projection pursuit. J Am Stat Assoc 82:259–266MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Hu L, Cao J, Xu G, Wang J, Gu Z, Cao L (2013) Cross-domain collaborative filtering via bilinear multilevel analysis. In: Proceedings of the twenty-third international joint conference on artificial intelligence. AAAI Press, Beijing, pp 2626–2632Google Scholar
  16. 16.
    Ishibashi H, Furukawa T (2016) Multilevel–multigroup analysis by hierarchical tensor SOM network. In: Proceedings of ICONIP, pp 459–466Google Scholar
  17. 17.
    Iwasaki T, Furukawa T (2016) Tensor SOM and Tensor GTM: nonlinear tensor analysis by topographic mappings. Neural Netw 77:107–125CrossRefGoogle Scholar
  18. 18.
    Iwata T, Saito K, Ueda N, Stromsten S, Griffiths TL, Tenenbaum JB (2007) Parametric embedding for class visualization. Neural Comput 19:2536–2556CrossRefMATHGoogle Scholar
  19. 19.
    Iwata T, Yamada T, Ueda N (2008) Probabilistic latent semantic visualization: topic model for visualizing documents. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 363–371. doi: 10.1145/1401890.1401937
  20. 20.
    Kamishima T, Kazawa H, Akaho S (2010) A survey and empirical comparison of object ranking methods. In: Fürnkranz J, Hüllermeier E (eds) Preference learning. Springer, Berlin, pp 181–201CrossRefGoogle Scholar
  21. 21.
    Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Kwok JTY, Tsang IWH (2004) The pre-image problem in kernel methods. IEEE Trans Neural Netw 15(6):1517–1525. doi: 10.1109/tnn.2004.837781.
  24. 24.
    Lawrence ND (2004) Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in neural information processing systems. Neural information processing systems foundationGoogle Scholar
  25. 25.
    Lawrence ND (2005) Probabilistic non-linear principal component analysis with Gaussian process. J Mach Learn Res 6:1783–1816MathSciNetMATHGoogle Scholar
  26. 26.
    Le T, Lauw HW (2015) Semantic visualization for spherical representation. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1007–1016Google Scholar
  27. 27.
    Nguyen V, Phung DQ, Nguyen X, Venkatesh S, Bui HH (2014) Bayesian nonparametric multilevel clustering with group-level contexts. In: Proceedings of international conference on machine learning (ICML), pp 288–296Google Scholar
  28. 28.
    Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359CrossRefGoogle Scholar
  29. 29.
    Schouteden M, Van Deun K, Wilderjans TF, Van Mechelen I (2013) Performing DISCO-SCA to search for distinctive and common information in linked data. Behav Res Methods 46:576–587Google Scholar
  30. 30.
    Stefanovic P, Kurasova O (2011) Visual analysis of self-organzing maps. Nonlinear Anal Model Control 16(4):488–504Google Scholar
  31. 31.
    Ten Berge JMF, Kiers HAL, Van der Stel V (2011) A clusterwise simultaneous component method for capturing within-cluster differences in component variances and correlations. Br J Math Stat Psychol 66:81–102. doi: 10.1111/j.2044-8317.2012.02040.x MathSciNetGoogle Scholar
  32. 32.
    Ten Berge JMF, Kiers HAL, Van der Stel V (2013) Simultaneous components analysis. Stat Appl 4:377–392Google Scholar
  33. 33.
    Timmerman ME (2006) Multilevel component analysis. Br J Math Stat Psychol 59:301–320MathSciNetCrossRefGoogle Scholar
  34. 34.
    Tsai CF (2012) Bag-of-words representation in image annotation: a review. In: Proceedings of the 9th European conference on computer visionGoogle Scholar
  35. 35.
    Ultsch A, Siemon HP (1990) Kohonen’s self organizing feature maps for exploratory data analysis. In: Proceedings INNC’90, international neural network conference, pp 305–308Google Scholar
  36. 36.
    Wulsin D, Jensen S, Litt B (2012) A hierarchical dirichlet process model with multiple levels of clustering for human eeg seizure modeling. In: Proceedings of the 29th international conference on machine learningGoogle Scholar
  37. 37.
    Zhou G, Cichocki A, Zhang Y, Mandic D (2015) Group component analysis for multiblock data: common and individual feature extraction. IEEE Trans Neural Netw Learn Syst 27:2426–2439MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Human Intelligence SystemsKyushu Institute of TechnologyKitakyushuJapan

Personalised recommendations