Advertisement

Visualization of Structured Data via Generative Probabilistic Modeling

  • Nikolaos Gianniotis
  • Peter Tiňo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5400)

Abstract

We propose a generative probabilistic approach to constructing topographic maps of sequences and tree-structured data. The model formulation specifies a low-dimensional manifold of local noise models on the structured data. The manifold of noise models is induced by a smooth mapping from a low dimensional Euclidean latent space to the parameter space of local noise models. In this paper, we consider noise models endowed with hidden Markovian state space structure, namely Hidden Markov Tree Models (HMTM) and Hidden Markov Models (HMM). Compared with recursive extensions of the traditional Self-Organizing Map that can be used to visualize sequential or tree-structured data, topographic maps formulated within this framework possess a number of advantages such as a well defined cost function that drives the model optimization, the ability to test for overfitting and the accommodation of alternative local noise models implicitly expressing different notions of structured data similarity. Additionally, using information geometry one can calculate magnification factors on the constructed topographic maps. Magnification factors are a useful tool for cluster detection in non-linear topographic map formulations. We demonstrate the framework on two artificial data sets and chorals by J.S. Bach represented as sequences, as well as on images represented as trees.

Keywords

Hide Markov Model Data Item Noise Model Hide State Topographic Organization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kohonen, T.: The self-organizing map. Proceedings of the IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  2. 2.
    Hammer, B., Micheli, A., Strickert, M., Sperduti, A.: A general framework for unsupervised processing of structured data. Neurocomputing 57, 3–35 (2004)CrossRefGoogle Scholar
  3. 3.
    Svensén, M.: GTM: The Generative Topographic Mapping. Ph.D thesis, Aston University, UK (1998)Google Scholar
  4. 4.
    Kabán, A., Girolami, M.: A combined latent class and trait model for the analysis and visualization of discrete data. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(8), 859–872 (2001)CrossRefGoogle Scholar
  5. 5.
    Kleiberg, E., van de Wetering, H., van Wijk, J.J.: Botanical visualization of huge hierarchies. In: IEEE Symposium on Information Visualization, INFOVIS, pp. 87–94 (2001)Google Scholar
  6. 6.
    Chappell, G., Taylor, J.: The temporal kohonen map. Neural Networks 6, 441–445 (1993)CrossRefGoogle Scholar
  7. 7.
    Koskela, T., Heikkonen, M.V.z.J., Kaski, K.: Recurrent SOM with local linear models in time series prediction. In: 6th European Symposium on Artificial Neural Networks, pp. 167–172 (1998)Google Scholar
  8. 8.
    Horio, K., Yamakawa, T.: Feedback self-organizing map and its application to spatio-temporal pattern classification. International Journal of Computational Intelligence and Applications 1(1), 1–18 (2001)CrossRefGoogle Scholar
  9. 9.
    Voegtlin, T.: Recursive self-organizing maps. Neural Networks 15(8-9), 979–991 (2002)CrossRefPubMedGoogle Scholar
  10. 10.
    Strickert, M., Hammer, B.: Merge SOM for temporal data. Neurocomputing 64, 39–71 (2005)CrossRefGoogle Scholar
  11. 11.
    Hagenbuchner, M., Sperduti, A., Tsoi, A.C.: A self-organizing map for adaptive processing of structured data. IEEE Transactions on Neural Networks 14(3), 491–505 (2003)CrossRefPubMedGoogle Scholar
  12. 12.
    Hagenbuchner, M., Sperduti, A., Tsoi, A.C.: Contextual processing of graphs using self-organizing maps. In: Proceedings of the European Symposium on Artificial Neural Networks (ESANN), pp. 399–404 (2005)Google Scholar
  13. 13.
    Heskes, T.: Energy functions for self-organizing maps. In: Oja, S., Kaski, E. (eds.) Kohonen Maps, pp. 303–315. Elsevier, Amsterdam (1999)CrossRefGoogle Scholar
  14. 14.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10(1), 215–234 (1998)CrossRefGoogle Scholar
  15. 15.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  16. 16.
    Gianniotis, N., Tiňo, P.: Visualisation of tree-structured data through generative topographic mapping. IEEE Transactions on Neural Networks (2008) (in press)Google Scholar
  17. 17.
    Crouse, M., Nowak, R., Baraniuk, R.: Wavelet -Based Statistical Signal Processing Using Hidden Markov Models. IEEE Transactions on Signal Processing 46(4), 886–902 (1998)CrossRefGoogle Scholar
  18. 18.
    Durand, J.B.: Gonçalvès, P.: Statistical inference for hidden Markov tree models and application to wavelet trees. Technical Report 4248, INRIA (2001)Google Scholar
  19. 19.
    Durand, J.B., Gonçalvès, P., Guedon, Y.: Computational methods for hidden markov tree models-an application to wavelet trees. IEEE Transactions on Signal Processing 52(9), 2552–2560 (2004)CrossRefGoogle Scholar
  20. 20.
    Tiňo, P., Kaban, A., Sun, Y.: A generative probabilistic approach to visualizing sets of symbolic sequences. In: KDD 2004: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–706. ACM Press, New York (2004)Google Scholar
  21. 21.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: Magnification factors for the gtm algorithm. In: Proceedings IEE Fifth International Conference on Artificial Neural Networks, pp. 64–69 (1997)Google Scholar
  22. 22.
    Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley-Interscience, Hoboken (1991)CrossRefGoogle Scholar
  23. 23.
    Kullback, S.: Information theory and statistics. Wiley, New York (1959)Google Scholar
  24. 24.
    Tiňo, P., Gianniotis, N.: Metric properties of structured data visualizations through generative probabilistic modeling. In: IJCAI 2007: 20th International Joint Conference on Artificial Intelligence, pp. 1083–1088. AAAI Press, Menlo Park (2007)Google Scholar
  25. 25.
    Do, M.N.: Fast approximation of Kullback-Leibler distance for dependence trees and hidden markov models. IEEE Signal Processing Letters 10(4), 115–118 (2003)CrossRefGoogle Scholar
  26. 26.
    Merz, C., Murphy, P.: UCI repository of machine learning databases (1998)Google Scholar
  27. 27.
    Hagenbuchner, M., Tsoi, A.: The traffic policeman benchmark. In: Verleysen, M. (ed.) European Symposium on Artificial Neural Networks, April 1999, pp. 63–68. D-Facto (1999)Google Scholar
  28. 28.
    Kabán, A., Girolami, M.: A combined latent class and trait model for the analysis and visualization of discrete data. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(8), 859–872 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Nikolaos Gianniotis
    • 1
  • Peter Tiňo
    • 1
  1. 1.School of Computer ScienceUniversity of BirminghamBirminghamUnited Kingdom

Personalised recommendations