Hierarchical Representation Using NMF
In this paper, we propose a representation model that demonstrates hierarchical feature learning using nsNMF. We stack simple unit algorithm into several layers to take step-by-step approach in learning. By utilizing NMF as unit algorithm, our proposed network provides intuitive understanding of the feature development process. It is able to represent the underlying structure of feature hierarchies present in complex data in intuitively understandable manner. Experiments with document data successfully discovered feature hierarchies of concepts in data. We also observed that proposed method results in much better classification and reconstruction performance, especially for small number of features.
KeywordsHierarchical representation NMF unsupervised feature learning multi-layer deep learning
Unable to display preview. Download preview PDF.
- 1.Ahn, J.-H., Choi, S., Oh, J.-H.: A multiplicative up-propagation algorithm. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 3. ACM (2004)Google Scholar
- 3.Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Advances in Neural Information Processing Systems 19, 153 (2007)Google Scholar
- 6.Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology 160(1), 106 (1962)Google Scholar
- 8.Marcrquote Aurelio Ranzato, C.P., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy-based model. Advances in neural information processing systems 19, 1137–1144 (2007)Google Scholar
- 11.Song, H.A., Lee, S.Y.: Hierarchical data representation model-multi-layer nmf. arXiv preprint arXiv:1301.6316 (2013)Google Scholar