Towards Sparsity and Selectivity: Bayesian Learning of Restricted Boltzmann Machine for Early Visual Features
This paper exploits how Bayesian learning of restricted Boltzmann machine (RBM) can discover more biologically-resembled early visual features. The study is mainly motivated by the sparsity and selectivity of visual neurons’ activations in V1 area. Most previous work of computational modeling emphasize selectivity and sparsity independently, which neglects the underlying connections between them. In this paper, a prior on parameters is defined to simultaneously enhance these two properties, and a Bayesian learning framework of RBM is introduced to infer the maximum posterior of the parameters. The proposed prior performs as the lateral inhibition between neurons. According to our empirical results, the visual features learned from the proposed Bayesian framework yield better discriminative and generalization capability than the ones learned with maximum likelihood, or other state-of-the-art training strategies.
KeywordsMarkov Chain Monte Carlo Natural Image Sparse Code Contrastive Divergence Bayesian Learning
Unable to display preview. Download preview PDF.
- 2.Berkes, P., White, B., Fiser, J.: No evidence for active sparsification in the visual cortex. In: NIPS, pp. 108–116 (2009)Google Scholar
- 5.Goh, H., Thome, N., Cord, M.: Biasing restricted Boltzmann machines to manipulate latent selectivity and sparsity. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2010)Google Scholar
- 8.Lee, H., Ekanadham, C., Ng, A.Y.: Sparse deep belief net model for visual area V2. In: NIPS (2007)Google Scholar
- 9.Luo, H., Shen, R., Niu, C., Ullrich, C.: Sparse group restricted boltzmann machines. In: AAAI (2011)Google Scholar
- 11.Salakhutdinov, R., Murray, I.: On the quantitative analysis of deep belief networks. In: ICML (2008)Google Scholar