Neural Processing Letters

, Volume 38, Issue 1, pp 17–27 | Cite as

Convolutional Deep Networks for Visual Data Classification



This paper develops a semi-supervised learning algorithm called convolutional deep networks (CDN), to address the image classification problem with deep learning. First, we construct the previous several hidden layers using convolutional restricted Boltzmann machines, which can reduce the dimension and abstract the information of the images effectively. Second, we construct the following hidden layers using restricted Boltzmann machines, which can abstract the information of images quickly. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. CDN can reduce the dimension and abstract the information of the images at the same time efficiently. More importantly, the abstraction and classification procedure of CDN use the same deep architecture to optimize the same parameter in different steps continuously, which can improve the learning ability effectively. We did several experiments on two standard image datasets, and show that CDN are competitive with both representative semi-supervised classifiers and existing deep learning techniques.


Semi-supervised learning Deep learning Convolutional neural networks Visual data classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, CambridgeGoogle Scholar
  2. 2.
    Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7: 1687–1712MathSciNetMATHGoogle Scholar
  3. 3.
    Desjardins G, Bengio Y (2008) Empirical evaluation of convolutional rbms for vision. Tech. repGoogle Scholar
  4. 4.
    Feng GY, Hu DW, Zhou ZT (2008) A direct locality preserving projections (DLPP) algorithm for image recognition. Neural Process Lett 27(3): 247–255CrossRefGoogle Scholar
  5. 5.
    Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8): 1771–1800MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Hinton GE (2010) Learning to represent visual input. Philos Trans R Soc B-Biol Sci 365(1537): 177–184CrossRefGoogle Scholar
  7. 7.
    Hinton GE, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313: 504–507MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18: 1527–1554MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4): 541–551CrossRefGoogle Scholar
  10. 10.
    Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324CrossRefGoogle Scholar
  11. 11.
    Lee H, Grosse R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: International conference on machine learning. ACM, Montreal, Canada, pp 609–616Google Scholar
  12. 12.
    Li F, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR workshop on generative-model based vision. Elsevier Science Inc., New York, pp 1–9Google Scholar
  13. 13.
    Liu Y, Zhou S, Chen Q (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44(10–11): 2287–2296MATHCrossRefGoogle Scholar
  14. 14.
    Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: International conference on multimedia. ACM, New York, pp 83–92Google Scholar
  15. 15.
    Salakhutdinov R, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. J Mach Learn Res 2: 412–419Google Scholar
  16. 16.
    Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. In: International conference on machine learning. ACM, Helsinki, pp 1168–1175Google Scholar
  17. 17.
    Zhu X (2007) Semi-supervised learning literature survey. Tech. rep.. University of Wisconsin Madison, MadisonGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Key Laboratory of Network Oriented Intelligent Computation, Shenzhen Graduate SchoolHarbin Institute of TechnologyHarbinPeople’s Republic of China

Personalised recommendations