Skip to main content
Log in

Convolutional Deep Networks for Visual Data Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper develops a semi-supervised learning algorithm called convolutional deep networks (CDN), to address the image classification problem with deep learning. First, we construct the previous several hidden layers using convolutional restricted Boltzmann machines, which can reduce the dimension and abstract the information of the images effectively. Second, we construct the following hidden layers using restricted Boltzmann machines, which can abstract the information of images quickly. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. CDN can reduce the dimension and abstract the information of the images at the same time efficiently. More importantly, the abstraction and classification procedure of CDN use the same deep architecture to optimize the same parameter in different steps continuously, which can improve the learning ability effectively. We did several experiments on two standard image datasets, and show that CDN are competitive with both representative semi-supervised classifiers and existing deep learning techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge

    Google Scholar 

  2. Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7: 1687–1712

    MathSciNet  MATH  Google Scholar 

  3. Desjardins G, Bengio Y (2008) Empirical evaluation of convolutional rbms for vision. Tech. rep

  4. Feng GY, Hu DW, Zhou ZT (2008) A direct locality preserving projections (DLPP) algorithm for image recognition. Neural Process Lett 27(3): 247–255

    Article  Google Scholar 

  5. Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8): 1771–1800

    Article  MathSciNet  MATH  Google Scholar 

  6. Hinton GE (2010) Learning to represent visual input. Philos Trans R Soc B-Biol Sci 365(1537): 177–184

    Article  Google Scholar 

  7. Hinton GE, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313: 504–507

    Article  MathSciNet  MATH  Google Scholar 

  8. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18: 1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  9. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4): 541–551

    Article  Google Scholar 

  10. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324

    Article  Google Scholar 

  11. Lee H, Grosse R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: International conference on machine learning. ACM, Montreal, Canada, pp 609–616

  12. Li F, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR workshop on generative-model based vision. Elsevier Science Inc., New York, pp 1–9

  13. Liu Y, Zhou S, Chen Q (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44(10–11): 2287–2296

    Article  MATH  Google Scholar 

  14. Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: International conference on multimedia. ACM, New York, pp 83–92

  15. Salakhutdinov R, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. J Mach Learn Res 2: 412–419

    Google Scholar 

  16. Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. In: International conference on machine learning. ACM, Helsinki, pp 1168–1175

  17. Zhu X (2007) Semi-supervised learning literature survey. Tech. rep.. University of Wisconsin Madison, Madison

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shusen Zhou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, S., Chen, Q. & Wang, X. Convolutional Deep Networks for Visual Data Classification. Neural Process Lett 38, 17–27 (2013). https://doi.org/10.1007/s11063-012-9260-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9260-y

Keywords

Navigation