Advertisement

Improving Deep Neural Network Performance by Reusing Features Trained with Transductive Transference

  • Chetak Kandaswamy
  • Luís M. Silva
  • Luís A. Alexandre
  • Jorge M. Santos
  • Joaquim Marques de Sá
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)

Abstract

Transfer Learning is a paradigm in machine learning to solve a target problem by reusing the learning with minor modifications from a different but related source problem. In this paper we propose a novel feature transference approach, especially when the source and the target problems are drawn from different distributions. We use deep neural networks to transfer either low or middle or higher-layer features for a machine trained in either unsupervised or supervised way. Applying this feature transference approach on Convolutional Neural Network and Stacked Denoising Autoencoder on four different datasets, we achieve lower classification error rate with significant reduction in computation time with lower-layer features trained in supervised way and higher-layer features trained in unsupervised way for classifying images of uppercase and lowercase letters dataset.

Keywords

Feature Transference Deep Neural Network 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vapnik, V.N.: An overview of statistical learning theory. IEEE Transactions on Neural Networks / a Publication of the IEEE Neural Networks Council 10(5), 988–999 (1999)Google Scholar
  2. 2.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives (2013)Google Scholar
  3. 3.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010); Cited by 0083Google Scholar
  4. 4.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998); Cited by 2008Google Scholar
  5. 5.
    Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Amaral, T., Silva, L.M., Alexandre, L.A., Kandaswamy, C., Santos, J.M., de Sá, J.M.: Using different cost functions to train stacked auto-encoders. In: Proceedings of the 12th Mexican International Conference on Artificial Intelligence. IEEE (2013)Google Scholar
  7. 7.
    Bruzzone, L., Marconcini, M.: Domain adaptation problems: A dasvm classification technique and a circular validation strategy. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 770–787 (2010)CrossRefGoogle Scholar
  8. 8.
    de Campos, T., Babu, B.R., Varma, M.: Character recognition in natural images (2009)Google Scholar
  9. 9.
    Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), vol. 4 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Chetak Kandaswamy
    • 1
    • 2
  • Luís M. Silva
    • 1
    • 3
  • Luís A. Alexandre
    • 4
  • Jorge M. Santos
    • 1
    • 5
  • Joaquim Marques de Sá
    • 1
    • 2
  1. 1.Instituto de Engenharia Biomédica (INEB)PortoPortugal
  2. 2.Dep. de EngenhariaElectronica e de Computadores at FEUPPortoPortugal
  3. 3.Dep. de Matemática at Universidade de AveiroPortugal
  4. 4.Universidade da Beira Interior and Instituto de TelecomunicaçõesCovilhãPortugal
  5. 5.Dep. de Matemática at InstitutoSuperior de Engenharia do Instituto Politécnico do PortoPortugal

Personalised recommendations