Skip to main content
Log in

Compressing arrays of classifiers using Volterra-neural network: application to face recognition

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Model compression is required when large models are used, for example, for a classification task, but there are transmission, space, time, or computing constraints that have to be fulfilled. Multilayer perceptron (MLP) models have been traditionally used as classifiers. Depending on the problem, they may need a large number of parameters (neuron functions, weights, and bias) to obtain an acceptable performance. This work proposes a technique to compress an array of MLPs, through the weights of a Volterra-neural network (Volterra-NN), maintaining its classification performance. It will be shown that several MLP topologies can be well-compressed into the first-, second-, and third-order (Volterra-NN) outputs. The obtained results show that these outputs can be used to build an array of (Volterra-NN) that needs significantly less parameters than the original array of MLPs, furthermore having the same high accuracy. The Volterra-NN compression capabilities were tested for solving a face recognition problem. Experimental results are presented on two well-known face databases: ORL and FERET.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html.

  2. http://www.itl.nist.gov/iad/humanid/feret.

References

  1. Aitkenhead MJ, McDonald AJS (2003) A neural network face recognition system. Eng Appl Artif Intell 16(3):167–176

    Article  Google Scholar 

  2. Bianchini M, Maggini M, Sarti L, Scarselli F (2005) Recursive neural networks learn to localize faces. Pattern Recognit Lett 26(12):1885–1895

    Article  Google Scholar 

  3. Buciluǎ C, Caruana R, Niculescu-Mizil A (2006) Model compression. In: KDD’06: proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, pp 535–541

  4. Capello D, Martinez C, Milone D, Stegmayer G (2009) Array of multilayer perceptrons with no-class resampling training fr face recognition. Revista Iberoamericana de Inteligencia Artificial 13(44):5–13

    Google Scholar 

  5. Cun YL, Denker JS, Solla SA (1990) Optimal brain damage. In: Touretzky DS (ed) Advances in neural information processing systems. Morgan Kaufmann, Los Altos, pp 598–605

  6. Duda R, Hart P (2003) Pattern classification and scene analysis. Wiley, London

    Google Scholar 

  7. Dzeroski S, Zenko B (2004) Is combining classifiers with stacking better than selecting the best one? Mach Learn 54:255–273

    Article  MATH  Google Scholar 

  8. Hassibi B, Stork DG, Com SCR (1993) Second order derivatives for network pruning: optimal brain surgeon. In: Hanson SJ, Cowan JD, Giles CL (eds) Advances in neural information processing systems 5. Morgan Kaufmann, Los Altos, pp 164–171

  9. Haykin S (1999) Neural networks: a comprehensive foundation. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  10. Jackson J (1991) A user’s guide to principal components. Wiley series in probability and mathematical statistics: applied probability and statistics. Wiley, London. http://books.google.com.ar/books?id=qhQYvH8CFQQC

  11. Kirby M, Sirovich L (1990) Application of the Karhunen–Loeve procedure for the characterization of human faces. IEEE Trans Pattern Anal Mach Intell 12(1):103–108

    Article  Google Scholar 

  12. Kong S, Heo J, Abidi B, Palk J, Abidi M (2005) Recent advances in visual and infrared face recognition-a review. Comput Vis Image Underst 97(1):103–135

    Article  Google Scholar 

  13. Korenberg M, David R, Hunter I, Solomon J (2001) Parallel cascade identification and its application to protein family prediction. J Biotechnol 91:35–47

    Article  Google Scholar 

  14. Li S, Jain A (eds) (2004) Handbook of face recognition. Springer, Berlin

    Google Scholar 

  15. Madsen K, Nielsen HB, Tingleff O (2004) Methods for non-linear least squares problems

  16. Marquardt D (1963) An algorithm for least-squares estimation of nonlinear parameters. SIAM J Appl Math 11(2):431–441

    Article  MathSciNet  MATH  Google Scholar 

  17. Martinez A, Kak A (2001) Pca versus lda. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  18. Orengo G, Colantonio P, Serino A, Giannini F, Stegmayer G, Pirola M, Ghione G (2007) Neural networks and Volterra-series for time-domain pa behavioral models. Int J RF Microw CAD Eng 17(2):160–168

    Google Scholar 

  19. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The feret evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22:1090–1104

    Article  Google Scholar 

  20. Rahman A, Verma B (2011) Novel layered clustering-based approach for generating ensemble of classifiers. IEEE Trans Neural Netw 22(5):781–792

    Article  Google Scholar 

  21. Rubiolo M, Stegmayer G, Milone D (2010) Compressing a neural network classifier using a volterra-neural network model. In: IEEE international joint conference on neural networks (IJCNN), Barcelona, Spain, pp 1–7

  22. Salomon D (2007) Data compression: the complete reference. Springer, Berlin

    Google Scholar 

  23. Stegmayer G, Chiotti O (2009) Volterra NN-based behavioral model for new wireless communications devices. Neural Comput Appl 18:283–291

    Article  Google Scholar 

  24. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):72–86

    Google Scholar 

  25. Volterra V (1959) Theory of functionals and integral and integro-differential equations. Dover, New York

    MATH  Google Scholar 

  26. Zhang D, Wangmeng Z (2007) Computational intelligence-based biometric technologies. IEEE Comput Intell Mag 2(2):26–36

    Article  Google Scholar 

  27. Zhao W, Chellappa R, Phillips P, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. Rubiolo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

PDF (128 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rubiolo, M., Stegmayer, G. & Milone, D. Compressing arrays of classifiers using Volterra-neural network: application to face recognition. Neural Comput & Applic 23, 1687–1701 (2013). https://doi.org/10.1007/s00521-012-1129-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1129-5

Keywords

Navigation