Local Invariance Representation Learning Algorithm with Multi-layer Extreme Learning Machine

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9950)


Multi-layer extreme learning machine (ML-ELM) is a stacked extreme learning machine based auto-encoding (ELM-AE). It provides an effective solution for deep feature extraction with higher training efficiency. To enhance the local-input invariance of feature extraction, we propose a contractive multi-layer extreme learning machine (C-ML-ELM) by adding a penalty term in the optimization function to minimize derivative of output to input at each hidden layer. In this way, the extracted feature is supposed to keep consecutiveness attribution of an image. The experiments have been done on MNIST handwriting dataset and face expression dataset CAFÉ. The results show that it outperforms several state-of-art classification algorithms with less error and higher training efficiency.


Local invariant representation learning Multi-layer extreme learning Contractive auto-encoder 



This research is partially sponsored by the National Nature Science Foundation of China (Nos. 61672070, 61370113, 91546111), Beijing Municipal Natural Science Foundation (No. 4152005), Key projects of Beijing Municipal Education Commission (No. KZ201610005009).


  1. 1.
    Kumar, G., Bhatia, P.K.: A detailed review of feature extraction in image processing systems. In: IEEE Fourth International Conference on Advanced Computing and Communication Technologies, pp. 5–12 (2014)Google Scholar
  2. 2.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)CrossRefMATHGoogle Scholar
  3. 3.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2013)Google Scholar
  4. 4.
    Jia, X., Zhang, Y., Powers, D., Ali, H.B.: Multi-classifier fusion based facial expression recognition approach. KSII Trans. Internet Inf. Syst. 8(1), 196–212 (2014)CrossRefGoogle Scholar
  5. 5.
    Zavaschi, T.H.H., Britto Jr., A.S., Oliveira, L.E.S., Koerich, A.L.: Fusion of feature sets and classifiers for facial expression recognition. Expert Syst. Appl. 40(2), 646–655 (2013)CrossRefGoogle Scholar
  6. 6.
    Jing, P., Bhanu, B.: Delayed reinforcement learning for adaptive image segmentation and feature extraction. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 28(3), 482–488 (1998)CrossRefGoogle Scholar
  7. 7.
    Liu, S., Bai, X.: Discriminative features for image classification and retrieval. In: IEEE International Conference on Image and Graphics, pp. 744–751 (2011)Google Scholar
  8. 8.
    Chen, M., Zhang, L., Allebach, J.P.: Learning deep features for image emotion classification. In: IEEE International Conference on Image Processing. IEEE (2015)Google Scholar
  9. 9.
    Kasun, L.L.C., Zhou, H., Huang, G.-B., Vong, C.M.: Representational learning with extreme learning machine for big data. IEEE Intell. Syst. 28(6), 31–34 (2013)Google Scholar
  10. 10.
    Vincent, P., et al.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(6), 3371–3408 (2010)MathSciNetMATHGoogle Scholar
  11. 11.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Rifai, S., Vincent, P., Muller, X., et al.: Contractive auto-encoders: explicit invariance during feature extraction. In: Proceedings of the 28th International Conference on Machine Learning (ICML-2011), Bellevue, WA, USA, pp. 833–840 (2011)Google Scholar
  13. 13.
    LeCun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  14. 14.
    Dailey, M.N., Cottrell, G.W., Reilly, J.: CAlifornia Facial Expressions (CAFE) (2001). Computer Science and Engineering Department, UCSD, La Jolla (2011). http://www.cs.ucsd.edu/users/gary/CAFE/

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Beijing Key Laboratory on Integration and Analysis of Large-scale Stream DataBeijing University of TechnologyBeijingChina
  2. 2.Center for Research in Intelligent SystemsUniversity of California at RiversideRiversideUSA

Personalised recommendations