Skip to main content
SpringerLink
Log in

Sparse Representation Using Deep Learning to Classify Multi-Class Complex Data

  • Research paper
  • Published:
Iranian Journal of Science and Technology, Transactions of Electrical Engineering Aims and scope Submit manuscript

Abstract

Extracting best feature set to reinforce discrimination is always a challenge in machine learning. In this paper, a method named General Locally Linear Combination (GLLC) is proposed to extract automatic features using a deep autoencoder and also to reconstruct a sample based on the other samples sparsely in a low-dimensional space. Extracting features along with the discrimination ability of the sparse models have created a robust classifier that shows simultaneous reduction in samples and features. To enhance the capability of this scheme, some feature sets from several layers of an autoencoder are combined and an extension of GLLC has been proposed that called here as Multi-modal General Locally Linear Combination. Although the main application of the proposed methods is in visual classification and face recognition, they have been used in other applications. Extensive experiments are conducted to demonstrate that the proposed algorithms gain high accuracy on various datasets and outperform the rival methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

(Cai et al. 2007)

Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041. https://doi.org/10.1109/TPAMI.2006.244

    Article  MATH  Google Scholar 

  • Alpaydin E (2010) Introduction to machine learning. MIT press, Cambridge

    MATH  Google Scholar 

  • Antal M, Szabó LZ, László I (2015) Keystroke dynamics on android platform. Proc Technol 19:820–826. https://doi.org/10.1016/j.protcy.2015.02.118

    Article  Google Scholar 

  • Argyriou A, Evgeniou T, Pontil M (2008) Convex multi-task feature learning. Mach Learn 73(3):243–272

    Article  Google Scholar 

  • Bach F, Jenatton R, Mairal J, Obozinski G (2012) Structured sparsity through convex optimization. Stat Sci 27:450–468

    Article  MathSciNet  MATH  Google Scholar 

  • Becker S, Bobin J, Candès EJ (2011) NESTA: a fast and accurate first-order method for sparse recovery. SIAM J Imaging Sci 4(1):1–39

    Article  MathSciNet  MATH  Google Scholar 

  • Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MATH  Google Scholar 

  • Bengio Y, Delalleau O (2011) On the expressive power of deep architectures. In: International conference on algorithmic learning theory, Springer, pp 18–36

  • Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, New York

    MATH  Google Scholar 

  • Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition machine learning (CVPR’07)

  • Caruana R (1998) Multitask learning. In: Thrun S, Pratt L (eds) Learning to learn. Springer, Berlin, pp 95–133

    Chapter  Google Scholar 

  • Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159. https://doi.org/10.1137/S003614450037906X

    Article  MathSciNet  MATH  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), IEEE, vol 1, pp 886–893

  • Demmel JW (1997) Applied numerical linear algebra. Society for Industrial and Applied Mathematics, Philadelphia

    Book  MATH  Google Scholar 

  • Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley, Hoboken

    MATH  Google Scholar 

  • He R, Zheng WS, Hu BG, Kong XW (2011) A regularized correntropy framework for robust pattern recognition. Neural Comput 23(8):2074–2100

    Article  MATH  Google Scholar 

  • He Y, Kavukcuoglu K, Wang Y, Szlam A, Qi Y (2014) Unsupervised feature learning by deep sparse coding. In: SDM, SIAM, pp 902–910

  • Ho J, Yang MH, Lim J, Lee KC, Kriegman D (2003) Clustering appearances of objects under varying illumination conditions. In: 2003 IEEE computer society conference on computer vision and pattern recognition, 2003 Proceedings. IEEE, vol 1, pp I–11

  • Huang Y, Wu Z, Wang L, Tan T (2014) Feature coding in image classification: a comprehensive study. IEEE Trans Pattern Anal Mach Intell 36(3):493–506

    Article  Google Scholar 

  • Lazebnik S, Schmid C, Ponce J (2004) Semi-local affine parts for object recognition. In: British machine vision conference (BMVC’04), The British Machine Vision Association (BMVA), pp 779–788

  • LeCun Y (2013) Deep learning tutorial. Citeseer

  • Lee H, Battle A, Raina R, Ng AY (2006) Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808

  • Li SZ, Lu J (1999) Face recognition using the nearest feature line method. IEEE Trans Neural Netw 10:439–443

    Article  Google Scholar 

  • Liou CY, Cheng WC, Liou JW, Liou DR (2014) Autoencoder for words. Neurocomputing 139:84–96

    Article  Google Scholar 

  • Lowe DG (1999) Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on computer vision, 1999, IEEE, vol 2, pp 1150–1157

  • Mairal J, Chieze J-P, Chen Y, Jenatton R, Bach F, Ponce J, Obozinski R, Yu B, Sapiro G, Harchaoui Z (2014) SPAMS: a SPArse Modeling Software, v2.5. https://spams-devel.gforge.inria.fr/downloads.html

  • Martinez AM, Kak AC (2001) PCA versus lDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  • Martinez AM (1998) Benavente,“ The AR Face Database,”. Technical report, CVC Technical Report

  • Nesterov Y et al (2007) Gradient methods for minimizing composite objective function. Technical report, UCL

  • Obozinski G, Taskar B, Jordan MI (2010) Joint covariate selection and joint subspace selection for multiple classification problems. Stat Comput 20(2):231–252

    Article  MathSciNet  Google Scholar 

  • Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res 37(23):3311–3325

    Article  Google Scholar 

  • Ozawa S, Roy A, Roussinov D (2009) A multitask learning model for online pattern recognition. IEEE Trans Neural Netw 20(3):430–445

    Article  Google Scholar 

  • Ramirez C, Kreinovich V, Argaez M (2013) Why l1 is a good approximation to l0: a geometric explanation. J Uncertain Syst 7(3):203–207

    Google Scholar 

  • Rao RPN, Olshausen BA, Lewicki MS (2002) Probabilistic models of the brain: perception and neural function. MIT Press, Cambridge

    Book  Google Scholar 

  • Singaraju D, Tron R, Elhamifar E, Yang AY, Sastry SS (2012) On the Lagrangian biduality of sparsity minimization problems. In: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 3429–3432

  • Sural S, Qian G, Pramanik S (2002) Segmentation and histogram generation using the HSV color space for image retrieval. In: 2002 International conference on image processing. 2002. Proceedings, IEEE, vol 2, pp II–589

  • Tseng P (2008) On accelerated proximal gradient methods for convex-concave optimization. SIAM J Optim

  • Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on machine learning, ACM, pp 1096–1103

  • Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(Dec):3371–3408

    MathSciNet  MATH  Google Scholar 

  • Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227. https://doi.org/10.1109/TPAMI.2008.79

    Article  Google Scholar 

  • Wright SJ (2015) Coordinate descent algorithms. Math Program 151(1):3–34

    Article  MathSciNet  MATH  Google Scholar 

  • Yan M, Liu H, Xu X, Song E, Qian Y, Pan N, Jin R, Jin L, Cheng S, Hung CC (2017) An improved label fusion approach with sparse patch-based representation for MRI brain image segmentation. Int J Imaging Syst Technol 27(1):23–32

    Article  Google Scholar 

  • Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B 68(1):49–67

    Article  MathSciNet  MATH  Google Scholar 

  • Yuan XT, Liu X, Yan S (2012) Visual classification with multitask joint sparse representation. IEEE Trans Image Process 21(10):4349–4360. https://doi.org/10.1109/TIP.2012.2205006

    Article  MathSciNet  MATH  Google Scholar 

  • Zhao P, Rocha G, Yu B (2009) The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat 37:3468–3497

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu Q, Sun H, Feng Q, Wang J (2014) CCEDA: building bridge between subspace projection learning and sparse representation-based classification. Electron Lett 50(25):1919–1921

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank professor Ali Ghodsi for insightful teachings and his supports.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shiraz University, Shiraz, Iran

    Seyed Mehdi Hazrati Fard & Sattar Hashemi

Authors
  1. Seyed Mehdi Hazrati Fard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sattar Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyed Mehdi Hazrati Fard.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hazrati Fard, S., Hashemi, S. Sparse Representation Using Deep Learning to Classify Multi-Class Complex Data. Iran J Sci Technol Trans Electr Eng 43 (Suppl 1), 637–647 (2019). https://doi.org/10.1007/s40998-018-0154-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40998-018-0154-5

Keywords

Search

Navigation