Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 5, pp 5443–5459 | Cite as

A deep stacked wavelet auto-encoders to supervised feature extraction to pattern classification

Article

Abstract

The major issue in pattern classification is in the extraction of features in the training phase. The focus of this work is on combining the ability of wavelet networks and the deep learning techniques to propose a new supervised feature extraction method to pattern classification. This new approach allows the classification of all classes of the dataset by the reconstruction of a Deep Stacked wavelet Auto-Encoder. This Network is obtained after a series of wavelet Auto-Encoders followed by a Softmax classifier at the last layer. Finally, a fine-tuning is applied for the improvement of our result using a back propagation algorithm. Our approach is tested with different image datasets which are the COIL-100, the APTI and the ImageNet datasets and is also tested with two other audio corpuses that contain Arabic words and French words. The experimental test demonstrates the efficiency of our network for image and audio classification compared to other methods.

Keywords

Deep learning Wavelet network Pattern classification Feature extraction 

Notes

Acknowledgements

The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUBprogram.

References

  1. 1.
    Amin A, Al-Sadoun H, Fischer S (1996) Hand-printed arabic character recognition system using an artificial network. Pattern Recogn 29(4):663–675. doi: 10.1016/0031-3203(95)00110-7. URL http://www.sciencedirect.com/science/article/pii/0031320395001107 CrossRefGoogle Scholar
  2. 2.
    Ben Amar C, Zaied M, Alim A (2005) Beta wavelets. Synthesis and application tolossy image compression In: Adv Eng Softw 36:459–474. doi: 10.1016/j.advengsoft.2005.01.013 MATHGoogle Scholar
  3. 3.
    Bengio Y (2009), "Learning Deep Architectures for AI", Foundations and Trends® in Machine Learning: 2(1), 1–127. doi: 10.1561/2200000006
  4. 4.
    Chen Z, Wang J, He H, Huang X (2014) A Fast Deep Learning System Using GPU (1), 1552–1555Google Scholar
  5. 5.
    Dammak M, Mejdoub M, Zaied M, C. Ben Amar (2012) Feature vector approximation based on wavelet network, in: Proceedings of the 4th International Conference on Agents and Artificial Intelligence (ICAART), 394–399.Google Scholar
  6. 6.
    Daubechies I (1992) Ten lectures on wavelets, number 61 in cbms-nsf series in applied mathematicsGoogle Scholar
  7. 7.
    Deng L, Yu D (2014) DEEP LEARNING: methods and applications. Found Trends SignalProcess 7(3–4):1–134Google Scholar
  8. 8.
    Ejbali R, Benayed Y, Zaied M, Alimi MA (2009) Wavelet networks for phonemes recognition, International conference on systems and information processingGoogle Scholar
  9. 9.
    Ejbali R, Zaied M, Amar CB (2010) Intelligent approach to train wavelet networks forRecognition system of Arabic words. KDIR, In, pp 518–522Google Scholar
  10. 10.
    Ejbali R, Zaied M, C. Ben Amar (2012) Multi-input multi-output beta wavelet network: modeling of acoustic units for speech recognition Int. J. Adv. Comput. Sci. Appl. IJACSA12, Sci. Inf. Organ. (SAI), 3Google Scholar
  11. 11.
    El Adel A, Zaied M, Amar CB (2011) Learning wavelet networks based on multiresolution analysis: application to images copy detection. International Conference on Communications, Computing and Control Applications, pp 1–6Google Scholar
  12. 12.
    Eladel A, Ejbali R, Zaied M, C. Ben Amar (2014a) A New Semantic Approach for CBIRBased on BetaWavelet Network Modeling Shape Refined by Texture and Color Features pp. 378–385Google Scholar
  13. 13.
    Eladel A, Zaied M, C. Ben Amar (2014b) A new system for image retrieval using beta wavelet network for descriptors extraction and fuzzy decision. 232–236Google Scholar
  14. 14.
    Guedri B, Zaied M, Amar CB (2011) Indexing and images retrieval by content, International Conference on High Performance Computing & Simulation, pp: 369–375Google Scholar
  15. 15.
    Hassairi S, Ejbali R, Zaied M (2015a) A Deep Convolutional Neural Wavelet Network to supervised Arabic letter image classification. In: 15th Int. Conf. Intell. Syst. Des. Appl.Google Scholar
  16. 16.
    Hassairi S, Ejbali R, Zaied M (2015b) Supervised Image Classification Using DeepConvolutional Wavelets Network. In: 2015 I.E. 27th Int. Conf. Tools withArtif. Intell., 265–271. IEEE. doi: 10.1109/ICTAI.2015.49
  17. 17.
    He K, Zhang X, Renand S, Sun J (2015) "Deep residual learning for image recognition." arXiv preprint arXiv:1512.03385Google Scholar
  18. 18.
    Hinton GE, Osindero S, Teh YW (2006) A fastlearning algorithm for deep belief nets. Neural Comput 18(7):1527–1554. doi: 10.1162/neco.2006.18.7.1527 MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Jemai O, Zaied M, C. Ben Amar and Alimi MA (2010) FBWN: an architecture of fast betawavelet networks for image classification. In: Proc Int Jt Conf Neural Networks. doi: 10.1109/IJCNN.2010.5596876
  20. 20.
    Jemai O, Zaied M, Ben Amar C, Alimi MA (2011a) Fast learning algorithm of WaveletNetwork based on fast wavelet transform. Int J Pattern Recognit Artif Intell 25(08):1297–1319. doi: 10.1142/S0218001411009111 MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    O. Jemai, M. Zaied, C. Ben Amar and A. Alimi (2011b): Pyramidal hybrid approach: wavelet network with OLS algorithm-based image classification, Int. J. Wavelets, Multiresolution Inf. Process 9 111–130.Google Scholar
  22. 22.
    Jemai O, Ejbali R, Zaied M, Amar CB (2015) A speech recognition system based on hybrid wavelet network including a fuzzy decision support system, International Conference on Machine Vision (ICMV 2014), 944503–944503-7Google Scholar
  23. 23.
    Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst pp:1–9Google Scholar
  24. 24.
    Le QV (2013) Building high-level features using large scale unsupervised learning. 2013 I.E. Int. Conf. Acoust. Speech Signal Process. pp. 8595–8598. doi: 10.1109/ICASSP.2013.6639343. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6639343
  25. 25.
    LeCun Y (2012) Learning invariant feature hierarchies. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 7583 LNCS, 496–505Google Scholar
  26. 26.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRefGoogle Scholar
  27. 27.
    Y. Lin, F. Lv, S. Zhu, M. Yang, T. Cour, K. Yu, L. Cao and T. Huang (2011) Large-scaleimage classification: Fast feature extraction and svm training. In: Computer Visionand Pattern Recognition (CVPR), 2011 I.E. Conference on, 1689–696. doi: 10.1109/CVPR.2011.5995477
  28. 28.
    Liou CY, Cheng WC, Liou JW, Liou DR (2014) Autoencoder for words. Neurocomputing 139:84–96. doi: 10.1016/j.neucom.2013.09.055 CrossRefGoogle Scholar
  29. 29.
    Mejdoub M, Ben Amar C (2013) Classification improvement of local feature vectors over the KNN algorithm Multimed. Tools Appl 64(1):197–218CrossRefGoogle Scholar
  30. 30.
    Mejdoub M, Fonteles L, C. Ben Amar, Antonini M (2008) Fast indexing method for image retrieval using tree-structured lattices, in: International Workshop on Content-Based Multimedia Indexing (CBMI), 2008, Conference Proceedings, art. no. 4564970, pp. 365–372.Google Scholar
  31. 31.
    Nene S, Nayar S, Murase H (1996) Columbia Object Image Library (COIL-100). Tech.rep. (1996). URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.5914
  32. 32.
    X. Peng, R. Yan, B. Zhao, H. Tang and Z. Yi (2015) Fast low rank representation based spatialpyramid matching for image classification. Knowledge-Based Systems 90, 14–22Google Scholar
  33. 33.
    Picone J (1993) Signal modeling techniques in speech recognition. Proc.IEEE 81(9), 1215–1247. doi: 10.1109/5.237532. URLhttp://ieeexplore.ieee.org/articleDetails.jsp?arnumber=237532
  34. 34.
    Rastegari M, Ordonez V, Redmon J, Farhadi A (2016) XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. arXiv preprint arXiv:1603.05279Google Scholar
  35. 35.
    Said S, B. Ben Amor, Zaied M, C. Ben Amar, Daoudi M (2009) Fast and efficient 3D face recognition using wavelet networks, in: The 16th IEEE International Conference on Image Processing, Cairo, Egypt, November 4153–4156Google Scholar
  36. 36.
    Sermanet P, Chintala S, LeCun Y (2012) Convolutional neural networks applied to housenumbers digit classification. Proc. Int. Conf. Pattern Recognit. ICPR12 pp. 10–13Google Scholar
  37. 37.
    Shin HC, Orton MR, Collins DJ, Doran SJ, Leach MO (2013) Stacked autoencodersfor unsupervised feature learning and multiple organ detection in a pilot study using4D patient data. IEEE Trans. Pattern Anal. Mach. Intell 35(8):1930–1943. doi: 10.1109/TPAMI.2012.277 CrossRefGoogle Scholar
  38. 38.
    Slimane F, Kanoun S, Hennebert J, Alimi AM, Ingold R (2013) A study on font-family and font-size recognition applied to Arabic word images at ultra-low resolution. Pattern Recognit. Lett 34(2):209–218. doi: 10.1016/j.patrec.2012.09.012 CrossRefGoogle Scholar
  39. 39.
    Smagghe P, Buessler JL, Urban JP (2013) Novelty detection in image recognition usingirf neural networks properties. ESANN, InGoogle Scholar
  40. 40.
    Tsai Cw, Huang Wc, Chiang Mc (2014) Mobile, Ubiquitous, and Intelligent Computing. Lect Notes Electr Eng 274:629–636. doi: 10.1007/978-3-642-40675-1 CrossRefGoogle Scholar
  41. 41.
    P. Vincent: Stacked DenoisingAutoencoders: learning useful representations in aDeep network with a local denoising criterion 11, 3371–3408 (2010)Google Scholar
  42. 42.
    Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked DenoisingAutoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408MathSciNetMATHGoogle Scholar
  43. 43.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: Fromerror visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. doi: 10.1109/TIP.2003.819861 CrossRefGoogle Scholar
  44. 44.
    Witkowski B (2015) Autoencoders for Image Classification.Jagiellonian University Faculty of Mathematics and Computer Science INSTITUTE OF COMPUTER SCIENCEApr 28Google Scholar
  45. 45.
    Zaied M, C. Ben Amar, Alimi MA (2003) Award a new wavelet based beta function, International conference on signal, system and design, 185–191Google Scholar
  46. 46.
    M. Zaied, S. Said, O. Jemai and C. Ben Amar (2011) a Novel Approach for Face RecognitionBased on Fast Learning Algorithm and Wavelet Network Theory. Int. J. Wavelets,Multiresolution Inf. Process 09(06), 923–945. doi: 10.1142/S0219691311004389
  47. 47.
    Zaied M, Mohamed R, C. ben Amar (2012) A power tool for content-based image retrieval using multiresolution wavelet network modeling and dynamic histograms. Int. REv.Comput. Softw. (IRECOS), 7Google Scholar
  48. 48.
    Zeiler MD, Fergus R (2013) Visualizing and Understanding Convolutional Networks. arXivPrepr. arXiv1311.2901 (2013). URL http://arxiv.org/abs/1311.2901
  49. 49.
    Zhang Q, Benveniste A (1992) Wavelet networks. IEEE Trans Neural Networks 3(6):889–898. doi: 10.1109/72.165591 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.RTIM: Research Team in Intelligent MachinesGabesTunisia
  2. 2.National Engineering School of Sfax (ENIS)SfaxTunisia

Personalised recommendations