Advertisement

A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning

  • Shaveta Dargan
  • Munish KumarEmail author
  • Maruthi Rohit Ayyagari
  • Gulshan Kumar
Original Paper

Abstract

Nowadays, deep learning is a current and a stimulating field of machine learning. Deep learning is the most effective, supervised, time and cost efficient machine learning approach. Deep learning is not a restricted learning approach, but it abides various procedures and topographies which can be applied to an immense speculum of complicated problems. The technique learns the illustrative and differential features in a very stratified way. Deep learning methods have made a significant breakthrough with appreciable performance in a wide variety of applications with useful security tools. It is considered to be the best choice for discovering complex architecture in high-dimensional data by employing back propagation algorithm. As deep learning has made significant advancements and tremendous performance in numerous applications, the widely used domains of deep learning are business, science and government which further includes adaptive testing, biological image classification, computer vision, cancer detection, natural language processing, object detection, face recognition, handwriting recognition, speech recognition, stock market analysis, smart city and many more. This paper focuses on the concepts of deep learning, its basic and advanced architectures, techniques, motivational aspects, characteristics and the limitations. The paper also presents the major differences between the deep learning, classical machine learning and conventional learning approaches and the major challenges ahead. The main intention of this paper is to explore and present chronologically, a comprehensive survey of the major applications of deep learning covering variety of areas, study of the techniques and architectures used and further the contribution of that respective application in the real world. Finally, the paper ends with the conclusion and future aspects.

Notes

Compliance with Ethical Standards

Conflict of interest

Authors have no conflict of interest.

References

  1. 1.
    Abadi M, Paul B, Jianmin C, Zhifeng C, Andy D, Jeffrey D, Matthieu D (2016) Tensorflow: a system for large-scale machine learning. In: The proceedings of the 12th USENIX symposium on operating systems design and implementation (OSDI’16), vol 16, pp 265–283Google Scholar
  2. 2.
    Abbas Q, Ibrahim MEA, Jaffar MA (2018) A comprehensive review of recent advances on deep vision systems. Artif Intell Rev.  https://doi.org/10.1007/s10462-018-9633-3 CrossRefGoogle Scholar
  3. 3.
    Affonso C, Rossi ALD, Vieria FHA, Carvalho ACPDLFD (2017) Deep learning for biological image classification. Expert Syst Appl 85:114–122CrossRefGoogle Scholar
  4. 4.
    Alwzwazy HA, Albehadili HA, Alwan YS, Islam NE (2016) Handwritten digit recognition using convolutional neural networks. In: Proceedings of international journal of innovative research in computer and communication engineering, vol 4(2), pp 1101–1106Google Scholar
  5. 5.
    Amato G, Carrara F, Falchi F, Gennaro C, Meghini C, Vairo C (2017) Deep learning for decentralized parking lot occupancy detection. Expert Syst Appl 72:327–334CrossRefGoogle Scholar
  6. 6.
    Araque O, Corcuera-Platas I, Sánchez-Rada JF, Iglesias CA (2017) Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Syst Appl 77:236–246CrossRefGoogle Scholar
  7. 7.
    Ashiquzzaman A, Tushar AK (2017) Handwritten arabic numeral recognition using deep learning neural networks. In: Proceedings of IEEE international conference on imaging, vision & pattern recognition, pp 1–4.  https://doi.org/10.1109/ICIVPR.2017.7890866
  8. 8.
    Azar MY, Hamey L (2017) Text summarization using unsupervised deep learning. Expert Syst Appl 68:93–105CrossRefGoogle Scholar
  9. 9.
    Chen XW, Lin X (2014) Big data deep learning: challenges and perspectives. IEEE 2:514–525.  https://doi.org/10.1109/ACCESS.2014.2325029 CrossRefGoogle Scholar
  10. 10.
    Chen CH, Lee CR, Lu WCH (2016) A mobile cloud framework for deep learning and its application to smart car camera. In: Proceedings of the international conference on internet of vehicles, pp 14–25.  https://doi.org/10.1007/978-3-319-51969-22
  11. 11.
    Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848CrossRefGoogle Scholar
  12. 12.
    Cheng D, Gong Y, Changb X, Shia W, Hauptmannb A, Zhenga N (2018) Deep feature learning via structured graph Laplacian embedding for person re-identification. Pattern Recogn 82:94–104CrossRefGoogle Scholar
  13. 13.
    Chong E, Han C, Park FC (2017) Deep learning network for stock market analysis and prediction: methodology, data representations and case studies. Expert Syst Appl 83:187–205CrossRefGoogle Scholar
  14. 14.
    Chu J, Srihari S (2014) Writer identification using a deep neural network. In: Proceedings of the 2014 Indian conference on computer vision graphics and image processing, pp 1–7Google Scholar
  15. 15.
    Dai Y, Wang G (2018) A deep inference learning framework for healthcare. Pattern Recogn Lett.  https://doi.org/10.1016/j.patrec.2018.02.009 CrossRefGoogle Scholar
  16. 16.
    Dhieb T, Ouarda W, Boubaker H, Alilmi AM (2016) Deep neural network for online writer identification using Beta-elliptic model. In: Proceedings of the international joint conference on neural networks, pp 1863–1870Google Scholar
  17. 17.
    Falcini F, Lami G, Costanza AM (2017) Deep learning in automotive software. IEEE Softw 34(3):56–63.  https://doi.org/10.1109/MS.2017.79 CrossRefGoogle Scholar
  18. 18.
    Gheisari M, Wang G, Bhuiyan MZA (2017) A survey on deep learning in big data. In: Proceedings of the IEEE international conference on embedded and ubiquitous computing (EUC), pp 1–8Google Scholar
  19. 19.
    Ghosh MMA, Maghari AY (2017) A comparative study on handwriting digit recognition using neural networks. In: Proceedings of the promising electronic technologies (ICPET), pp 77–81Google Scholar
  20. 20.
    Gurjar N, Sudholt S, Fink GA (2018) Learning deep representations for word spotting under weak supervision. In: Proceedings of the 13th IAPR international workshop on document analysis systems (DAS), pp 7s–12sGoogle Scholar
  21. 21.
    Hamid OA, Jiang H (2013) Rapid and effective speaker adaptation of convolutional neural network based models for speech recognition. In: INTERSPEECH, pp 1248–1252Google Scholar
  22. 22.
    Ignatov A (2018) Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl Soft Comput 62:915–922CrossRefGoogle Scholar
  23. 23.
    Jia X (2017) image recognition method based on deep learning. In: Proceedings of the 29th IEEE, Chinese control and decision conference (CCDC), pp 4730–4735Google Scholar
  24. 24.
    Kannan RJ, Subramanian S (2015) An adaptive approach of tamil character recognition using deep learning with big data-a survey. Adv Intell Syst Comput: 557–567Google Scholar
  25. 25.
    Kaushal M, Khehra B, Sharma A (2018) Soft computing based object detection and tracking approaches: state-of-the-art survey. Appl Soft Comput 70:423–464CrossRefGoogle Scholar
  26. 26.
    Krishnan P, Dutta K, Jawahar CV (2018) Word spotting and recognition using deep embedding. In: Proceedings of 13th IAPR international workshop on document analysis systems (DAS).  https://doi.org/10.1109/das.2018.70
  27. 27.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:1–10MathSciNetCrossRefGoogle Scholar
  28. 28.
    Lee SH, Chan CS, Mayo SJ, Remagnino P (2017) How deep learning extracts and learns leaf features for plant classification. Pattern Recogn 71:1–13CrossRefGoogle Scholar
  29. 29.
    Ling ZH, Kang SY, Zen H, Senior A, Schuster M, Qian XJ, Meng HM, Deng L (2015) Deep learning for acoustic modeling in parametric speech generation: a systematic review of existing techniques and future trends. IEEE Signal Process Mag 32(3):35–52CrossRefGoogle Scholar
  30. 30.
    Ling Y, Jin C, Guoru D, Ya T, Jian Y, Jiachen S (2018) Spectrum prediction based on Taguchi method in deep learning with long short-term memory. IEEE Access 6(1):45923–45933Google Scholar
  31. 31.
    Liu PH, Su SF, Chen MC, Hsiao CC (2015) Deep learning and its application to general image classification. In: Proceedings of the international conference on informatics and cybernetics for computational social systems, pp 1–4Google Scholar
  32. 32.
    Looks M, Herreshoff M, Hutchins D, Norvig P (2017) Deep learning with dynamic computation graphs. In: Proceedings of the international conference on learning representation, pp 1–12Google Scholar
  33. 33.
    Lopez D, Rivas E, Gualdron O (2017) Primary user characterization for cognitive radio wireless networks using a neural system based on deep learning. Artif Intell Rev: 1–27Google Scholar
  34. 34.
    Luckow A, Cook M, Ashcraft N, Weill E, Djerekarov E, Vorster B (2017) Deep learning in the automotive industry: applications and tools. In: Proceedings of the IEEE international conference on big data, pp 3759–3768Google Scholar
  35. 35.
    Makhmudov AZ, Abdukarimov SS (2016) Speech recognition using deep learning algorithms. In: Proceedings of the international conference on informatics: problems, methodology, technologies, pp 10–15Google Scholar
  36. 36.
    Markovnikov N, Kipyatkova I, Karpov A, Filchenkov A (2018) Deep neural networks in russian speech recognition. Artif Intell Nat Lang Commun Comput Inf Sci 789:54–67.  https://doi.org/10.1007/978-3-319-71746-3_5 CrossRefGoogle Scholar
  37. 37.
    Mohamed A, Dahl G, Geoffrey H (2009) Deep belief networks for phone recognition. In: Proceedings of the nips workshop on deep learning for speech recognition and related applications, pp 1–9Google Scholar
  38. 38.
    Mohsen AM, El-Makky NM, Ghanem N (2017) Author identification using deep learning. In: Proceedings of the 15th IEEE international conference on machine learning and applications, pp 898–903Google Scholar
  39. 39.
    Nguyen HD, Le AD, Nakagawa M (2015) Deep neural networks for recognizing online handwritten mathematical symbols. In: Proceedings of the 3rd IAPR IEEE Asian conference on pattern recognition (ACPR), pp 121–125Google Scholar
  40. 40.
    Noda K, Yamaguchi Y, Nakadai K, Okuno HG, Ogata T (2015) Audio-visual speech recognition using deep learning. Appl Intell 42(4):722–737CrossRefGoogle Scholar
  41. 41.
    Nweke HF, Teh YW, Al-garadi MA, Alo UR (2018) Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst Appl: 1–87Google Scholar
  42. 42.
    Poznanski A, Wolf L (2016) CNN-N-gram for handwriting word recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2305–2314Google Scholar
  43. 43.
    Prabhanjan S, Dinesh R (2017) deep learning approach for devanagari script recognition. Proc Int J Image Graph 17(3):1750016.  https://doi.org/10.1142/S0219467817500164 CrossRefGoogle Scholar
  44. 44.
    Puthussery AR, Haradi KP, Erol BA, Benavidez P, Rad P, Jamshidi M (2017) A deep vision landmark framework for robot navigation. In: Proceedings of the system of systems engineering conference, pp 1–6Google Scholar
  45. 45.
    Razzak MI, Naz S, Zaib A (2018) Deep learning for medical image processing: overview, challenges and the future. Classif BioApps Lect Notes Comput Vis Biomech 26:323–350Google Scholar
  46. 46.
    Ripoll VJR, Wojdel A, Romero A, Ramos P, Brugada J (2016) ECG assessment based on neural networks with pertaining. Appl Soft Comput 49:399–406CrossRefGoogle Scholar
  47. 47.
    Rudin F, Li GJ, Wang K (2017) An algorithm for power system fault analysis based on convolutional deep learning neural networks. Int J Res Educ Sci Methods 5(9):11–18Google Scholar
  48. 48.
    Salazar F, Toledo MA, González JM, Oñate E (2012) Early detection of anomalies in dam performance: a methodology based on boosted regression trees. Struct Control Health Monit 24(11):2012–2017CrossRefGoogle Scholar
  49. 49.
    Salazar F, Toledo MA, Morán R, Oñate E (2015) An empirical comparison of machine learning techniques for dam behaviour modelling structural safety. Struct Saf 56:9–17CrossRefGoogle Scholar
  50. 50.
    Salazar F, Toledo MA, Oñate E, Suárez B (2016) Interpretation of dam deformation and leakage with boosted regression trees. Eng Struct 119:230–251CrossRefGoogle Scholar
  51. 51.
    Salazar F, Oñate E, Toledo MA (2017a) A machine learning based methodology for anomaly detection in dam behaviour, CIMNE, monograph no M170, 250 pp, BarcelonaGoogle Scholar
  52. 52.
    Salazar F, Moran R, Toledo MA, Oñate E (2017) Data-based models for the prediction of dam behaviour: a review and some methodological considerations. Arch Comput Methods Eng 24(1):1–21zbMATHCrossRefGoogle Scholar
  53. 53.
    Sanakoyeu A, Bautista MA, Ommer B (2018) Deep unsupervised learning of visual similarities. Pattern Recogn 78:331–343CrossRefGoogle Scholar
  54. 54.
    Santana LMQD, Santos RM, Matos LN, Macedo HT (2018) Deep neural networks for acoustic modeling in the presence of noise. IEEE Latin Am Trans 16(3):918–925CrossRefGoogle Scholar
  55. 55.
    Serizel RGD (2016) Deep-neural network approaches for speech recognition with heterogeneous groups of speakers including children. Nat Lang Eng 1(3):1–26Google Scholar
  56. 56.
    Soniya, Paul S, Singh L (2015) A review on advances in deep learning. In: Proceedings of IEEE workshop on computational intelligence: theories, applications and future directions (WCI), pp 1–6.  https://doi.org/10.1109/wci.2015.7495514
  57. 57.
    Sudholt S, Fink GA (2017) Attribute CNNs for word spotting in handwritten documents. Int J Doc Anal Recognit (IJDAR).  https://doi.org/10.1007/s10032-018-0295-0 CrossRefGoogle Scholar
  58. 58.
    Thomas S, Chatelain C, Heutte L, Paquet T, Kessentini Y (2015) A deep HMM model for multiple keywords spotting in handwritten documents. Pattern Anal Appl 18(4):1003–1015MathSciNetCrossRefGoogle Scholar
  59. 59.
    Ucar A, Demir Y, Guzelis C (2017) Object recognition and detection with deep learning for autonomous driving applications. Int Trans Soc Model Simul 93(9):759–769CrossRefGoogle Scholar
  60. 60.
    Vasconcelos CN, Vasconcwlos BN (2017) Experiment using deep learning for dermoscopy image analysis. Pattern Recognit Lett.  https://doi.org/10.1016/j.patrec.2017.11.005 CrossRefGoogle Scholar
  61. 61.
    Wang Y, Liu M, Bao Z (2016) Deep learning neural network for power system fault diagnosis. In: Proceedings of the 35th Chinese control conference, 1–6Google Scholar
  62. 62.
    Wang T, Wen CK, Wang H, Gao F, Jiang F, Jin S (2017) Deep learning for wireless physical layer: opportunities and challenges. China Commun 14(11):92–111CrossRefGoogle Scholar
  63. 63.
    Wicht B, Fischer A, Hennebert J (2016) Deep learning features for handwritten keyword spotting. In: Proceedings of the 23rd international conference on pattern recognition (ICPR).  https://doi.org/10.1109/icpr.2016.7900165
  64. 64.
    Wu Z, Swietojanski P, Veaux C, Renals S, King S (2015) A study of speaker adaptation for DNN-based speech synthesis. In: Proceedings of the sixteenth annual conference of the international speech communication association, pp 879–883Google Scholar
  65. 65.
    Xiao B, Xiong J, Shi Y (2016) Novel applications of deep learning hidden features for adaptive testing. In: Proceedings of the 21st Asia and South Pacific design automation conference, pp 743–748Google Scholar
  66. 66.
    Xue S, Hamid OA, Jiang H, Dai L, Liu Q (2014) Fast adaptation of deep neural network based on discriminant codes for speech recognition. IEEE/ACM Trans Audio Speech Lang Process 22(12):1713–1725CrossRefGoogle Scholar
  67. 67.
    Yadav U, Verma S, Xaxa DK, Mahobiya C (2017) A deep learning based character recognition system from multimedia document. In: Proceedings of the international conference on innovations in power and advanced computing technologies, pp 1–7Google Scholar
  68. 68.
    Yonel B, Mason E, Yazici B (2017) Deep learning for passive synthetic aperture radar. IEEE J Sel Top Signal Process 12(1):90–103CrossRefGoogle Scholar
  69. 69.
    Yu X, Wu X, Luo C, Ren P (2017) Deep learning in remote sensing scene classification: a data augmentation enhanced convolutional neural network framework. GIScience & Remote Sens: 1–19Google Scholar
  70. 70.
    Yuan X, Xie L, Abouelenien M (2018) A regularized ensemble framework of deep learning for cancer detection from multi-class, imbalanced training data. Pattern Recogn 77:160–172CrossRefGoogle Scholar
  71. 71.
    Zhang L, Zhang L, Du B (2016) Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geosci Remote Sens Mag 4(2):22–40CrossRefGoogle Scholar
  72. 72.
    Zhao C, Chen K, Wei Z, Chen Y, Miao D, Wang W (2018) Multilevel triplet deep learning model for person reidentification. Pattern Recogn Lett.  https://doi.org/10.1016/j.patrec.2018.04.029 CrossRefGoogle Scholar
  73. 73.
    Zhong SH, Li Y, Le B (2015) Query oriented unsupervised multi document summarization via deep learning. Expert Syst Appl, pp 1–10Google Scholar
  74. 74.
    Zhou X, Gong W, Fu W, Du F (2017) Application of deep learning in object detection. In: Proceedings of the IEEE/ACIS 16th international conference on computer and information science (ICIS), pp 631–634Google Scholar
  75. 75.
    Zhu XX, Tuia D, Mou L, Xia GS, Zhang L, Xu F, Fraundorfer F (2017) Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geosci Remote Sens Mag 5(4):8–36CrossRefGoogle Scholar
  76. 76.
    Zulkarneev M, Grigoryan R, Shamraev N (2013) Acoustic modeling with deep belief networks for Russian speech recognition. In: Proceedings of the international conference on speech and computer, pp 17–24Google Scholar
  77. 77.
    Chandra B, Sharma RK (2016) Deep learning withadaptive learning rate using Laplacian score, expert systems with applications. Int J 63(C):1–7Google Scholar
  78. 78.
    Wu Z, Swietozanski P, Veaux C, Renals S (2015) A study of speaker adaptation for DNN-based speech synthesis. In: Proceedings of the interspeech conference, pp 1–5Google Scholar
  79. 79.
    Xing L, Qiao Y (2016) DeepWriter: a multi-stream deep CNN for text-independent writer identification. Comput Vis Pattern Recognit. arXiv:1606.06472
  80. 80.
    Roy P, Bhunia AK, Das A, Dey P (2016) HMM-based indic handwritten word recognition using zone segmentation. Pattern Recognit 60:1057–1075.  https://doi.org/10.1016/j.patcog.2016.04.012 CrossRefGoogle Scholar
  81. 81.
    Loh BCS, Then PHH (2017) Deep learning for cardiac computer-aided diagnosis: benefits, issues & solutions. M Health.  https://doi.org/10.21037/mhealth.2017.09.01 CrossRefGoogle Scholar
  82. 82.
    Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3642–3649Google Scholar
  83. 83.
    Ota K, Dao MS, Mezaris V, Natale FGBD (2017) Deep learning for mobile multimedia: a survey. ACM Trans Multimed Comput Commun Appl (TOMM) (TOMM) 13(3s):34Google Scholar
  84. 84.
    Wang L, Sng D (2015) Deep learning algorithms with applications to video analytics for a smart city: a survey. arXiv, preprint arXiv: 1512.03131
  85. 85.
    Papakostas M, Giannakopoulos T (2018) Speech-music discrimination using deep visual feature extractors. Expert Syst Appl 114:334–344CrossRefGoogle Scholar
  86. 86.
    Arevalo A, Niño J, Hernández G, Sandoval J (2016) High-frequency trading strategy based on deep neural networks. In: Proceedings of the international conference on intelligent computing, pp 424–436Google Scholar
  87. 87.
    Zhang XL, Wu J (2013) Denoising deep neural networks based voice activity detection. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing, pp 853–857Google Scholar

Copyright information

© CIMNE, Barcelona, Spain 2019

Authors and Affiliations

  1. 1.Department of Computational SciencesMaharaja Ranjit Singh Punjab Technical UniversityBathindaIndia
  2. 2.College of BusinessUniversity of DallasIrvingUSA
  3. 3.Department of Computer ApplicationsShaheed Bhagat Singh State Technical CampusFerozepurIndia

Personalised recommendations