Deep Learning Theory Simplified

  • Adilya Bakambekova
  • Alex Pappachen JamesEmail author
Part of the Modeling and Optimization in Science and Technologies book series (MOST, volume 14)


Deep Learning is a promising field of Artificial Intelligence algorithms that have proven to be capable of solving a wide range of tasks including classification, object detection, regression, face recognition, augmented and virtual reality, self-driving cars and many more. This chapter introduces the reader to Deep Learning, its basic principles, and applications. It covers the essential elements of any Deep Learning system, as well as explains how to connect these elements to form a neural network. The reader will understand the reasoning behind the Deep Learning and why it is so useful nowadays. The training algorithm of the neural network is also covered in this chapter.


  1. 1.
  2. 2.
    Agrawal S, Agrawal J (2015) Neural network techniques for cancer prediction: a survey. Proc Comput Sci 60:769–774. Scholar
  3. 3.
    Blanchini F, Franco E (2011) Structurally robust biological networks. BMC Syst Biol 5(1):74. Scholar
  4. 4.
    Chien JT, Ku YC (2016) Bayesian recurrent neural network for language modeling. IEEE Trans Neural Netw Learn Syst 27(2):361–374MathSciNetCrossRefGoogle Scholar
  5. 5.
    Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078
  6. 6.
    Chowdhury I, Nguyen K, Fookes C, Sridharan S (2017) A cascaded long short-term memory (lstm) driven generic visual question answering (vqa). In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 842–1846Google Scholar
  7. 7.
    Deng Z, Lei L, Sun H, Zou H, Zhou S, Zhao J (2017) An enhanced deep convolutional neural network for densely packed objects detection in remote sensing images. In: 2017 international workshop on remote sensing with intelligent processing (RSIP). IEEE, pp 1–4Google Scholar
  8. 8.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D,Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680Google Scholar
  9. 9.
    Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J (2017) Lstm: a search space odyssey. IEEE Trans Neural Netw Learn Syst 28(10):2222–2232MathSciNetCrossRefGoogle Scholar
  10. 10.
    Herculano-Houzel S (2009) The human brain in numbers: a linearly scaled-up primate brain. Front Human Neurosci 3 (2009).
  11. 11.
    Kim HG, Han SH, Choi HJ (2017) Discriminative restricted boltzmann machine for emergency detection on healthcare robot. In: 2017 IEEE international conference on big data and smart computing (BigComp). IEEE, pp 407–409Google Scholar
  12. 12.
    Le Callet P, Viard-Gaudin C, Barba D (2006) A convolutional neural network approach for objective video quality assessment. IEEE Trans Neural Netw 17(5):1316–1327CrossRefGoogle Scholar
  13. 13.
    Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent neural networks for sequence learning. arXiv:1506.00019
  14. 14.
    Ma W, Pan Z, Guo J, Lei B (2018) Super-resolution of remote sensing images based on transferred generative adversarial network. In: IGARSS 2018-2018 IEEE international geoscience and remote sensing symposium. IEEE, pp 1148–1151Google Scholar
  15. 15.
    Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78(10):1629–1636. Scholar
  16. 16.
    Mead WR, Kurzweil R (2006) The singularity is near: when humans transcend biology. Foreign Affairs 85(3):160. Scholar
  17. 17.
    Rezaee M, Mahdianpari M, Zhang Y, Salehi B (2018) Deep convolutional neural network for complex wetland classification using optical remote sensing imagery. IEEE J Sel Top Appl Earth Obs Remote Sens 11(9):3030–3039CrossRefGoogle Scholar
  18. 18.
    Salakhutdinov R, Mnih A, Hinton G (2007) Restricted boltzmann machines for collaborative filtering. In: Proceedings of the 24th international conference on machine learning. ACM, pp 791–798Google Scholar
  19. 19.
    Samarawickrama A, Fernando, T (2017) A recurrent neural network approach in predicting daily stock prices an application to the sri lankan stock market. In: 2017 IEEE international conference on industrial and information systems (ICIIS). IEEE, pp 1–6 (2017)Google Scholar
  20. 20.
    Sangwan VK, Lee HS, Bergeron H, Balla I, Beck ME, Chen KS, Hersam MC (2018) Multi-terminal memtransistors from polycrystalline monolayer molybdenum disulfide. Nature 554(7693):500–504. Scholar
  21. 21.
    Skovajsová L (2017) Long short-term memory description and its application in text processing. In: Communication and information technologies (KIT). IEEE, pp 1–4Google Scholar
  22. 22.
    Tan WR, Chan CS, Aguirre H, Tanaka K (2017) Improved artgan for conditional synthesis of natural image and artwork. arXiv:1708.09533
  23. 23.
    Tang Y, Huang Y, Wu Z, Meng H, Xu M, Cai L (2016) Question detection from acoustic features using recurrent neural network with gated recurrent unit. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6125–6129 (2016)Google Scholar
  24. 24.
    Weng CY, Curless B, Kemelmacher-Shlizerman I (2018) Photo wake-up: 3d character animation from a single photo. arXiv:1812.02246
  25. 25.
    Wu J (2017) Introduction to convolutional neural networksGoogle Scholar
  26. 26.
    Xiaoyun Q, Xiaoning K, Chao Z, Shuai J, Xiuda M (2016) Short-term prediction of wind power based on deep long short-term memory. In: 2016 IEEE PES Asia-Pacific power and energy engineering conference (APPEEC). IEEE, pp 1148–1152Google Scholar
  27. 27.
    Yedder HB, Zakia U, Ahmed A, Trajković L (2017) Modeling prediction in recommender systems using restricted boltzmann machine. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 2063–2068Google Scholar
  28. 28.
    Zeki S (2015) A massively asynchronous, parallel brain. Philos Trans Roy Soc B: Biol Sci 370(1668):20140174–20140174. Scholar
  29. 29.
    Zhang, X., Zhu, X., Zhang, N., Li, P., Wang, L., et al.: Seggan: Semantic segmentation with generative adversarial network. In: 2018 IEEE fourth international conference on multimedia big data (BigMM). IEEE, pp 1–5 (2018)Google Scholar
  30. 30.
    Zhao Y, Li J, Xu S, Xu B (2016) Investigating gated recurrent neural networks for acoustic modeling. In: 2016 10th international symposium on Chinese spoken language processing (ISCSLP). IEEE, pp 1–5Google Scholar
  31. 31.
    Zhu F, Fan Z, Wu X (2014) Voice conversion using conditional restricted boltzmann machine. In: 2014 IEEE China summit & international conference on signal and information processing (ChinaSIP). IEEE, pp 110–114Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Nazarbayev UniversityAstanaKazakhstan

Personalised recommendations