Abstract
Deep learning is one of the most widely used machine learning techniques which has achieved enormous success in applications such as anomaly detection, image detection, pattern recognition, and natural language processing. Deep learning architectures have revolutionized the analytical landscape for big data amidst wide-scale deployment of sensory networks and improved communication protocols. In this chapter, we will discuss multiple deep learning architectures and explain their underlying mathematical concepts. An up-to-date overview here presented concerns three main categories of neural networks, namely, Convolutional Neural Networks, Pretrained Unspervised Networks, and Recurrent/Recursive Neural Networks. Applications of each of these architectures in selected areas such as pattern recognition and image detection are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Boden, M.: A guide to recurrent neural networks and backpropagation. The Dallas project (2002)
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., Bharath, A.A.: Generative adversarial networks: an overview. IEEE Signal Process. Mag. 35(1), 53–65 (2018)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press. http://www.deeplearningbook.org (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hosseini, M., Pompili, D., Elisevich, K., Soltanian-Zadeh, H.: Optimized deep learning for EEG big data and seizure prediction BCI via internet of things. IEEE Trans. Big Data 3(4), 392–404 (2017)
Hosseini, M.-P.: Developing a cloud based platform as a service to improve public health of epileptic patients in urban places. Reimagining Health in Cities: New Directions in Urban Health Research, Drexel University School of Public Health, Philadelphia, USA (2015)
Hosseini, M.-P.: Proposing a new artificial intelligent system for automatic detection of epileptic seizures. J. Neurol. Disorders 3(4) (2015)
Hosseini, M.-P.: A cloud-based brain computer interface to analyze medical big data for epileptic seizure detection. In: The 3rd Annual New Jersey Big Data Alliance (NJBDA) Symposium (2016)
Hosseini, M.P.: Brain-computer interface for analyzing epileptic big data. Ph.D. thesis, Rutgers University-School of Graduate Studies (2018)
Hosseini, M.-P., Hajisami, A., Pompili, D.: Real-time epileptic seizure detection from EEG signals via random subspace ensemble learning. In: 2016 IEEE International Conference on Autonomic Computing (ICAC), pp. 209–218. IEEE (2016)
Hosseini, M.P., Lau, A., Lu, S., Phoa, A.: Deep learning in medical imaging, a review. IEEE Rev. Biomed. Eng. (2019)
Hosseini, M.-P., Pompili, D., Elisevich, K., Soltanian-Zadeh, H.: Random ensemble learning for EEG classification. Artif. Intell. Med. 84, 146–158 (2018)
Hosseini, M.P., Soltanian-Zadeh, H., Akhlaghpoor, S.: Three cuts method for identification of COPD. Acta Medica Iranica 771–778 (2013)
Hosseini, M.-P., Soltanian-Zadeh, H., Elisevich, K., Pompili, D.: Cloud-based deep learning of big EEG data for epileptic seizure prediction. In: 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 1151–1155. IEEE (2016)
Hosseini, M.-P., Tran, T.X., Pompili, D., Elisevich, K., Soltanian-Zadeh, H.: Deep learning with edge computing for localization of epileptogenicity using multimodal rs-fMRI and EEG big data. In: 2017 IEEE International Conference on Autonomic Computing (ICAC), pp. 83–92. IEEE (2017)
Karnin, E.D.: A simple procedure for pruning back-propagation trained neural networks. IEEE Trans. Neural Netw. 1(2), 239–242 (1990)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Le Cun, Y., Jackel, L.D., Boser, B., Denker, J.S., Graf, H.P., Guyon, I., Henderson, D., Howard, R.E., Hubbard, W.: Handwritten digit recognition: applications of neural network chips and automatic learning. IEEE Commun. Mag. 27(11), 41–46 (1989)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Liang, N.-Y., Huang, G.-B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Netw. 17(6), 1411–1423 (2006)
Liao, D., Lu, H.: Classify autism and control based on deep learning and community structure on resting-state fMRI. In: 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), pp. 289–294. IEEE (2018)
Patterson, J., Gibson, A.: Deep Learning: A Practitioner’s Approach. O’Reilly Media, Inc. (2017)
Puskorius, G., Feldkamp, L.. Truncated backpropagation through time and kalman filter training for neurocontrol. In: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), vol. 4, pp. 2488–2493. IEEE (1994)
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., Lee, H.: Generative adversarial text to image synthesis (2016). arXiv preprint arXiv:1605.05396
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition (CVPR) (2015)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wallach, I., Dzamba, M., Heifets, A.: AtomNet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery (2015). arXiv preprint arXiv:1510.02855
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hosseini, MP., Lu, S., Kamaraj, K., Slowikowski, A., Venkatesh, H.C. (2020). Deep Learning Architectures. In: Pedrycz, W., Chen, SM. (eds) Deep Learning: Concepts and Architectures. Studies in Computational Intelligence, vol 866. Springer, Cham. https://doi.org/10.1007/978-3-030-31756-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-31756-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31755-3
Online ISBN: 978-3-030-31756-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)