Skip to main content
SpringerLink
Log in

Several Misconceptions and Misuses of Deep Neural Networks and Deep Learning

  • Conference paper
  • First Online:
Communications, Networking, and Information Systems (CNIS 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1839))

  • 183 Accesses

Abstract

Deep learning is widely accepted as a major drive of this artificial intelligence era. In this article, we first briefly describe the milestones of deep learning. Then, based on a mini-review of the literature, we point out some common misconceptions of deep neural networks and deep learning. We also point out that the deep learning approach is being misused in the present days.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Du, K.-L., Swamy, M.N.S.: Neural Networks in a Softcomputing Framework. Springer, London (2006). https://doi.org/10.1007/1-84628-303-5

    Book  MATH  Google Scholar 

  2. Du, K.-L., Swamy, M.N.S.: Neural Networks and Statistical Learning, 2nd edn. Springer, London (2019). https://doi.org/10.1007/978-1-4471-7452-3

    Book  MATH  Google Scholar 

  3. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., et al.: Handwritten digit recognition with a back-propagation network. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2, pp. 396–404. Morgan Kaufmann, San Mateo (1989)

    Google Scholar 

  4. Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016). https://doi.org/10.1038/nature16961

  6. Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., et al.: Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). https://doi.org/10.1038/s41586-021-03819-2

  7. Editorial: How AlphaFold can realize AI’s full potential in structural biology. Nature 608(8) (2022). https://doi.org/10.1038/d41586-022-02088-x

  8. IBM: What is Deep Learning? https://www.ibm.com/cloud/learn/deep-learning. Accessed 13 Mar 2023

  9. DeepAI: Deep Learning Definition. https://deepai.org/machine-learning-glossary-and-terms/deep-learning. Accessed 13 Mar 2023

  10. Oracle: What is Deep Learning? https://www.oracle.com/artificial-intelligence/machine-learning/what-is-deep-learning/. Accessed 13 Mar 2023

  11. Hardesty, L.: MIT News Office: Explained: Neural networks (2017). https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414. Accessed 13 Mar 2023

  12. Zhang, W.J., Yang, G., Lin, Y., Ji, C., Gupta, M.M.: On definition of deep learning. In: Proceedings of the IEEE 2018 World Automation Congress, Stevenson, WA, USA, pp. 232–236 (2018)

    Google Scholar 

  13. IBM: What is Machine Learning? https://www.ibm.com/cloud/learn/machine-learning. Accessed 13 Mar 2023

  14. Microsoft Azure: What is machine learning? https://azure.microsoft.com/en-us/resources/cloud-computing-dictionary/what-is-machine-learning-platform/. Accessed 13 Mar 2023

  15. Oracle: What is Machine Learning? https://www.oracle.com/artificial-intelligence/machine-learning/what-is-machine-learning/. Accessed 13 Mar 2023

  16. Google: What is Machine Learning? https://cloud.google.com/learn/what-is-machine-learning. Accessed 13 Mar 2023

  17. IBM: What are Neural Networks? https://www.ibm.com/cloud/learn/neural-networks. Accessed 13 Mar 2023

  18. DeepAI: What is a Neural Network? https://deepai.org/machine-learning-glossary-and-terms/neural-network. Accessed 13 Mar 2023

  19. AWS: What is a Neural Network? https://aws.amazon.com/what-is/neural-network/?nc1=h_ls. Accessed 13 Mar 2023

  20. Yu, F., Xiu, X., Li, Y.: A survey on deep transfer learning and beyond. Mathematics 10(3619) (2022). https://doi.org/10.3390/math10193619

  21. Zhuang, F., et al.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2021). https://doi.org/10.1109/JPROC.2020.3004555

    Article  Google Scholar 

  22. Yarotsky, D.: Error bounds for approximations with deep ReLU networks. Neural Netw. 94, 103–114 (2017)

    Article  MATH  Google Scholar 

  23. Telgarsky, M.: Benefits of depth in neural networks. In: Proceedings of the 29th Annual Conference on Learning Theory (PMLR), New York, NY, vol. 49, pp. 1517–1539 (2016)

    Google Scholar 

  24. Eldan, R., Shamir, O.: The power of depth for feedforward neural networks. In: Proceedings of the 29th Annual Conference on Learning Theory (PMLR), New York, NY, vol. 49, pp. 907–940 (2016)

    Google Scholar 

  25. Szymanski, L., McCane, B.: Deep networks are effective encoders of periodicity. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1816–1827 (2014)

    Article  Google Scholar 

  26. Mhaskar, H., Liao, Q., Poggio, T.: Learning functions: when is deep better than shallow. CBMM Memo No. 045 (2016). https://arxiv.org/pdf/1603.00988v4.pdf

  27. Baldi, P., Vershynin, R.: The capacity of feedforward neural networks. Neural Netw. 116, 288–311 (2019)

    Article  MATH  Google Scholar 

  28. Veit, A., Wilber, M., Belongie, S.: Residual networks behave like ensembles of relatively shallow networks. In: Advances in Neural Information Processing Systems, vol. 29, pp. 550–558 (2016)

    Google Scholar 

  29. He, F., Liu, T., Tao, D.: Why ResNet works? residuals generalize. IEEE Trans. Neural Netw. Learn. Syst. 31(12), 5349–5362 (2020). https://doi.org/10.1109/TNNLS.2020.2966319

    Article  MathSciNet  Google Scholar 

  30. Zagoruyko, S., Komodakis, N.: Wide residual networks. In: Proceedings of British Machine Vision Conference, Newcastle, UK, pp. 87.1–87.12 (2016)

    Google Scholar 

  31. Simard, P.Y., Steinkraus, D., Platt, J.: Best practice for convolutional neural networks applied to visual document analysis. In: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), pp. 958–962. IEEE Computer Society, Los Alamitos (2003)

    Google Scholar 

  32. Decoste, D., Schoelkopf, B.: Training invariant support vector machines. Mach. Learn. 46, 161–190 (2002)

    Article  MATH  Google Scholar 

  33. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)

    Article  Google Scholar 

  34. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  35. Ranzato, M.A., Poultney, C., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy-based model. In: Advances in Neural Information Processing Systems, vol. 19, 1137–1144 (2006)

    Google Scholar 

  36. Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep, big, simple neural nets for handwritten digit recognition. Neural Comput. 22(12), 3207–3220 (2010)

    Article  Google Scholar 

  37. Illing, B., Gerstner, W., Brea, J.: Biologically plausible deep learning - but how far can we go with shallow networks? Neural Netw. 118, 90–101 (2019). https://doi.org/10.1016/j.neunet.2019.06.001

  38. Du, K.-L., Swamy, M.N.S.: Search and Optimization by Metaheuristics. Springer, New York (2016). https://doi.org/10.1007/978-3-319-41192-7

    Book  MATH  Google Scholar 

  39. Du, K.-L., Leung, C.-S., Mow, W.H., Swamy, M.N.S.: Perceptron: learning, generalization, model selection, fault tolerance, and role in the deep learning era. Mathematics 10, 4730 (2022). https://doi.org/10.3390/math10244730

    Article  Google Scholar 

  40. Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70, 489–501 (2006)

    Article  Google Scholar 

  41. Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.W.: Feedforward neural networks with random weights. In: Proceedings of 11th IAPR International Conference on Pattern Recognition, The Hague, Netherlands, vol. 2, pp. 1–4 (1992) https://doi.org/10.1109/ICPR.1992.201708

  42. Suganthan, P.N., Katuwal, R.: On the origins of randomization-based feedforward neural networks. Appl. Soft Comput. 105, 107239 (2021)

    Article  Google Scholar 

  43. Jaeger, H.: The “echo state” approach to analyzing and training recurrent neural networks. GMD Technical Report 148. German National Research Center for Information Technology, Sankt Augustin, Germany (2001)

    Google Scholar 

  44. Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Advances in Neural Information Processing Systems, vol. 2, pp. 524–532 (1990)

    Google Scholar 

  45. Birost: “Yann LeCun: Who can explain where the extreme learning machine (ELM) is?” https://blog.birost.com/a?ID=e170d2e1-62f6-43e0-9b64-f6510be36803. Accessed 13 Mar 2023

  46. Mhaskar, H.N., Poggio, T.: An analysis of training and generalization errors in shallow and deep networks. Neural Netw. 121, 229–241 (2020). https://doi.org/10.1016/j.neunet.2019.08.028

    Article  MATH  Google Scholar 

  47. Martin, C.H., Mahoney, M.W.: Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning. J. Mach. Learn. Res. 22, 1–73 (2021)

    MathSciNet  MATH  Google Scholar 

  48. Liu, B., Liu, Z., Zhang, T., Yuan, T.: Non-differentiable saddle points and sub-optimal local minima exist for deep ReLU networks. Neural Netw. 144, 75–89 (2021). https://doi.org/10.1016/j.neunet.2021.08.005

    Article  Google Scholar 

  49. Petzka, H., Sminchisescu, C.: Non-attracting regions of local minima in deep and wide neural networks. J. Mach. Learn. Res. 22, 1–34 (2021)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

The author would like to thank Dr. M.N.S. Swamy for insightful discussions.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada

    K.-L. Du

Authors
  1. K.-L. Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K.-L. Du .

Editor information

Editors and Affiliations

  1. Colorado State University, Fort Collins, CO, USA

    Haonan Chen

  2. Tsinghua University, Beijing, China

    Pingyi Fan

  3. Nanyang Technological University, Singapore, Singapore

    Lipo Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, KL. (2023). Several Misconceptions and Misuses of Deep Neural Networks and Deep Learning. In: Chen, H., Fan, P., Wang, L. (eds) Communications, Networking, and Information Systems. CNIS 2023. Communications in Computer and Information Science, vol 1839. Springer, Singapore. https://doi.org/10.1007/978-981-99-3581-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-3581-9_10

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-3580-2

  • Online ISBN: 978-981-99-3581-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation