Advertisement

Archives of Computational Methods in Engineering

, Volume 25, Issue 1, pp 121–129 | Cite as

Background Information of Deep Learning for Structural Engineering

  • Seunghye Lee
  • Jingwan Ha
  • Mehriniso Zokhirova
  • Hyeonjoon Moon
  • Jaehong LeeEmail author
S.I.: Machine learning in computational mechanics

Abstract

Since the first journal article on structural engineering applications of neural networks (NN) was published, there have been a large number of articles about structural analysis and design problems using machine learning techniques. However, due to a fundamental limitation of traditional methods, attempts to apply artificial NN concept to structural analysis problems have been reduced significantly over the last decade. Recent advances in deep learning techniques can provide a more suitable solution to those problems. In this study, versatile background information, such as alleviating overfitting methods with hyper-parameters, is presented. A well-known ten bar truss example is presented to show condition for neural networks, and role of hyper-parameters in the structures.

Notes

Funding

This research was funded by a grant (NRF-2017R1A4A1015660) from NRF (National Research Foundation of Korea) funded by MEST (Ministry of Education and Science Technology) of Korean government.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Bengio Y, Goodfellow IJ, Courville A (2015) Deep learning. MIT Press, CambridgezbMATHGoogle Scholar
  2. 2.
    Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3(3):210–229MathSciNetCrossRefGoogle Scholar
  3. 3.
    Carbonell JG, Michalski RS, Mitchell TM (1983) Machine learning: a historical and methodological analysis. AI Mag 4(3):69Google Scholar
  4. 4.
    Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386CrossRefGoogle Scholar
  5. 5.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Nair V, Hinton G E (2010) Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) (pp 807–814)Google Scholar
  7. 7.
    Hinton GE, Srivastava,N, Krizhevsky A, Sutskever I, Salakhutdinov, RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580
  8. 8.
    Deng L (2014) A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans Signal Inform Process 3:e2CrossRefGoogle Scholar
  9. 9.
    Adeli H (2001) Neural networks in civil engineering: 19892000. Comput-Aided Civ Infrastruct Eng 16(2):126–142CrossRefGoogle Scholar
  10. 10.
    Gupta T, Sharma RK (2011) Structural analysis and design of buildings using neural network: a review. Int J Eng Manag Sci 2(4):216–220Google Scholar
  11. 11.
    Haftka RT, Grdal Z (2012) Elements of structural optimization, vol 11. Springer, DordrechtGoogle Scholar
  12. 12.
    Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, CambridgezbMATHGoogle Scholar
  13. 13.
    Riedmiller, M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: IEEE international conference on neural networks, 1993, (pp 586–591)Google Scholar
  14. 14.
    Rumelhart, D. E., McClelland, J. L., and PDP Research Group. (1988). Parallel distributed processing. In: IEEE (Vol. 1, pp. 443–453)Google Scholar
  15. 15.
    Rojas R (1996) The backpropagation algorithm. In: Neural networks. Springer, Berlin, pp 149–182Google Scholar
  16. 16.
    Hinton G (2010) A practical guide to training restricted Boltzmann machines. Momentum 9(1):926Google Scholar
  17. 17.
    Hawkins DM (2004) The problem of overfitting. J Chem Inf and Comput Sci 44(1):1–12MathSciNetCrossRefGoogle Scholar
  18. 18.
    Bastien F, Lamblin,P, Pascanu R, Bergstra J, Goodfellow I, Bergeron A, Bouchard N, Warde-Farley D, Bengio, Y (2012) Theano: new features and speed improvements. arXiv preprint arXiv:1211.5590
  19. 19.
    Chollet F (2015) Keras: Theano-based deep learning library. Code: https://github.com/fchollet. Documentation: http://keras. ioGoogle Scholar
  20. 20.
    Hajela P, Berke L (1991) Neurobiological computational models in structural analysis and design. Computers and Structures 41(4):657–667CrossRefzbMATHGoogle Scholar
  21. 21.
    Lawrence S, Giles CL, Tsoi AC (1996) What size neural network gives optimal generalization? Convergence properties of backpropagation. Technical Report UMIACS-TR-96-22 and CS-TR-3617, Institute for Advanced Computer Studies, University of MarylandGoogle Scholar
  22. 22.
    Dugas C, Bengio Y, Blisle F, Nadeau C, Garcia R (2001) Incorporating second-order functional knowledge for better option pricing. Adv Neural Inf Process Syst 472–478Google Scholar
  23. 23.
    Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159MathSciNetzbMATHGoogle Scholar
  24. 24.
    Zeiler, M. D. (2012). ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701
  25. 25.
    Tieleman, T. and Hinton, G. Lecture 6.5 - RMSProp, COURSERA: Neural Networks for Machine Learning. Technical report, 2012Google Scholar
  26. 26.
    Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
  27. 27.
    Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747

Copyright information

© CIMNE, Barcelona, Spain 2017

Authors and Affiliations

  • Seunghye Lee
    • 1
  • Jingwan Ha
    • 2
  • Mehriniso Zokhirova
    • 1
  • Hyeonjoon Moon
    • 2
  • Jaehong Lee
    • 1
    Email author
  1. 1.Deep Learning Architecture Research CenterSejong UniversitySeoulKorea
  2. 2.Department of Computer Science and EngineeringSejong UniversitySeoulKorea

Personalised recommendations