A Comparative Study of Neural Network Training Algorithms for the Intelligent Security Monitoring of Industrial Control Systems

  • Jaedeok KimEmail author
  • Guillermo Francia


In this chapter, we present a comparative study on the performance of Neural Network training algorithms towards the goal of developing an intelligent system that can classify, in real-time, the behavior of control systems. An investigation on the performance of five neural network training algorithms: Levenberg–Marquardt, Broyden–Fletcher–Goldfarb–Shanno (BFGS) Quasi–Newton, Resilient Backpropagation, Scaled Conjugate Gradient, and Gradient Descent with Momentum and Adaptive Learning Rate, in classifying 30,000 records of simulated operational data on a typical industrial control system is conducted. The comparisons are made on four neural network system metrics: network error performance, success rate, run time, and number of epochs (iterations). The results are tabulated and analyzed. The chapter concludes with perceptive observations and offers avenues for future research extensions. We envision this small scale study would pave the way to the utilization of intelligent analytics as an avenue towards the realization of an enhanced security posture of our nation’s critical infrastructures. Further, this case study on the application of machine learning technology on information security may offer additional forum for academic inquisition.


Continuous security monitoring Neural network training algorithm Mean square error Cross Entropy error Comparative analysis Continuous input data Discrete input data 



This work is supported in part by a Center for Academic Excellence (CAE) Cyber Security Research Program grant (Grant Award Number H98230-15-1-0270) from the National Security Agency (NSA). Opinions expressed are those of the authors and not necessarily of the granting agency.


  1. 1.
    National Institute of Standards and Technology (NIST). (2011). Special Publication 800-137: Information Security Continuous Monitoring (ISCM) for Federal Systems and Organizations, NIST.Google Scholar
  2. 2.
    U.S. Department of Energy. (2015). Energy sector cybersecurity framework implementation guidance. DOE.Google Scholar
  3. 3.
    National Institute of Standards and Technology (NIST). (2014). Framework for improving critical infrastructure cybersecurity. National Institute of Standards and Technology (NIST).Google Scholar
  4. 4.
    North American Electric Reliability Corporation (NERC). (2015). Critical infrastructure protection (CIP) standards. North American Electric Reliability Corporation (NERC).Google Scholar
  5. 5.
    ICS-CERT. (2014). Incident response/vulnerability coordination in 2014. ICS-CERT Monitor, Industrial Control Systems Cyber Emergency Response Team (ICS-CERT).Google Scholar
  6. 6.
    Francia, G., Bekhouche, N., & Marbut, T. (2011). Design and implementation of a critical infrastructure security and assessment laboratory. In Proceedings of the security and management 2011 conference, Las Vegas, NV.Google Scholar
  7. 7.
    Francia, G. (2011). Critical infrastructure curriculum modules. In Proceedings of the 2011 information security curriculum development (INFOSECCD) conference, Kennesaw, GA.Google Scholar
  8. 8.
    De Boer, P., Kroese, D. K., Mannor, S., & Rubinstein, R. Y. (2005). A tutorial on the cross-entropy method. Annals of Operations Research, 134, 19–67.MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    McCaffrey, J. (2014, November 4). Neural network cross entropy error. Visual Studio Magazine.Google Scholar
  10. 10.
    Magnus, J. R., & Neudecker, H. (1999). Matrix differential calculus. Chichester: John Wiley & Sons, Ltd..zbMATHGoogle Scholar
  11. 11.
    Bishop, C. M. (2007). Patern recognition and machine learning. Heidelberg: Springer.Google Scholar
  12. 12.
    Marquardt, D. (1963). An algorithm for least-squares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics, 11(2), 431–441.MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Dennis, J. E., & Schnabel, R. B. (1983). Numerical methods for unconstrained optimization and nonlinear equations. Englewoods Cliffs, NJ: Prentice-Hall.zbMATHGoogle Scholar
  14. 14.
    Riedmiller, M., & Braun, H.. (1993). A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In Proceedings of the IEEE international conference on neural networks.Google Scholar
  15. 15.
    Moller, M. (1993). A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6, 525–533.CrossRefGoogle Scholar
  16. 16.
    Hagan, M. T., Demuth, H. B., & Beale, M. H. (1996). Neural network design. Boston, MA: PWS Publishing.Google Scholar
  17. 17.
    Scales, L. (1985). Introduction to non-linear optimization. New York: Springer-Verlag.CrossRefGoogle Scholar
  18. 18.
    Golik, P., Doetsch, P., & Ney, H. (2013). Cross-entropy vs. squared error training: a theoretical and experimental comparison. In Proceeding of the 14th annual conference of the international speech communication association, Lyon.Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Jacksonville State UniversityJacksonvilleUSA

Personalised recommendations