A Comparative Study of Neural Network Training Algorithms for the Intelligent Security Monitoring of Industrial Control Systems
In this chapter, we present a comparative study on the performance of Neural Network training algorithms towards the goal of developing an intelligent system that can classify, in real-time, the behavior of control systems. An investigation on the performance of five neural network training algorithms: Levenberg–Marquardt, Broyden–Fletcher–Goldfarb–Shanno (BFGS) Quasi–Newton, Resilient Backpropagation, Scaled Conjugate Gradient, and Gradient Descent with Momentum and Adaptive Learning Rate, in classifying 30,000 records of simulated operational data on a typical industrial control system is conducted. The comparisons are made on four neural network system metrics: network error performance, success rate, run time, and number of epochs (iterations). The results are tabulated and analyzed. The chapter concludes with perceptive observations and offers avenues for future research extensions. We envision this small scale study would pave the way to the utilization of intelligent analytics as an avenue towards the realization of an enhanced security posture of our nation’s critical infrastructures. Further, this case study on the application of machine learning technology on information security may offer additional forum for academic inquisition.
KeywordsContinuous security monitoring Neural network training algorithm Mean square error Cross Entropy error Comparative analysis Continuous input data Discrete input data
This work is supported in part by a Center for Academic Excellence (CAE) Cyber Security Research Program grant (Grant Award Number H98230-15-1-0270) from the National Security Agency (NSA). Opinions expressed are those of the authors and not necessarily of the granting agency.
- 1.National Institute of Standards and Technology (NIST). (2011). Special Publication 800-137: Information Security Continuous Monitoring (ISCM) for Federal Systems and Organizations, NIST.Google Scholar
- 2.U.S. Department of Energy. (2015). Energy sector cybersecurity framework implementation guidance. DOE.Google Scholar
- 3.National Institute of Standards and Technology (NIST). (2014). Framework for improving critical infrastructure cybersecurity. National Institute of Standards and Technology (NIST).Google Scholar
- 4.North American Electric Reliability Corporation (NERC). (2015). Critical infrastructure protection (CIP) standards. North American Electric Reliability Corporation (NERC).Google Scholar
- 5.ICS-CERT. (2014). Incident response/vulnerability coordination in 2014. ICS-CERT Monitor, Industrial Control Systems Cyber Emergency Response Team (ICS-CERT).Google Scholar
- 6.Francia, G., Bekhouche, N., & Marbut, T. (2011). Design and implementation of a critical infrastructure security and assessment laboratory. In Proceedings of the security and management 2011 conference, Las Vegas, NV.Google Scholar
- 7.Francia, G. (2011). Critical infrastructure curriculum modules. In Proceedings of the 2011 information security curriculum development (INFOSECCD) conference, Kennesaw, GA.Google Scholar
- 9.McCaffrey, J. (2014, November 4). Neural network cross entropy error. Visual Studio Magazine.Google Scholar
- 11.Bishop, C. M. (2007). Patern recognition and machine learning. Heidelberg: Springer.Google Scholar
- 14.Riedmiller, M., & Braun, H.. (1993). A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In Proceedings of the IEEE international conference on neural networks.Google Scholar
- 16.Hagan, M. T., Demuth, H. B., & Beale, M. H. (1996). Neural network design. Boston, MA: PWS Publishing.Google Scholar
- 18.Golik, P., Doetsch, P., & Ney, H. (2013). Cross-entropy vs. squared error training: a theoretical and experimental comparison. In Proceeding of the 14th annual conference of the international speech communication association, Lyon.Google Scholar