Abstract
Biologically inspired soft computing paradigms such as neural networks are popular learning models adopted in online adaptive systems for their ability to cope with the demands of a changing environment. However, continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure the reliable performance of such systems. In this paper, we discuss a dynamic approach to validate the adaptive system component. Our approach consists of two run-time techniques: (1) a statistical learning tool that detects unforeseen data; and (2) a reliability measure of the neural network output after it accommodates the environmental changes. A case study on NASA F-15 flight control system demonstrates that our techniques effectively detect unusual events and provide validation inferences in a real-time manner.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Boyd, M. A., Schumann, J., Brat, G., Giannakopoulou, D., Cukic, B.,& Mili, A. (2001). Validation and verification process guide for software and neural nets, Technical report, NASA Ames Research Center, September.
Bruske, J. & Sommer, G. (1995). Dynamic cell structure learns perfectly topology preserving map. Neural Computation, 7(4) 845–865.
Darrah, M., Taylor, B., & Skias, S. (2004). Rule extraction from dynamic CEll structure neural networks used in a safety critical application. In Proc. of the Seventeenth International Conference of the Florida Artificial Intelligence Research Society, Miami Beach, FL, USA, September 2004.
Del Gobbo, D., & Cukic, B. (2001). Validating on-line neural networks. Technical report, Lane department of computer science and electrical engineering, West Virginia University, December 2001.
Institute of Software Reseach. (2001). Dynamic cell structure neural network report for the intelligent flight control system, Technical report, Document ID: IFC-DCSR-D002-UNCLASS-010401, January 2001.
Jorgensen, C. C. (1991). Feedback linearized aircraft control using dynamic cell structures. InWorld Automation Congress (ISSCI) (pp. 050.1-050.6). Alaska.
Kohonen T. (1990). The self-organizing map. Proceedings. of the IEEE, 78(9), 1464–1480.
Leonard, J. A., Kramer, M. A., & Ungar, L. H. (1992). Using radial bais functions to approximate a function and its error bounds. IEEE Transactions on Neural Networks, 3(4), 624–627.
Liu, Y., Cukic, B., Menzies, T., Gururajan, S., Napolitano, M. (2003). “Validating an on-line adaptive system using support vector data description”. In Proceedings of fifteenth international conference on tools with artificial intelligence, Sacramento, CA.
Mackall, D., Nelson, S., & Schumann, J. (2002). Verification and validation of neural networks of aerospace applications, Technical report, CR-211409, NASA.
Martinetz, T., & Schulten, K. (1994). Topology representing networks. Neural Networks, 7(3), 507–522.
Mili, A., Cukic, B., Liu, Y., & Ben Ayed, R. (2003). Towards the verification and validation of on-line learning adaptive systems. In Computational methods in software engineering. Kluwer Scientific Publishing.
Napolitano, M., Molinaro, G., Innocenti, M., & Martinelli, D. (1998). A complete hardware package for a fault tolerant flight control system using online learning neural networks. Appears in Proc. of the 1999 American Control Conference (Vol. 4, pp. 2615–2619). San Diego, CA, USA, 1999.
Raz, O. (2000). Validation of online artificial neural networks–an informal classification of related approaches. Technical report, NASA Ames Research Center, Moffet Field, CA.
Schumann, J., & Nelson, S. (2002). Towards V&V of neural network based controllers. Workshop on Self-Healing Systems.
Schumann, J., & Gupta, P. Monitoring the Performance of a neuro-adaptive Controller. In Proc. of the twentyfourth international workshop on Bayesian inference and maximum entropy methods in Sscience and engineering, Garching bei München, Germany, 2004.
Tax, D. M. J., & Duin, R. P. W. (1999a). Data domain description using support vectors. In Proc. european symposium on artificial neural networks, Bruges, April 21–23, 1999 (pp. 251–257). Brussels: D-Facto.
Tax, D. M. J., & Duin, R. P. W. (1999b). Support vector domain description. Pattern Recognition Letters, 20(11–13), 1191–1199.
Tax, D. M. J. (2001). “One-class classification,” Dissertation, ISBN: 90-75691-05-x.
Vapnik, V. N. (1998). Statistical learning theory. NY: Wiley.
Yerramalla, S., Cukic, B., & Fuller, E. (2003a). Lyapunov stability analysis of quantization error for DCS neural networks. International joint conference on neural networks (IJCNN’03), Oregon.
Yerramalla, S., Fuller, E., & Cukic, B. (2003b). Lyapunov analysis of neural network stability in an adaptive flight control system. Sixth symposium on self stabilizing systems (SSS-03). San Francisco, CA, June 2003.
Yerramalla, S., Liu, Y., Fuller, E., Cukic B., & Gururajan, S. (2004). An approach to V&V of embedded adaptive systems. In Lecture notes in computer science (LNCS) Proceeding of third NASA-Goddard/IEEE workshop on formal approaches to agent-based systems, Springer-Verlag.
Yerramalla, S. (2005). Tability monitoring and analysis of online learning neural networks. Doctoral dissertation. WV: West Virginia University.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, Y., Cukic, B. & Gururajan, S. Validating neural network-based online adaptive systems: a case study. Software Qual J 15, 309–326 (2007). https://doi.org/10.1007/s11219-007-9017-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-007-9017-4