Performance Estimation of a Neural Network-Based Controller

  • Johann Schumann
  • Yan Liu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3972)


Biologically inspired soft computing paradigms such as neural networks are popular learning models adopted in adaptive control systems for their ability to cope with a changing environment. However, continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure a reliable system performance.

In this paper, we present a dynamic approach to estimate the performance of two types of neural networks employed in an adaptive flight controller: the validity index for the outputs of a Dynamic Cell Structure (DCS) network and confidence levels for the outputs of a Sigma-Pi (or MLP) network. Both tools provide statistical inference of the neural network predictions and an estimate of the current performance of the network. We further evaluate how the quality of each parameter of the network (e.g., weight) influences the output of the network by defining a metric for parameter sensitivity and parameter confidence for DCS and Sigma-Pi networks. Experimental results on the NASA F-15 flight control system demonstrate that our techniques effectively evaluate the network performance and provide validation inferences in a real-time manner.


Neural Network Performance Estimation Adaptive Controller Validity Index Reference Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahrns, I., Bruske, J., Sommer, G.: On-line Learning with Dynamic Cell Structures. In: Fogelman-Soulié, F., Gallinari, P. (eds.) Proc. Int. Conf. Artificial Neural Networks, Nanterre, France, vol. 2. EC2, pp. 141–146 (1995)Google Scholar
  2. 2.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  3. 3.
    Bruske, J., Sommer, G.: Dynamic Cell Structures. Proc. Neural Information Processing Systems 7, 497–504 (1995)Google Scholar
  4. 4.
    Calise, A., Rysdyk, R.: Nonlinear Adaptive Flight Control Using Neural Networks. IEEE Control Systems Magazine 21(6), 14–26 (1998)CrossRefGoogle Scholar
  5. 5.
    Fritzke, B.: Growing Cell Structures - a Self-organizing Network for Unsupervised and Supervised Learning. Neural Networks 7(9), 1441–1460 (1993)CrossRefGoogle Scholar
  6. 6.
    Fuller, E., Yerramalla, S., Cukic, B., Gururajan, S.: An Approach to Predicting Non-deterministic Neural Network Behavior. In: Proc. Intl. Joint Conference on Neural Networks, IJCNN (2005)Google Scholar
  7. 7.
    Gupta, P., Schumann, J.: A Tool for Verification and Validation of Neural Network Based Adaptive Controllers for High Assurance Systems. In: Proc. High Assurance Software Engineering. IEEE Press, Los Alamitos (2004)Google Scholar
  8. 8.
    Kohonen, T.: Self-Organizing Maps. Springer, New York (1997)MATHGoogle Scholar
  9. 9.
    Leonard, J.A., Kramer, M.A., Ungar, L.H.: Using Radial Basis Functions to Approximate a Function and Its Error Bounds. IEEE Transactions on Neural Networks 3(4), 624–627 (1992)CrossRefGoogle Scholar
  10. 10.
    Liu, Y.: Validating A Neural Network-based Online Adaptive System. PhD thesis, West Virginia University, Morgantown (2005)Google Scholar
  11. 11.
    Liu, Y., Cukic, B., Jiang, M., Xu, Z.: Predicting with Confidence - An Improved Dynamic Cell Structure. In: Wang., L., Chen, K., Ong., Y.S. (eds.) Advances in Neural Computation. LNCS, vol. 1, pp. 750–759. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Mackall, D., Nelson, S., Schumann, J.: Verification and Validation of Neural Networks of Aerospace Applications. Technical Report CR-211409, NASA (2002)Google Scholar
  13. 13.
    Martinez, T., Schulten, K.: Topology Representing Networks. Neural Networks 7(3), 507–522 (1994)CrossRefGoogle Scholar
  14. 14.
    Norgaard, M., Ravn, O., Poulsen, N., Hansen, L.K.: Neural Networks for Modeling and Control of Dynamic Systems. Springer, Heidelberg (2002)Google Scholar
  15. 15.
    Rumelhart, McClelland, The PDP Research Group: Parallel Distributed Processing. MIT Press, Cambridge (1986)Google Scholar
  16. 16.
    Rysdyk, R., Calise, A.: Fault Tolerant Flight Control via Adaptive Neural Network Augmentation. In: AIAA-98-4483, pp. 1722–1728 (1998)Google Scholar
  17. 17.
    Schumann, J., Gupta, P.: Monitoring the Performance of A Neuro-adaptive Controller. In: Fischer, R., Preuss, R., von Toussaint, U. (eds.) Proc. 24th International Workshop on Bayesian Inference and Maximum Entropy Methods in Sciences and Engineering (MAXENT), AIP, pp. 289–296 (2004)Google Scholar
  18. 18.
    Schumann, J., Gupta, P., Jacklin, S.: Toward Verification and Validation of Adaptive Aircraft Controllers. In: Proc. IEEE Aerospace Conference. IEEE Press, Los Alamitos (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Johann Schumann
    • 1
  • Yan Liu
    • 2
  1. 1.RIACS/NASA AmesMoffett FieldUSA
  2. 2.Motorola LabsSchaumburgUSA

Personalised recommendations