Stateless Q-Learning Algorithm for Training of Radial Basis Function Based Neural Networks in Medical Data Classification

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 230)

Abstract

In this article, the stateless Q-learning algorithm is used for the training process of two radial basis function based models: the radial basis function neural network (RBFNN) and the probabilistic neural network (PNN). The training process of considered models consists in the initialization and the adaptation of the smoothing parameter of the networks’ activation function in a hidden layer. The main idea of this approach is based on the appropriate computation of the smoothing parameter which relies on its update according to the stateless Q-learning algorithm. The proposed method is tested on six commonly available repository data sets. The prediction ability of the algorithm is assessed by computing the test set error on 10%, 20%, 30%, and 40% of examples drawn randomly from the entire input data. Obtained results are compared with the test errors achieved by PNN trained by means of the conjugate gradient procedure. It is shown that Q-learning method can be applied to the automatic adaptation of the smoothing parameter for both neural networks and provides better prediction ability results.

Keywords

radial basis function neural network probabilistic neural network stateless Q-learning algorithm smoothing parameter data classification prediction ability 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Broomhead, D.S., Lowe, D.: Multivariable Function Interpolation and Adaptive Networks. Complex Systems 2, 321–355 (1988)MathSciNetMATHGoogle Scholar
  2. 2.
    Specht, D.F.: Probabilistic neural networks. Neural Networks 3, 109–118 (1990)CrossRefGoogle Scholar
  3. 3.
    Maglogiannis, I., Sarimveis, H., Kiranoudis, C.T., Chatziioannou, A.A., Oikonomou, N., Aidinis, V.: Radial basis function neural networks classification for the recognition of idiopathic pulmonary fibrosis in microscopic images. IEEE Trans. Information Technology in Biomedicine 12, 42–54 (2008)CrossRefGoogle Scholar
  4. 4.
    Chu, F., Wang, L.: Applying RBF Neural Networks to Cancer Classification Based on Gene Expressions. In: International Joint Conference on Neural Networks, pp. 1930–1934. IEEE Press, Vancouver (2006)Google Scholar
  5. 5.
    Folland, R., Hines, E., Dutta, R., Boilot, P., Morgan, D.: Comparison of neural network predictors in the classification of tracheal-bronchial breath sounds by respiratory auscultation. Artificial Intelligence in Medicine 31, 211–220 (2004)CrossRefGoogle Scholar
  6. 6.
    Samanta, B., Bird, G.L., Kuijpers, M., Zimmerman, R.A., Jarvik, G.P., Wernovsky, G., et al.: Prediction of periventricular leukomalacia. Part II: Selection of hemodynamic features using computational intelligence. Artificial Intelligence in Medicine 46, 217–231 (2009)CrossRefGoogle Scholar
  7. 7.
    Mantzaris, D., Anastassopoulos, G., Adamopoulos, A.: Genetic algorithm pruning of probabilistic neural networks in medical disease estimation. Neural Networks 24, 831–835 (2011)CrossRefGoogle Scholar
  8. 8.
    Chen, S., Cowan, C.F.N., Grant, P.M.: Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans. Neural Networks 2, 302–309 (1991)CrossRefGoogle Scholar
  9. 9.
    Demuth, H., Beale, M.: Neural network toolbox user’s guide. The Mathworks, Inc. (1994)Google Scholar
  10. 10.
    Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall, New Jersey (1999)MATHGoogle Scholar
  11. 11.
    Specht, D.F., Romsdahl, H.: Experience with adaptive probabilistic neural networks and adaptive general regression neural networks. In: IEEE World Congress on Computational Intelligence 2, pp. 1203–1208. IEEE Press, Orlando (1994)Google Scholar
  12. 12.
    Chtioui, Y., Panigrahi, S., Marsh, R.: Conjugate gradient and approximate Newton methods for an optimal probabilistic neural network for food color classification. Optical Engineering 37, 3015–3023 (1998)CrossRefGoogle Scholar
  13. 13.
    Mao, K.Z., Tan, K.-C., Ser, W.: Probabilistic Neural Network Structure Determination for Pattern Classification. IEEE Trans. Neural Networks 11, 1009–1016 (2000)CrossRefGoogle Scholar
  14. 14.
    Gorunescu, F., Gorunescu, M., El-Darzi, E., Gorunescu, S.: An evolutionary computational approach to probabilistic neural network with application to hepatic cancer diagnosis. In: IEEE Symposium on Computer-Based Medical Systems, pp. 461–466. IEEE Press, Dublin (2005)CrossRefGoogle Scholar
  15. 15.
    Zhong, M., Coggeshall, D., Ghaneie, E., Pope, T., et al.: Gap-Based Estimation: Choosing the Smoothing Parameters for Probabilistic and General Regression Neural Networks. In: IEEE International Joint Conference on Neural Networks, pp. 1870–1877. IEEE Press, Vancouver (2006)Google Scholar
  16. 16.
    Watkins, C.J.C.H.: Learning from delayed Rewards. PhD Thesis, Cambridge University, Cambridge, England (1989)Google Scholar
  17. 17.
    Starzyk, J.A., Liu, Y., Batog, S.: A novel optimization algorithm based on reinforcement learning. In: Tenne, Y., Goh, C.-K. (eds.) Computational Intelligence in Optimization, ALO 7, pp. 27–47 (2010)Google Scholar
  18. 18.
    Parzen, E.: On estimation of a probability density function and mode. Annals of Mathematical Statistics 36, 1065–1076 (1962)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  20. 20.
    Lanzi, P.L.: Adaptive Agents with Reinforcement Learning and Internal Memory. In: Sixth International Conference on Simulation of Adaptive Behavior, pp. 333–342. The MIT Press, Cambridge (2000)Google Scholar
  21. 21.
    Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence, pp. 746–752. AAAI Press, Madison (1998)Google Scholar
  22. 22.
    McGlohon, M., Sandip, S.: Learning to cooperate in multi-agent systems by combining Q-learning and evolutionary strategy. In: First World Congress on Lateral-Computing, Bangalore, pp. 1–7 (2005)Google Scholar
  23. 23.
    UCI Machine Learning Repository, archive.ics.uci.edu/ml/datasets.html
  24. 24.
    Arlot, S.: A survey of cross-validation procedures for model selection. Statistics Surveys 4, 40–79 (2010)MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Kolmogorov, A.N.: On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Dokl. Akad. Nauk CCCP 114, 953–956 (1957)MathSciNetMATHGoogle Scholar
  26. 26.
    DTREG predictive modelling software (2013), http://www.dtreg.com

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Faculty of Electrical and Computer EngineeringRzeszow University of TechnologyRzeszowPoland

Personalised recommendations