Applied Intelligence

, Volume 17, Issue 2, pp 141–169 | Cite as

Neural and Neuro-Fuzzy Integration in a Knowledge-Based System for Air Quality Prediction

  • Ciprian-Daniel Neagu
  • Nikolaos Avouris
  • Elias Kalapanidas
  • Vasile Palade


In this paper we propose a unified approach for integrating implicit and explicit knowledge in neurosymbolic systems as a combination of neural and neuro-fuzzy modules. In the developed hybrid system, training data set is used for building neuro-fuzzy modules, and represents implicit domain knowledge. The explicit domain knowledge on the other hand is represented by fuzzy rules, which are directly mapped into equivalent neural structures. The aim of this approach is to improve the abilities of modular neural structures, which are based on incomplete learning data sets, since the knowledge acquired from human experts is taken into account for adapting the general neural architecture. Three methods to combine the explicit and implicit knowledge modules are proposed. The techniques used to extract fuzzy rules from neural implicit knowledge modules are described. These techniques improve the structure and the behavior of the entire system. The proposed methodology has been applied in the field of air quality prediction with very encouraging results. These experiments show that the method is worth further investigation.

neural and neuro-fuzzy integration modular structure fuzzy rule-based system air quality prediction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. Becraft, P.L. Lee, and R.B. Newell, “Integration of neural networks and expert systems for process fault diagnosis,” in Procs. Of the International Joint Conference on Artificial Intelligence, 1991, pp. 832–837.Google Scholar
  2. 2.
    B. Kosko, Neural Networks and Fuzzy Systems, Prentice-Hall: Englewood Cliffs, NJ, 1992.Google Scholar
  3. 3.
    C.-T. Lin and C.S.G. Lee, “Neural-network-based fuzzy logic control and decision system,” IEEE Trans. On Computers, vol. 40, no. 12, pp. 1320–1336, 1991.Google Scholar
  4. 4.
    V. Palade, “Hybrid intelligent systems for process control,” Ph.D. Dissertation, University “Dunarea de Jos” of Galati, 1999.Google Scholar
  5. 5.
    A.F. da Rocha, Neural Nets: A Theory for Brains and Machines, Lecture Notes in Artificial Intelligence, Springer-Verlag: Berlin, 1992.Google Scholar
  6. 6.
    J. Sima and J. Cervenka, “Neural knowledge processing in expert systems,” TR no. V-735, Institute of Computer Science, Academy of Sciences of the Czech Republic, 1997.Google Scholar
  7. 7.
    H. Takagi, “Cooperative system of neural networks and fuzzy logic and its application to consumer products,” in Industrial Applications of Fuzzy Control and Intelligent Systems, edited by J. Yen, R. Langari, and L.A. Zadeh, Van Nostrand Reinhold: NY, 1994.Google Scholar
  8. 8.
    S. Wermter and R. Sun, Hybrid Neural Systems, SpringerVerlag: Heidelberg, 2000.Google Scholar
  9. 9.
    M. Funabashi and A. Maeda, “Fuzzy and neural hybrid expert systems: Synergetic AI,” IEEE Expert, pp. 32–40, 1995.Google Scholar
  10. 10.
    L. Shastri and V. Ajjanagadde, “From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings,” Behavioral and Brain Sciences, vol. 16, no. 3, pp. 417–494, 1993.Google Scholar
  11. 11.
    R. Sun, Integrating Rules and Connectionism for Robust Commonsense Reasoning, Wiley: New York, 1994.Google Scholar
  12. 12.
    M. Hilario, “An overview of strategies for neurosymbolic integration,” in Connectionist-Symbolic Integration: From Uni-fied to Hybrid Approaches, edited by R. Sun and F. Alexandre, Lawrence Erlbaum Associates, ch. 2, 1997.Google Scholar
  13. 13.
    R. Khosla and T.S. Dillon, Engineering Hybrid Multiagent Systems, Kluwer: Dordrecht, 1997.Google Scholar
  14. 14.
    A.F. da Rocha, “The fuzzy neuron: Biology and mathematics,” in Proc. of IFSA'91, Brussels, vol. 1, 1991, pp. 176–179.Google Scholar
  15. 15.
    R. Sun, “Integrating rules and connectionism for robust reasoning,” TR no. CS-90-154, Waltham, MA, Brandeis University, 1991.Google Scholar
  16. 16.
    R. Sun, “CONSYDERR: A two level hybrid architecture for structuring knowledge for commonsense reasoning,” in Procs. of the First International Symposium on Integrating Knowledge and Neural Heuristics, Florida, USA, 1994, pp. 32–39.Google Scholar
  17. 17.
    S.K. Pal and A. Mitra, “Multilayer perceptron, fuzzy sets, and classification,” IEEE Transactions on Neural Networks, vol. 3, no. 5, pp. 683–697, 1992.Google Scholar
  18. 18.
    J.M. Benitez, A. Blanco, M. Delgado, and I. Requena, “Neural methods for obtaining fuzzy rules,” Mathware Soft Computing, vol. 3, pp. 371–382, 1996.Google Scholar
  19. 19.
    J.M. Benitez, J.L. Castro, and I. Requena, “Are artificial neural networks black boxes?” IEEE Transactions on Neural Networks, vol. 8, no. 5, pp. 1157–1164, 1997.Google Scholar
  20. 20.
    C.-D. Neagu and V. Palade, “An interactive fuzzy operator used in rule extraction from neural networks,” Neural Networks World Journal, special issue, July 2000.Google Scholar
  21. 21.
    C.W. Omlin and C.L. Giles, “Extraction of rules from discretetime recurrent neural networks,” Neural Networks, vol. 9, pp. 41–52, 1996.Google Scholar
  22. 22.
    V. Palade, “GA optimization of knowledge extraction from neural networks,” in Procs. of ICONIP99, Sidney, 1999.Google Scholar
  23. 23.
    J.J. Buckley and Y. Hayashi, “Neural nets for fuzzy systems,” Fuzzy Sets and Systems, vol. 71, pp. 265–276, 1995.Google Scholar
  24. 24.
    R. Fuller, Introduction to Neuro-Fuzzy Systems, Advances in Soft Computing Series, Springer-Verlag: Berlin, 1999.Google Scholar
  25. 25.
    D.E. Rumelhart and J.L. McClelland, Parallel Distributed Processing, Explanations in the Microstructure of Cognition, MIT Press: Cambridge, MA, 1986.Google Scholar
  26. 26.
    W. Pedrycz and A.F. da Rocha, “Fuzzy-set based models of neurons and knowledge-based networks,” IEEE Transactions on Fuzzy Systems, vol. I, no. 5, pp. 254–266, 1993.Google Scholar
  27. 27.
    L.A. Zadeh, “The role of fuzzy logic in the management of uncertainty in expert systems,” Fuzzy Sets and Systems, vol. 11, no. 3, pp. 199–227, 1983.Google Scholar
  28. 28.
    R.J.S. Jang and C.T. Sun, “Functional equivalence between radial basis function networks and fuzzy inference systems,” IEEE Transactions on Neural Networks, vol. 4, no. 1, pp. 156–159, 1993.Google Scholar
  29. 29.
    I. Jagielska, C. Matthews, and T. Whitfort, “A study in experimental evaluation of neural network and genetic algorithm techniques for knowledge acquisition in fuzzy classification systems,” in Proc. of IEEE ICNN Conf. on Neural Networks, Houston, 1997.Google Scholar
  30. 30.
    I. Jagielska, “Linguistic rule extraction from neural networks for descriptive datamining,” in Proc. of 1998 Second International Conference on Knowledge-Based Intelligent Electronic Systems, edited by L.C. Jain and R.K. Jain, Adelaide, 1998.Google Scholar
  31. 31.
    N. Kasabov, “Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems,” Fuzzy Sets and Systems, vol. 82, pp. 135–149, 1996.Google Scholar
  32. 32.
    I. Enbutsu, K. Baba, and N. Hara, “Fuzzy rule extraction from a multilayered network,” in Procs. of IJCNN'91, Seattle, 1991.Google Scholar
  33. 33.
    W. Pedrycz, “Fuzzy neural networks and neurocomputations,” Fuzzy Sets and Systems, vol. 56, pp. 1–28, 1993.Google Scholar
  34. 34.
    R. Langari, “Synthesis of nonlinear control strategies via fuzzy logic,” in Proc. of American Control Conference, 1993, pp. 1855–1859.Google Scholar
  35. 35.
    C.-D. Neagu and V. Palade, “Fuzzy computing in a multi purpose neural network implementation,” in Proc. of the International Conference 6th Fuzzy Days in Dortmund, edited by B. Reusch, Springer Verlag: Berlin, pp. 697–700, 1999.Google Scholar
  36. 36.
    R.E. Abdel and M.A. Elhadidy, “Modeling and forecasting the daily maximum temperature using abductive machine learning,” Oceanographic Literature Review, vol. 43, no. 1, pp. 25–37, 1996.Google Scholar
  37. 37.
    C.-D. Neagu and V. Palade, “A neuro-fuzzy approach to photochemical pollution prediction,” in Proc. of International Symposium On Systems Theory, Automation, Robotics, Computers, Informatics, Electronics and Instrumentation, SINTES10, Craiova, 2000.Google Scholar
  38. 38.
    C.-D. Neagu and S. Bumbaru, “Explicit knowledge representation using multi purpose neural networks,” in Proc. of the International Conference on Control Systems and Computer Science CSCS12, edited by I. Dumitrache and M. Dobre, Bucharest, vol. 2, pp. 37–42, 1999.Google Scholar
  39. 39.
    C.-D. Neagu, S. Bumbaru, and N. Icriverzi, “Artificial neuronal operators for fuzzy computing,” in Procs. of the 10th Symposium on Modeling, Simulation and Identification Systems SIMSIS10'98, Galati, 1998, pp. 12–15.Google Scholar
  40. 40.
    C.-D. Neagu, M. Negoita, and V. Palade, “Aspects of integration of explicit and implicit knowledge in connectionist expert systems,” in Proc. of ICONIP'99, vol. 2, Sidney, 1999, pp. 759–764.Google Scholar
  41. 41.
    E.H. Mamdani, “Application of fuzzy logic to approximate reasoning using linguistic synthesis,” IEEE Trans. Computers, vol. C-26, no. 12, pp. 1192–1191, 1977.Google Scholar
  42. 42.
    S. Haykin, Neural Networks: A Comprehensive Foundation, IEEE Press: New Jersey, 1994.Google Scholar
  43. 43.
    R.A. Jacobs, M.I. Jordan, and A.G. Barto, “Task decomposition through competition in a modular connectionist architecture: The what and where vision tasks,” Cognitive Science, vol. 15, pp. 219–250, 1991.Google Scholar
  44. 44.
    S. Hashem, “Optimal linear combinations of neural networks,” Neural Networks, vol. 10, no. 4, pp. 599–614, 1997.Google Scholar
  45. 45.
    E. Prem, M. Mackinger, G. Dorffner, G. Porenta, and H. Sochor, “Concept support as a method for programming neural networks with symbolic knowledge,” TR-93-04, OEFAI, 1993.Google Scholar
  46. 46.
    J.S. Bridle, “Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters,” in Advances in Neural Information Processing Systems, edited by D.J. Touretzky, Kaufmann: New York, vol. 2, pp. 211–217, 1990.Google Scholar
  47. 47.
    M. Hagan and M. Beale, Neural Network Design, PWS: Boston, 1996.Google Scholar
  48. 48.
    M. Sugeno and G.T. Kang, “Structure identification of fuzzy model,” Fuzzy Sets and Systems, vol. 28, pp. 15–33, 1988.Google Scholar
  49. 49.
    S.S. Lee, “Predicting atmospheric ozone using neural networks as compared to some statistical methods,” in Proc. of the IEEE Technical Applications Conference and Workshops, NORTHCON' 95, 1995, pp. 309–316.Google Scholar
  50. 50.
    L.E. Sucar, J. Perez-Brito, J.C. Ruiz-Suarez, and E. Morales, “Learning structure from data and its application to ozone prediction,” Applied Intelligence, vol. 7, pp. 327–338, 1997.Google Scholar
  51. 51.
    N.M. Avouris, “Cooperating knowledge-based systems for environmental decision support,” Knowledge-Based Systems, vol. 8, no. 1, pp. 39–54, 1995.Google Scholar
  52. 52.
    M. Boznar, “Pattern selection strategies for a neural networkbased short term air pollution prediction model,” in Proc. of the IEEE International Conference on Intelligent Information Systems, IIS'97, 1997, pp. 340–344.Google Scholar
  53. 53.
    N.M. Avouris and E. Kalapanidas, “Expert systems and artificial intelligence techniques in air pollution prediction,” in Mathematical Modelling of Atmospheric Pollution, edited by N. Moussiopoulos, Thessaloniki, 1997.Google Scholar
  54. 54.
    S.K. Murthy, S. Kasif, and S. Salzberg, “A system for induction of oblique decision trees,” Journal of Artificial Intelligence Research, vol. 2, pp. 1–32, 1994.Google Scholar
  55. 55.
    G.P. Lekkas, N.M.Avouris, and L.G.Viras, “Case-based reasoning in environmental monitoring applications,” Applied Artificial Intelligence, vol. 8, no. 3, pp. 359–376, 1994.Google Scholar
  56. 56.
    M. Zickus, “Influence of meteorological parameters on the urban air pollution and its forecast,” Ph.D. Thesis, University of Vilnius, 1999.Google Scholar
  57. 57.
    WHO (World Health Organization), Air quality guidelines for Europe, WHO: Copenhagen, 1987.Google Scholar
  58. 58.
    B.J. Finlayson-Pitts and J.N. Pitts, Atmospheric Chemistry: Fundamentals & Experimental Techniques, Wiley: NY, 1986.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Ciprian-Daniel Neagu
    • 1
  • Nikolaos Avouris
    • 2
  • Elias Kalapanidas
    • 2
  • Vasile Palade
    • 1
  1. 1.Department of Applied Informatics, “Dunarea de Jos”University of GalatiRomania
  2. 2.Electrical and Computer Engineering DepartmentUniversity of PatrasGreece

Personalised recommendations