Skip to main content
SpringerLink
Log in

Evolutionary Multiobjective Neural Network Models Identification: Evolving Task-Optimised Models

  • Chapter
New Advances in Intelligent Signal Processing

Part of the book series: Studies in Computational Intelligence ((SCI,volume 372))

Abstract

In the system identification context, neural networks are black-box models, meaning that both their parameters and structure need to be determined from data. Their identification is often done iteratively in an ad-hoc fashion focusing the first aspect. Frequently the selection of inputs, model structure, and model order are underlooked subjects by practitioners, because the number of possibilities is commonly huge, thus leaving the designer at the hands of the curse of dimensionality. Moreover, the design criteria may include multiple conflicting objectives, which gives to the model identification problem a multiobjective combinatorial optimisation character. Evolutionary multiobjective optimisation algorithms are particularly well suited to address this problem because they can evolve optimised model structures that meet pre-specified design criteria in acceptable computing time. In this article the subject is reviewed, the authors present their approach to the problem in the context of identifying neural network models for time-series prediction and for classification purposes, and two application case studies are described, one in each of these fields.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amari, S., Murata, N., Müller, K.R., Finke, M., Yang, H.: Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks 8(5), 985–996 (1997), doi:10.1109/72.623200

    Article  Google Scholar 

  2. Bauer, M., Buchtala, O., Horeis, T., Kern, R., Sick, B., Wagner, R.: Technical data mining with evolutionary radial basis function classifiers. Applied Soft Computing 9, 765–774 (2009), doi:10.1016/j.asoc.2008.07.007

    Article  Google Scholar 

  3. Billings, S.A., Zheng, G.L.: Radial basis function network configuration using genetic algorithms. Neural Networks 8(6), 877–890 (1995)

    Article  Google Scholar 

  4. Branke, J.: Evolutionary algorithms for neural network design and training. Tech. Rep. 322, University of Karlsruhe, Institute AIFB, Karlsruhe, Germany (1995)

    Google Scholar 

  5. Carse, B., Pipe, A.G., Fogarty, T.C., Hill, T.: Evolving radial basis function neural networks using a genetic algorithm. In: IEEE International Conference on Evolutionary Computation, vol. 1, pp. 300–305 (1995), doi:10.1109/ICEC.1995.489163

    Google Scholar 

  6. Chen, S., Wu, Y., Luk, B.: Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Transactions on Neural Networks 10(5), 1239–1243 (1999)

    Article  Google Scholar 

  7. Coello Coello, C.: Recent trends in evolutionary multiobjective optimization. In: Jain, L., Wu, X., Abraham, A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization, Advanced Information and Knowledge Processing, pp. 7–32. Springer, Heidelberg (2005), http://dx.doi.org/10.1007/1-84628-137-7_2 , doi:10.1007/1-84628-137-7-2

    Chapter  Google Scholar 

  8. Coello Coello, C.: Evolutionary multi-objective optimization: a historical view of the field. IEEE Computational Intelligence Magazine 1(1), 28–36 (2006), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1597059&tag=1 , doi:10.1109/MCI.2006.1597059

    Article  MathSciNet  Google Scholar 

  9. Crispim, E.M., Ferreira, P.M., Ruano, A.E.: Prediction of the solar radiation evolution using computational intelligence techniques and cloudiness indices. International Journal of Innovative Computing, Information and Control 4(5), 1121–1133 (2008)

    Google Scholar 

  10. Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. John Wiley & Sons., Ltd, Chichester (2001)

    MATH  Google Scholar 

  11. Ferreira, P.M., Faria, E., Ruano, A.E.: Neural network models in greenhouse air temperature prediction. Neurocomputing 43(1-4), 51–75 (2002)

    Article  MATH  Google Scholar 

  12. Ferreira, P.M., Martins, I.A., Ruano, A.E.: Cloud and clear sky pixel classification in ground-based all-sky hemispherical digital images. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)

    Google Scholar 

  13. Ferreira, P.M., Ruano, A.E.: Exploiting the separability of linear and non-linear parameters in radial basis function neural networks. In: IEEE Symposium 2000: Adaptive Systems for Signal Processing, Communications, and Control, Canada, pp. 321–326 (2000), http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=882493 , doi:10.1109/ASSPCC.2000.882493

  14. Ferreira, P.M., Ruano, A.E.: Application of computational intelligence methods to greenhouse environmental modelling. In: (WCCI 2008) IJCNN 2008 (IEEE World Congress on Computational Intelligence). 2008 IEEE International Joint Conference on Neural Networks, pp. 3582–3589 (2008), doi:10.1109/IJCNN.2008.4634310

    Google Scholar 

  15. Ferreira, P.M., Ruano, A.E.: On-line sliding-window methods for process model adaptation. IEEE Transactions on Instrumentation and Measurement 58(9), 3012–3020 (2009), doi:10.1109/tim.2009.2016818

    Article  Google Scholar 

  16. Ferreira, P.M., Ruano, A.E., Fonseca, C.: Genetic assisted selection of rbf model structures for greenhouse inside air temperature prediction. In: IEEE Conference on Control Applications, Turkey, pp. 576–581 (2003)

    Google Scholar 

  17. Ferreira, P.M., Ruano, A.E., Fonseca, C.: Evolutionary multi-objective design of radial basis function networks for greenhouse environmental control. In: IFAC World Congress on Automatic Control 16th, Czech Republic (2005)

    Google Scholar 

  18. Ferreira, P.M., Ruano, A.E., Pestana, R.: Improving the identification of rbf predictive models to forecast the portuguese electricity consumption. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)

    Google Scholar 

  19. Ferreira, P.M., Ruano, A.E., Pestana, R., Kóczy, L.T.: Evolving rbf predictive models to forecast the portuguese electricity consumption. In: ICONS 2009: The 2nd IFAC Int. Conference on Intelligent Control Systems and Signal Processing, Istanbul, Turkey (2009)

    Google Scholar 

  20. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley Interscience, Hoboken (2000)

    Book  Google Scholar 

  21. Fonseca, C., Fleming, P.: Non-linear system identification with multiobjective genetic algorithms. In: Proceedings of the 13 IFAC World Congress, vol. C, pp. 187–192 (1996)

    Google Scholar 

  22. Fonseca, C., Fleming, P.: Multiobjective optimization and multiple constraint handling with evolutionary algorithms i: A unified formulation. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 28(1), 26–37 (1998), doi:10.1109/3468.650319

    Article  Google Scholar 

  23. Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, Inc., London (1981)

    MATH  Google Scholar 

  24. Griffin, I., Fleming, P.: An overview of non-linear identification and control with neural networks. In: Ruano, A.E. (ed.) Intelligent Control Using Soft-Computing Methodologies. Control Series, pp. 89–118. IEE Publishing (2005)

    Google Scholar 

  25. Guillén, A., Pomares, H., González, J., Rojas, I., Valenzuela, O., Prieto, B.: Parallel multiobjective memetic rbfnns design and feature selection for function approximation problems. Neurocomputing 72(16-18), 3541–3555 (2009), doi:10.1016/j.neucom.2008.12.037

    Article  Google Scholar 

  26. Haykin, S.: Neural Networks: a Comprehensive Foundation, 2nd edn. Prentice Hall, Inc., Englewood Cliffs (1999)

    MATH  Google Scholar 

  27. Jung, J., Reggia, J.: Evolutionary design of neural network architectures using a descriptive encoding language. IEEE Transactions on Evolutionary Computation 10(6), 676–688 (2006), doi:10.1109/TEVC.2006.872346

    Article  Google Scholar 

  28. Kaylani, A., Georgiopoulos, M., Mollaghasemi, M., Anagnostopoulos, G.C., Sentelle, C., Zhong, M.: An adaptive multiobjective approach to evolving art architectures. IEEE Transactions on Neural Networks 21(4), 529–550 (2010)

    Article  Google Scholar 

  29. Lee, C.W., Shin, Y.C.: Growing radial basis function networks using genetic algorithm and orthogonalization. International Journal of Innovative Computing, Information and Control 5(11(A)), 3933–3948 (2009)

    Google Scholar 

  30. Leung, C., Lam, F.: Performance analysis for a class of iterative image thresholding algorithms. Pattern Recognition 29(9), 1523–1530 (1996)

    Article  Google Scholar 

  31. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics 11(2), 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  32. McDonnell, J., Waagen, D.: Determining neural network hidden layer size using evolutionary programming. In: Proceedings of the 1993 World Congress on Neural Networks, vol. III, pp. 564–657 (1993)

    Google Scholar 

  33. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man and Cybernetics SMC-9(1), 62–66 (1979)

    MathSciNet  Google Scholar 

  34. Peck, C., Dhawan, A., Meyer, C.: Genetic algorithm based input selection for a neural network function approximator with applications to ssme health monitoring. In: IEEE International Conference on Neural Networks, vol. 2, pp. 1115–1122 (1993), doi:10.1109/ICNN.1993.298714

    Google Scholar 

  35. Ridler, T., Calvard, S.: Picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man and Cybernetics SMC-8(8), 630–632 (1978)

    Google Scholar 

  36. Rodríguez-Vázquez, K., Fonseca, C., Fleming, P.: Identifying the structure of nonlinear dynamic systems using multiobjective genetic programming. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 34(4), 531–545 (2004), doi:10.1109/TSMCA.2004.826299

    Article  Google Scholar 

  37. Ruano, A., Crispim, E., Frazão, P.: Moga design of neural network predictors of inside temperature in public buildings. In: Balas, V., Fodor, J., Várkonyi-Kóczy, A. (eds.) Soft Computing Based Modeling in Intelligent Systems. SCI, vol. 196, pp. 35–61. Springer, Heidelberg (2009), doi:10.1007/978-3-642-00448-3-3

    Chapter  Google Scholar 

  38. Ruano, A., Fleming, P., Jones, D.: Connectionist approach to pid autotuning. IEE Proceedings (part D), 139(3), 279–285 (1992), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=141517

    Article  Google Scholar 

  39. Ruano, A., Jones, D., Fleming, P.: A new formulation of the learning problem of a neural network controller. In: Proceedings of the 30th IEEE Conference on Decision and Control, vol. 1, pp. 865–866 (1991), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=261439 , doi:10.1109/CDC.1991.261439

  40. Ruano, A.E., Ferreira, P.M., Cabrita, C., Matos, S.: Training neural networks and neuro-fuzzy systems: A unified view. In: Proceedings of the 15th IFAC World Congress, vol. 15 (2002)

    Google Scholar 

  41. Ruano, A.E., Ferreira, P.M., Fonseca, C.: An overview of non-linear identification and control with neural networks. In: Ruano, A.E. (ed.) Intelligent Control Using Soft-Computing Methodologies. Control Series, pp. 37–87. IEE Publishing (2005)

    Google Scholar 

  42. Ruano, A.E., Ferreira, P.M., Mendes, H.: Moga design of temperature and relative humidity models for predictive thermal comfort. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)

    Google Scholar 

  43. Ruano, A.E., Fleming, P., Teixeira, C., Rodríguez-Vázquez, K., Fonseca, C.: Nonlinear identification of aircraft gas-turbine dynamics. Neurocomputing 55(3-4), 551–579 (2003), doi:10.1016/S0925-2312(03)00393-X

    Article  Google Scholar 

  44. Sezgin, M., Sankur, B.: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 13(1), 146–165 (2004)

    Article  Google Scholar 

  45. Sjöberg, J., Ljung, L.: Overtraining, regularization, and searching for minimum with application to neural networks. In: Preprint IFAC Symposium on Adaptive Systems in Control and Signal Processing, pp. 669–674 (1994)

    Google Scholar 

  46. Trussel, H.: Comments on picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man and Cybernetics SMC-9(5), 311 (1979)

    Article  Google Scholar 

  47. Yen, G.: Multi-objective evolutionary algorithm for radial basis function neural network design. In: Jin, Y. (ed.) Multi-objective machine learning. SCI, vol. 16, pp. 221–239. Springer, Heidelberg (2006), doi:10.1007/3-540-33019-4-10

    Chapter  Google Scholar 

  48. Zitzler, E., Laumanns, M., Bleuler, S.: A tutorial on evolutionary multiobjective optimization. In: Gandibleux, X., et al. (eds.) Metaheuristics for Multiobjective Optimisation. Lecture Notes in Economics and Mathematical Systems, vol. 535, pp. 3–37. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Algarve Science & Technology Park, University of Algarve, Campus de Gambelas - Pav. A5, 8005-139, Faro, Portugal

    Pedro M. Ferreira

  2. Centre for Intelligent Systems, University of Algarve - FCT, Campus de Gambelas, 8005-139, Faro, Portugal

    Pedro M. Ferreira & António E. Ruano

Authors
  1. Pedro M. Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. António E. Ruano
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CSI / Centre for Intelligent Systems, Universidade do Algarve, Campus de Gambelas, 8000, Faro, Portugal

    António E. Ruano

  2. Dept.of Mechatronics andVehicle Engineering, Óbuda University, Népszínház u. 8, 1081, Budapest, Hungary

    Annamária R. Várkonyi-Kóczy

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ferreira, P.M., Ruano, A.E. (2011). Evolutionary Multiobjective Neural Network Models Identification: Evolving Task-Optimised Models. In: Ruano, A.E., Várkonyi-Kóczy, A.R. (eds) New Advances in Intelligent Signal Processing. Studies in Computational Intelligence, vol 372. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11739-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-11739-8_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-11738-1

  • Online ISBN: 978-3-642-11739-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation