Abstract
Reviews of several methods for modeling of complex multidimensional nonlinear systems were presented. It turns out that power of neural networks grows linearly with its width and exponentially with its depth. Unfortunately training of traditional MLP Multi-Layer Perceptron deep architectures is very difficult. The paper presents couple solution to this problem. One solution is to use neural network with connections across layers such as FCC Fully Connected Cascade or BMLP - Bridged Multi-Layer Perceptron. Another alternative is to use DNN - Dual Neural Networks which is described with more details.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wilamowski, B.M.: Neural Network Architectures and Learning algorithms – How Not to Be Frustrated with Neural Networks. IEEE Industrial Electronics Magazine 3(4), 56–63 (2009) (best paper award)
Hecht-Nielsen, R.: Theory of the Back Propagation Neural Network. In: Proc. 1989 IEEE IJCNN, pp. 1593–1605. IEEE Press, New York (1989)
Peng, J., Li, K., Irwin, G.W.: A New Jacobian Matrix for Optimal Learning of Single-Layer Neural Networks. IEEE Trans. on Neural Networks 19(1), 119–129 (2008)
Wilamowski, B.M., Yu, H.: Improved Computation for Levenberg Marquardt Training. IEEE Trans. on Neural Networks 21(6), 930–937 (2010)
Wilamowski, B.M., Hunter, D., Malinowski, A.: Solving parity-N problems with feedforward neural networks. In: Proc. 2003 IEEE IJCNN, pp. 2546–2551. IEEE Press (2003)
Wilamowski, B.M., Yu, H., Chung, K.T.: Parity-N Problems as a Vehicle to Compare Efficiency of Neural Network Architectures. In: Industrial Electronics Handbook, 2nd edn. Intelligent Systems, vol. 5, ch. 10, pp. 10-1–10-8. CRC Press (2011)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)
Levenberg, K.: A method for the solution of certain problems in least squares. Quarterly of Applied Mathematics 2(2), 164–168 (1944)
Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Trans. on Neural Networks 5(6), 989–993 (1994)
Zadeh, L.A.: Fuzzy Sets. Information and Control 8, 338–353 (1965)
Mamdani, E.H.: Application of Fuzzy Algorithms for Control of Simple Dynamic Plant. IEEE Proceedings 121(12), 1585–1588 (1974)
Sugeno, M., Kang, G.T.: Structure Identification of Fuzzy Model. Fuzzy Sets and Systems 28(1), 15–33 (1988)
Roger, J.S., Sun, C.T.: Functional Equivalence Between Radial Basis Function Networks and Fuzzy Inference Systems. IEEE Trans. on Neural Networks 4(1), 156–159 (1993)
Jin, Y., Sendhoff, B.: Extracting Interpretable Fuzzy Rules from RBF Networks. Neural Process. Lett. 17(2), 149–164 (2003)
Li, W., Hori, Y.: An Algorithm for Extracting Fuzzy Rules Based on RBF Neural Network. IEEE Trans. on Industrial Electronics 53(4) (August 2006)
Moody, J., Darken, C.J.: Learning with Localized Receptive Fields. In: Touretzky, D., Hinton, G., Sejnowski, T. (eds.) Proc. 1988 Connectionist Models Summer School, Carnegie Mellon University. Morgan Kaufmann Publishers (1988)
Hecht-Nielsen, R.: Counterpropagation networks. Applied Optics 26(23), 4979–4984 (1987)
Pao, Y.H.: Adaptive Pattern Recognition and Neural Networks. Addison-Wesley Publishing Co., Reading (1989)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer (1995)
Sch’́olkopf, B., Burges, C.J.C., Smola, A.J. (eds.): Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge (1999)
Arel, I., Rose, D.C., Karnowski, T.P.: Deep Machine Learning – A New Frontier in Artificial Intelligence Research. IEEE Computational Intelligence Magazine 5(4), 13–18 (2010)
Chen, K., Salman, A.: Learning Speaker-Specific Characteristics with a Deep Neural Architecture. IEEE Trans. on Neural Networks 22(11), 1744–1756 (2011)
Binfet, J., Wilamowski, B.: Microprocessor Implementation of Fuzzy Systems and Neural Networks. In: International Joint Conference on Neural Networks (IJCNN 2001), Washington, DC, pp. 234–239 (2001)
Xie, T., Yu, H., Wilamowski, B.: Replacing fuzzy systems with neural networks. In: HSI 2010, May 13-15, pp. 189–193 (2010)
Rutkowska, D., Hayashi, Y.: Neuro-fuzzy systems approaches. International Journal of Advanced Computational Intelligence 3(3), 177–185 (1999)
Yu, H., Xie, T., Wilamowski, B.M.: Neuro-Fuzzy System. In: Industrial Electronics Handbook, 2nd edn. Intelligent Systems, vol. 5, ch. 20, pp. 20-1–20-9. CRC Press (2011)
Cover, T.M.: Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition. IEEE Trans. on Electronic Computers EC–14, 326–334 (1965)
Wilamowski, B.M., Jaeger, R.C.: Implementation of RBF Networks by Feedforward Sigmoidal Neural Networks. In: Dagli, C.H., et al. (eds.) Intelligent Engineering Systems Through Artificial Neural Networks, New York, vol. 7, pp. 183–188 (1997)
Park, J., Sandberg, I.W.: Universal Approximation Using Radial-Basis-Function Networks. Neural Computation 3(2), 246–257 (1991)
Hunter, D., Yu, H., Pukish, M.S., Kolbusz, J., Wilamowski, B.M.: Selection of Proper Neural Network Sizes and Architectures - Comparative Study. IEEE Trans. on Industrial Informatics 8, 228–240 (2012)
Alvarez-Sanchez, J.R.: Injecting knowledge into the solution of the two-spiral problem. Neural Computing and Applications 8, 265–272 (1999)
Wilamowski, B.M., Cotton, N.J., Kaynak, O., Dundar, G.: Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks. IEEE Trans. on Industrial Electronics 55(10), 3784–3790 (2008)
Wilamowski, B.M., Yu, H.: Neural Network Learning without Backpropgation. IEEE Trans. on Neural Networks 21(11), 1793–1803 (2010)
Demuth, H.B., Beale, M.: Neural Network Toolbox: for use with MATLAB: Mathworks Natick, MA, USA (2000)
Yu, H., Xie, T., Paszczynski, S., Wilamowski, B.M.: Advantages of Radial Basis Function Networks for Dynamic System Design. IEEE Trans. on Industrial Electronics 58(12), 5438–5450 (2011)
Yu, H., Reiner, P.D., Xie, T., Bartczak, T., Wilamowski, B.M.: An Incremental Design of Radial Basis Function Networks. IEEE Trans. on Neural Networks and Learning Systems 25 (2014)
Xie, T., Yu, H., Hewlett, J., Rozycki, P., Wilamowski, B.M.: Fast and Efficient Second-Order Method for Training Radial Basis Function Networks. IEEE Trans. on Neural Networks and Learning Systems 23(4), 609–619 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wilamowski, B.M., Korniak, J. (2014). Modeling of Complex Multidimensional Nonlinear Systems Using Neural System with Deep Architectures. In: Fodor, J., Fullér, R. (eds) Advances in Soft Computing, Intelligent Robotics and Control. Topics in Intelligent Engineering and Informatics, vol 8. Springer, Cham. https://doi.org/10.1007/978-3-319-05945-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-05945-7_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05944-0
Online ISBN: 978-3-319-05945-7
eBook Packages: EngineeringEngineering (R0)