Skip to main content
SpringerLink
Log in

Optimal Training of Feedforward Neural Networks Using Teaching-Learning-Based Optimization with Modified Learning Phases

  • Conference paper
  • First Online:
Proceedings of the 12th National Technical Seminar on Unmanned System Technology 2020

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 770))

Abstract

Feedforward neural network (FNN) emerges as a promising machine learning algorithm widely used for crucial tasks such as prediction, classification, clustering, regression, and etc. due to its excellent performance in training, learning and organizing data. Conventional approaches such as gradient-based algorithm is commonly used to train FNNs, but it has drawbacks of being trapped into local optima easily, slow convergence rate and high sensitivity to initial solutions generated. Metaheuristic search algorithm such as teaching-learning-based optimization (TLBO) is envisioned as a potential solution to train FNNs due to their stochastic nature and excellent global search ability. In this paper, a new TLBO variant named as teaching-learning-based optimization with modified learning phases (TLBO-MLPs) is designed as the training algorithm of FNN, aiming to optimize its neuron weights, biases and selection of activation functions by referring to the datasets of given classification problems. The classification performances of FNN trained by TLBO-MLPs are evaluated by using the real-life classification datasets from UCI Machine Learning Repository and compared with those optimized by the other well-established TLBO variants. Rigorous simulation studies show that the FNN trained by TLBO-MLPs outperforms most of its peer algorithms significantly in solving different classification problems. Particularly, the FNN classifiers optimized by proposed TLBO-MLPs are reported to outperform its competitors up to 15.72% and 29.07% in terms of classification accuracy for both training and testing datasets, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Carvalho M, Ludermir TB (2007) Particle swarm optimization of neural network architectures and weights. In: 7th international conference on hybrid intelligent systems (HIS 2007). IEEE, pp 336–339

    Google Scholar 

  2. Rodriguez P, Wiles J, Elman JL (1999) A recurrent neural network that learns to count. Connection Sci 11:5–40

    Article  Google Scholar 

  3. Sumachev A, Kuzmin V, Borodin E (2018) River flow forecasting using artificial neural networks. Int J Mech Eng Technol 9:706–714

    Google Scholar 

  4. Sainath TN, Mohamed A-R, Kingsbury B, Ramabhadran B (2013) Deep convolutional neural networks for LVCSR. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 8614–8618

    Google Scholar 

  5. Syulistyo AR, Purnomo DMJ, Rachmadi MF, Wibowo A (2016) Particle swarm optimization (PSO) for training optimization on convolutional neural network (CNN). Jurnal Ilmu Komputer dan Informasi 9:52–58

    Article  Google Scholar 

  6. Vt SE, Shin YC (1994) Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems. IEEE Trans Neural Netw 5:594–603

    Article  Google Scholar 

  7. Wu H, Zhou Y, Luo Q, Basset MA (2016) Training feedforward neural networks using symbiotic organisms search algorithm. In: Computational intelligence and neuroscience 2016

    Google Scholar 

  8. Feng J, Lu S (2019) Performance analysis of various activation functions in artificial neural networks. J Phys Conf Ser 022030

    Google Scholar 

  9. Mirjalili S, Hashim SZM, Sardroudi HM (2012) Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218:11125–11137

    MathSciNet  MATH  Google Scholar 

  10. Tarkhaneh O, Shen H (2019) Training of feedforward neural networks for data classification using hybrid particle swarm optimization, Mantegna Lévy flight and neighborhood search. Heliyon 5:e01275

    Google Scholar 

  11. Ang KM, Lim WH, Isa NAM, Tiang SS, Wong CH (2020) A constrained multi-swarm particle swarm optimization without velocity for constrained optimization problems. Expert Syst Appl 140:112882

    Google Scholar 

  12. Al-Sanabani DGA, Solihin MI, Pui LP, Astuti W, Ang CK, Hong LW (2019) Development of non-destructive mango assessment using handheld spectroscopy and machine learning regression. J Phys Conf Ser 012030

    Google Scholar 

  13. Yao L, Shen J-Y, Lim WH (2016) Real-time energy management optimization for smart household. In: 2016 IEEE international conference on internet of things (iThings) and IEEE green computing and communications (GreenCom) and IEEE cyber, physical and social computing (CPSCom) and IEEE smart data (SmartData). IEEE, pp 20–26

    Google Scholar 

  14. Yao L, Damiran Z, Lim WH (2017) Energy management optimization scheme for smart home considering different types of appliances. In: 2017 IEEE international conference on environment and electrical engineering and 2017 IEEE industrial and commercial power systems Europe (EEEIC/I&CPS Europe). IEEE, pp 1–6

    Google Scholar 

  15. Yao L, Lim WH (2017) Optimal purchase strategy for demand bidding. IEEE Trans Power Syst 33:2754–2762

    Article  Google Scholar 

  16. Yao L, Lim WH, Tiang SS, Tan TH, Wong CH, Pang JY (2018) Demand bidding optimization for an aggregator with a genetic algorithm. Energies 11:2498

    Article  Google Scholar 

  17. Yao L, Yao L, Lim WH (2018) A soft curtailment of wide-area central air conditioning load. Energies 11:492

    Article  Google Scholar 

  18. Yao L, Chen Y-Q, Lim WH (2015) Internet of things for electric vehicle: an improved decentralized charging scheme. In: 2015 IEEE international conference on data science and data intensive systems. IEEE, pp 651–658

    Google Scholar 

  19. Natarajan E, Kaviarasan V, Lim WH, Tiang SS, Parasuraman S, Elango S (2019) Non-dominated sorting modified teaching–learning-based optimization for multi-objective machining of polytetrafluoroethylene (PTFE). J Intell Manuf 1–25

    Google Scholar 

  20. Yu L, Tarawni MA, Ahmad SH, Al-Banawi O, Batiha MA (2015) High performance thermoplastic elastomer (TPE) nanocomposite based on graphene nanoplates (GNPs). World J Eng 12(5):437–442

    Google Scholar 

  21. Yu LJ, Sahrim AH, Kong I, Mouad AT (2012) Microwave absorbing properties of nickel-zinc ferrite/multiwalled nanotube thermoplastic natural rubber composites. In: Advanced materials research. Trans Tech Publ, pp 24–28

    Google Scholar 

  22. Yu L-J, Ahmad SH, Kong I, Tarawneh MA, Flaifel MH (2013) Preparation and characterisation of NiZn ferrite/multiwalled nanotubes thermoplastic natural rubber composite. Int J Mater Eng Innovation 2(4):214–224

    Article  Google Scholar 

  23. Yu L-J, Ahmad SH, Kong I, Appadu S (2012) Sifat magnet, mikrostruktur dan morfologi komposit getah asli termoplastik berpengisi ferit NiZn/Mwnt. Sains Malaysiana 41:453–458

    Google Scholar 

  24. Chong OT, Lim WH, Isa NAM, Ang KM, Tiang SS, Ang CK (2020) A teaching-learning-based optimization with modified learning phases for continuous optimization. In: Science and information conference. Springer, Berlin, pp 103–124

    Google Scholar 

  25. Ang KM, Lim WH, Isa NAM, Tiang SS, Ang CK, Natarajan E, Solihin MI (2020) A constrained teaching-learning-based optimization with modified learning phases for constrained optimization. J Adv Res Dyn Control Syst 12:15

    Google Scholar 

  26. Kiranyaz S, Ince T, Yildirim A, Gabbouj M (2009) Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Netw 22:1448–1462

    Article  Google Scholar 

  27. Rao RV, Savsani VJ, Vakharia DP (2011) Teaching-learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43:303–315

    Article  Google Scholar 

  28. Yu K, Wang X, Wang Z (2016) Constrained optimization based on improved teaching–learning-based optimization algorithm. Inf Sci 352:61–78

    Article  Google Scholar 

  29. Savsani VJ, Tejani GG, Patel VK (2016) Truss topology optimization with static and dynamic constraints using modified subpopulation teaching–learning-based optimization. Eng Optim 48:1990–2006

    Article  MathSciNet  Google Scholar 

  30. Zheng H, Wang L, Zheng X (2017) Teaching-learning-based optimization algorithm for multi-skill resource constrained project scheduling problem. Soft Comput 21:12

    Google Scholar 

  31. Benali A, Hachama M, Bounif A, Nechnech A, Karray M (2019) A TLBO-optimized artificial neural network for modeling axial capacity of pile foundations. Eng Comput 1–10

    Google Scholar 

  32. Wang L, Zou F, Hei X, Yang D, Chen D, Jiang Q (2014) An improved teaching-learning-based optimization with neighborhood search for applications of ANN. Neurocomputing 143:231–247

    Article  Google Scholar 

  33. Li K, Xie X, Xue W, Dai X, Chen X, Yang X (2018) A hybrid teaching-learning artificial neural network for building electrical energy consumption prediction. Energy Build 174:323–334

    Article  Google Scholar 

  34. Zou F, Chen D, Xu Q (2019) A survey of teaching–learning-based optimization. Neurocomputing 335:366–383

    Article  Google Scholar 

  35. Akhtar J, Koshul BB, Awais MM (2013) A framework for evolutionary algorithms based on Charles Sanders Peirce’s evolutionary semiotics. Inf Sci 236:93–108

    Article  MathSciNet  Google Scholar 

  36. Shao W, Pi D, Shao Z (2017) An extended teaching-learning based optimization algorithm for solving no-wait flow shop scheduling problem. Appl Soft Comput 61:193–210

    Article  Google Scholar 

  37. Tizhoosh HR (2005) Reinforcement learning based on actions and opposite actions. In: International conference on artificial intelligence and machine learning

    Google Scholar 

  38. Roy PK, Sur A, Pradhan DK (2013) Optimal short-term hydro-thermal scheduling using quasi-oppositional teaching learning based optimization. Eng Appl Artif Intell 26:2516–2524

    Article  Google Scholar 

  39. Martín García JA, Gil Mena AJ (2013) Optimal distributed generation location and size using a modified teaching–learning based optimization algorithm. Int J Electr Power Energy Syst 50:65–75

    Article  Google Scholar 

  40. Ouyang H-B, Gao L-Q, Kong X-Y, Zou D-X, Li S (2015) Teaching-learning based optimization with global crossover for global optimization problems. Appl Math Comput 265:533–556

    MathSciNet  MATH  Google Scholar 

  41. Cao J, Luo J (2015) A study on SVM based on the weighted elitist teaching-learning-based optimization and application in the fault diagnosis of chemical process. In: MATEC web of conferences. EDP Sciences, p 05016

    Google Scholar 

  42. Satapathy SC, Naik A, Parvathi K (2013) Weighted teaching-learning-based optimization for global function optimization. Appl Math 4:429

    Article  Google Scholar 

  43. Wu Z-S, Fu W-P, Xue R (2015) Nonlinear inertia weighted teaching-learning-based optimization for solving global optimization problem. Intell Neurosci 2015:87–87

    Google Scholar 

  44. Kandasamy T, Rajendran R (2018) Hybrid algorithm with variants for feed forward neural network. Int Arab J Inf Technol 15:240–245

    Google Scholar 

  45. Uzlu E, Kankal M, Akpınar A, Dede T (2014) Estimates of energy consumption in Turkey using neural networks with the teaching–learning-based optimization algorithm. Energy 75:295–303

    Article  Google Scholar 

  46. Lin G, Shen W (2018) Research on convolutional neural network based on improved Relu piecewise activation function. Procedia Comput Sci 131:977–984

    Article  Google Scholar 

  47. Xue Y, Tang T, Liu AX (2019) Large-scale feedforward neural network optimization by a self-adaptive strategy and parameter based particle swarm optimization. IEEE Access 7:52473–52483

    Article  Google Scholar 

  48. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1:3–18

    Article  Google Scholar 

  49. García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15:617

    Article  Google Scholar 

  50. Rao RV, Patel V (2013) An improved teaching-learning-based optimization algorithm for solving unconstrained optimization problems. Scientia Iranica 20:710–720

    Google Scholar 

  51. Zou F, Chen D, Wang J (2016) An improved teaching-learning-based optimization with the social character of PSO for global optimization. Comput Intell Neurosci 2016

    Google Scholar 

  52. Mahmoodabadi M, Ostadzadeh R (2019) CTLBO: Converged teaching-learning-based optimization. Cogent Eng 6:1654207

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Technology and Built Environment, UCSI University, 56000, Kuala Lumpur, Malaysia

    Koon Meng Ang, Wei Hong Lim, Sew Sun Tiang, Chun Kit Ang, Elango Natarajan & M. K. A. Ahamed Khan

Authors
  1. Koon Meng Ang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Hong Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sew Sun Tiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chun Kit Ang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elango Natarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. K. A. Ahamed Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Hong Lim .

Editor information

Editors and Affiliations

  1. Faculty of Electrical and Electronic Engineering, Universiti Tun Hussein Onn Malaysia, Parit Raja, Johor, Malaysia

    Khalid Isa

  2. Faculty of Electrical and Electronic Engineering, Universiti Malaysia Pahang, Pekan, Pahang, Malaysia

    Zainah Md. Zain

  3. School of Electrical and Electronic Engineering, Universiti Sains Malaysia, Engineering Campus, Nibong Tebal, Pulau Pinang, Malaysia

    Rosmiwati Mohd-Mokhtar

  4. Faculty of Electrical and Electronic Engineering, Universiti Malaysia Pahang, Pekan, Pahang, Malaysia

    Maziyah Mat Noh

  5. Malaysia-Japan International Institute of Technology (MJIIT), Universiti Teknologi Malaysia, Jalan Semarak, Kuala Lumpur, Malaysia

    Zool H. Ismail

  6. Faculty of Electrical Engineering, Universiti Teknikal Malaysia Melaka, Durian Tunggal, Melaka, Malaysia

    Ahmad Anas Yusof

  7. Faculty of Ocean Engineering Technology and Informatics (FTKKI), Universiti Malaysia Terengganu, Kuala Nerus, Terengganu, Malaysia

    Ahmad Faisal Mohamad Ayob

  8. Department of Electrical and Electronic Engineering, Universiti Teknologi Petronas, Bandar Seri Iskandar, Perak, Malaysia

    Syed Saad Azhar Ali

  9. Faculty of Electrical and Electronic Engineering, Universiti Tun Hussein Onn, Parit Raja, Johor, Malaysia

    Herdawatie Abdul Kadir

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ang, K.M., Lim, W.H., Tiang, S.S., Ang, C.K., Natarajan, E., Ahamed Khan, M.K.A. (2022). Optimal Training of Feedforward Neural Networks Using Teaching-Learning-Based Optimization with Modified Learning Phases. In: Isa, K., et al. Proceedings of the 12th National Technical Seminar on Unmanned System Technology 2020. Lecture Notes in Electrical Engineering, vol 770. Springer, Singapore. https://doi.org/10.1007/978-981-16-2406-3_65

Download citation

Publish with us

Policies and ethics

Search

Navigation