Non Linear Fitting Methods for Machine Learning

Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 13)


This manuscript presents an analysis of numerical fitting methods used for solving classification problems as discriminant functions in machine learning. Non linear polynomial, exponential, and trigonometric models are mathematically deduced and discussed. Analysis about their pros and cons, and their mathematical modelling are made on what method to chose for what type of highly non linear multi-dimension problems are more suitable to be solved. In this study only deterministic models with analytic solutions are involved, or parameters calculation by numeric methods, which the complete model can subsequently be treated as a theoretical model. Models deduction are summarised and presented as a survey.


  1. 1.
    Jabbar, H.K., Khan, R.Z.: Methods to avoid over-fitting and under-fitting in supervised machine learning (comparative study). Comp. Sci., Comm. Instr. Dev. 163–172 (2015)Google Scholar
  2. 2.
    Tetko, I.V., Livingstone, D.J., Luik, A.I.: Neural network studies. 1. Comparison of overfitting and overtraining. J. Chem. Inf. Comput. Sci. 35, 826–833 (1995)CrossRefGoogle Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2001)zbMATHGoogle Scholar
  4. 4.
    Riley, K.F., Hobson, M.P., Bence, S.J.: Mathematical Methods for Physics and Engineering, 3rd edn. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
  5. 5.
    Cheney, W., Kincais, D.: Numerical Mathematics and Computing, 6th edn. Cengage, Boston (2011)Google Scholar
  6. 6.
    Kreyszig, E.: Advanced Engineering Mathematics, 3rd edn. Limusa Wiley, Mexico City (2009)zbMATHGoogle Scholar
  7. 7.
    Van, D.A., Rubin, W.M., Rubin, V., Verbeek, H.M.W., Van Dongen, B.F., Kindler, E., Gunther, C.W.: Process mining: a two-step approach to balance between underfitting and overfitting. Softw. Syst. Model. 9(1), 87–111 (2010)CrossRefGoogle Scholar
  8. 8.
    Loughrey, J., Cunningham, P.: Using early-stopping to avoid overfitting in wrapper- based feature selection employing stochastic search (2005)Google Scholar
  9. 9.
    Schaffer, C.: Overfitting avoidance as bias. Mach. Learn. 10(2), 153–178 (1993)Google Scholar
  10. 10.
    Ruppert, D., Carroll, R.J.: Spatially-adaptive penalties for spline fitting. J. Statist. 42, 205–224 (2000)Google Scholar
  11. 11.
    Lawrence, S., Giles, C.L., Tsoi, A.C.: Lessons in neural network training: overfitting may be harder than expected. In: Proceedings of the 14th National Conference on AI, USA, pp. 540–545 (1997)Google Scholar
  12. 12.
    Raskutti, G., Wainwright, M.J., Yu, B.: Early stopping and non-parametric regression: an optimal data-dependent stopping rule. J. Mach. Learn. Res. 15(1), 335–366 (2014)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Bishop, C.M.: Neural Networks for Patter Recognition. Oxford University Press, Oxford (2005)Google Scholar
  14. 14.
    Freeman, J.A., Skapura, D.M.: Neural Networks, Algorithms, Applications and Programming Techniques. Computation and Neural Systems Series. Addison-Wesley, Reading (2002)zbMATHGoogle Scholar
  15. 15.
    Wen, U.P., Lan, K.M., Shih, H.S.: A review of Hopfield neural networks for solving mathematical programming problems. Eur. J. Oper. Res. 198(3), 675–687 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Caruana, R., Lawrence, S., Giles, L.: Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Advances in Neural Information Processing Systems, pp. 402–408 (2011)Google Scholar
  17. 17.
    Kazushi, M.: Avoiding overfitting in multilayer perceptrons with feeling-of-knowing using self-organizing maps. Biosystems 80(1), 37–40 (2005)CrossRefGoogle Scholar
  18. 18.
    Gaurang, P., Amit, G., Parth, S., Devyani, P.: Determination Of over-learning and over-fitting problem in back propagation neural network. Int. J. Soft Comput. 2(2), 40–51 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Universidad Autónoma de Ciudad JuárezCiudad JuárezMexico
  2. 2.University of Texas at El PasoEl PasoUSA
  3. 3.Opole UniversityOpolePoland
  4. 4.Singapore PolytechnicSingaporeSingapore
  5. 5.Singapore University of Technology and DesignSingaporeSingapore
  6. 6.Kazan Federal UniversityKazanRussian Federation

Personalised recommendations