Abstract
This chapter discusses general issues in the numerical solution of machine learning control problems. As parametric machine learning approach, the most popular and widespread apparatus of neural networks is considered. Theoretical substantiations are given for the general possibility of using machine learning methods for searching functions, namely the Kolmogorov–Arnold theorem. The only general approach of structural-parametric search of functions based on the methods of symbolic regression is presented. To overcome computational difficulties, it is proposed to use the principle of small variations. A description of the genetic algorithm is given as the main search mechanism in the space of structures, and in addition, it can also be used to adjust the parameters of a given structure of a function in parametric search.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cybenko, G.V.: Approximation by Superpositions of sigmoidal function, Mathematics of Control, Signals, and Systems, Vol. 5, No. 4, 1989, pp. 303–314. https://dx.doi.org/10.1007/BF02551274
Mitchell, T.: Machine Learning. McGraw-Hill Science/Engineering/Math. McGraw-Hill, New York (1997)
Han, J., Morag, C.: The influence of the sigmoid function parameters on the speed of backpropagation learning. In: Mira, J., Sandoval, F. (eds.) From Natural to Artificial Neural Computation. Lecture Notes in Computer Science, pp. 195–201. Springer, Berlin (1995)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: 27th International Conference on Machine Learning, pp. 807–814. Omnipress (2010)
Cruse, H.: Neural Networks as Cybernetic Systems. Brains, Minds & Media, Bielefeld, Germany (2006)
Hopfield, J.J., Tank, D.W.: Neural computation of decisions in optimization problems. Biolog. Cybern. 55, 141–146 (1985)
Kolmogorov, A.: On the representation of continuous functions of several variables by superpositions of continuous functions of a smaller number of variables. Am. Math. Soc. Transl. 17, 369–373 (1961)
Arnold, V.: On functions of three variables. Am. Math. Soc. Transl. 28, 51–54 (1963)
Hecht-Nielsen, R.: Kolmogorov’s mapping neural network existence theorem. In: Proceedings of the IEEE First International Conference on Neural Networks, San Diego, vol. III, pp. 11–13. IEEE, Piscataway (1987)
Holland, J.: Adaptation in Natural and Artificial Systems. MIT Press, Cambridge, MA (1992)
Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge, MA (1996)
Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Professional, Reading (1989)
Vose, M.: The Simple Genetic Algorithm: Foundations and Theory. MIT Press, Cambridge, MA (1999)
Kumar, M., Husian, M., Upreti, N., Gupta, D.: Genetic Algorithm: Review and Application (2010)
Federal Standard 1037C (1996). https://www.its.bldrdoc.gov/fs-1037/fs-1037c.htm
Diveev, A.I.: Small variations of basic solution method for non-numerical optimization. IFAC-PapersOnLine 48(25), 028–033 (2015)
Author information
Authors and Affiliations
Appendix
Appendix
Here is the description of the most often used machine-made functions presented in the form of free Pascal programming code, where notation Ro−N is used for functions with one argument, Xi−N is used for functions with two arguments, and Nu−N is used for functions with three arguments, where N is the number of functions.
Unit Machine-Made Functions
const
infinity=1e8;
eps=1e-8;
//*******************************
Function Ro−1 (z: real): real;
Begin
result:=z;
End;
//*******************************
Function Ro−2(z:real):real;
Begin
if abs(z)> sqrt(infinity)
then result:=infinity
else result:=sqr(z);
End;
//*******************************
Function Ro−3(z:real):real;
Begin
result:=-z;
End;
//*******************************
Function Ro−4(z:real):real;
Begin
result:=Ro−10(z)*sqrt(abs(z));
End;
//*******************************
Function Ro−5(z:real):real;
Begin
if abs(z)> eps
then result:=1/z
else result:=Ro−10(z)/eps;
End;
//*******************************
Function Ro−6(z:real):real;
Begin
if z> -ln(eps)
then result:=-ln(eps)
else result:=exp(z);
End;
//*******************************
Function Ro−7(z:real):real;
Begin
if abs(z)< exp(-pokmax)
then result:=ln(eps)
else result:=ln(abs(z));
End;
//*******************************
Function Ro−8(z:real):real;
Begin
if abs(z)> -ln(eps)
then result:=Ro−10(z)
else result:=(1-exp(-z))/(1+exp(-z));
End;
//*******************************
Function Ro−9(z:real):real;
Begin
if z> =0
then result:=1
else result:=0;
End;
//*******************************
Function Ro−10(z:real):real;
Begin
if z> =0
then result:=1
else result:=-1;
End;
//*******************************
Function Ro−11(z:real):real;
Begin
result:=cos(z);
End;
//*******************************
Function Ro−12(z:real):real;
Begin
result:=sin(z);
End;
//*******************************
Function Ro−13(z:real):real;
Begin
if abs(z)< eps
then result:=Ro−10(z)*pi/2
else result:=arctan(z);
End;
//*******************************
Function Ro−14(z:real):real;
Begin
if abs(z)> Ro−15(infinity)
then result:=Ro−10(z)*infinity
else result:=sqr(z)*z;
End;
//*******************************
Function Ro−15(z:real):real;
Begin
if abs(z)< eps
then result:=Ro−10(z)*eps
else result:=Ro−10(z)*exp(ln(abs(z))/3);
End;
//*******************************
Function Ro−16(z:real):real;
Begin
if abs(z)< 1
then result:=z
else result:=Ro−10(z);
End;
//*******************************
Function Ro−17(z:real):real;
Begin
result:=Ro−10(z)*ln(abs(z)+1);
End;
//*******************************
Function Ro−18(z:real):real;
Begin
if abs(z)> -ln(eps)
then result:=Ro−10(z)*infinity
else result:=Ro−10(z)*(exp(abs(z))-1);
End;
//*******************************
Function Ro−19(z:real):real;
Begin
if abs(z)> 1/eps
then result:=Ro−10(z)*eps
else result:=Ro−10(z)*exp(-abs(z));
End;
//*******************************
Function Ro−20(z:real):real;
Begin
Result:=z/2;
End;
//*******************************
Function Ro−21(z:real):real;
Begin
Result:=2*z;
End;
//*******************************
Function Ro−22(z:real):real;
Begin
if z< 0
then Result:=exp(z)-1
else Result:=1-exp(-abs(z));
End;
//*******************************
Function Ro−23(z:real):real;
Begin
if abs(z)> 1/eps
then result:=-Ro−10(z)/eps
else result:=z-z*sqr(z);
End;
//*******************************
Function Ro−24(z:real):real;
Begin
if z> infinity
then result:=1
else
if exp(-z)> infinity
then result:=0
else result:=1/(1+exp(-z));
End;
//*******************************
Function Ro−25(z:real):real;
Begin
if z> 0
then result:=1
else result:=0;
End;
//*******************************
Function Ro−26(z:real):real;
Begin
if abs(z)< eps1
then result:=0
else result:=Ro−10(z);
End;
//*******************************
Function Ro−27(z:real):real;
Begin
if abs(z)> 1
then result:=Ro−10(z)
else result:=Ro−10(z)*(1-sqrt(1-sqr(z)));
End;
//*******************************
Function Ro−28(z:real):real;
Begin
if z*z> ln(infinity)
then result:=z*(1-eps)
else result:=z*(1-exp(-sqr(z)));
End;
//*******************************
Function Xi−1(z1,z2:real):real;
Begin
result:=z1+z2;
End;
//*******************************
Function Xi−2(z1,z2:real):real;
Begin
if abs(z1*z2)> infinity
then result:=Ro−10(z1*z2)*infinity
else result:=z1*z2;
End;
//*******************************
Function Xi−3(z1,z2:real):real;
Begin
if z1> =z2
then result:=z1
else result:=z2;
End;
//*******************************
Function Xi−4(z1,z2:real):real;
Begin
if z1< z2
then result:=z1
else result:=z2;
End;
//*******************************
Function Xi−5(z1,z2:real):real;
Begin
result:=z1+z2-z1*z2;
End;
//*******************************
Function Xi−6(z1,z2:real):real;
Begin
result:=Ro−10(z1+z2)*sqrt(sqr(z1)+sqr(z2));
End;
//*******************************
Function Xi−7(z1,z2:real):real;
Begin
result:=Ro−10(z1+z2)*(abs(z1)+abs(z2));
End;
//*******************************
Function Xi−8(z1,z2:real):real;
Begin
result:=Ro−10(z1+z2)*Xi−2(abs(z1),abs(z2));
End;
//*******************************
Function Nu−1(z1,z2,z3:real):real;
Begin
if z1> 0
then result:=z2
else result:=z3;
End;
//*******************************
Function Nu−2(z1,z2,z3:real):real;
Begin
if z1> z2
then result:=z3
else result:=-z3;
End;
//*******************************
Function Nu−3(z1,z2,z3:real):real;
Begin
if z1> 0
then result:=z2+z3
else result:=z2-z3;
End;
//*******************************
Function Nu−4(z1,z2,z3:real):real;
Begin
if z1> z2
then
if z1> z3
then result:=z1
else result:=z3
else
if z2> z3
then result:=z2
else result:=z3;
End.
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Diveev, A., Shmalko, E. (2021). Numerical Solution of Machine Learning Control Problems. In: Machine Learning Control by Symbolic Regression. Springer, Cham. https://doi.org/10.1007/978-3-030-83213-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-83213-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-83212-4
Online ISBN: 978-3-030-83213-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)