Abstract
Supervised learning can be thought of as finding a mapping between spaces of input and output vectors. In the case that the function to be learned is multi-valued (so that there are several correct output values for a given input) the problem becomes ill-posed, and many standard methods fail to find good solutions. However, optimisation problems based on multi-valued functions are relatively common. They include reverse robot kinematics, and the research field of AutoML – which is becoming increasingly popular – where one seeks to establish optimal hyperparameters for a learning algorithm for a particular problem based on loss function values for trained networks, or to reuse training from previous networks. We present an analysis of this problem, together with an approach based on k-nearest neighbours, which we demonstrate on a set of simple examples, including two application areas of interest.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brouwer, R.K.: Feed-forward neural network for one-to-many mappings using fuzzy sets. Neurocomputing 57, 345–360 (2004)
Butterfield, J., Osentoski, S., Jay, G., Jenkins, O.C.: Learning from demonstration using a multi-valued function regressor for time-series data. In: 2010 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 328–333. IEEE (2010)
Damas, B., Santos-Victor, J.: Online learning of single-and multivalued functions with an infinite mixture of linear experts. Neural Comput. 25(11), 3044–3091 (2013)
Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 226–231 (1996)
Goldman, S.A., Rivest, R.L., Schapire, R.E.: Learning binary relations and total orders. SIAM J. Comput. 22(5), 1006–1034 (1993)
Hahn, K., Waschulzik, T.: On the use of local RBF networks to approximate multivalued functions and relations. In: Niklasson, L., Bodén, M., Ziemke, T. (eds.) ICANN 1998. PNC, pp. 505–510. Springer, London (1998). https://doi.org/10.1007/978-1-4471-1599-1_75
Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: Neural Networks for Perception, pp. 65–93. Elsevier (1992)
Kilp, M., Knauer, U., Mikhalev, A.: Monoids, Acts and Categories, With Applications to Wreath Products and Graphs. de Gruyter (2000)
Lee, K.W., Lee, T.: Design of neural networks for multi-value regression. In: 2001 Proceedings of International Joint Conference on Neural Networks, IJCNN 2001, vol. 1, pp. 93–98. IEEE (2001)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Shizawa, M.: Multivalued regularization network-a theory of multilayer networks for learning many-to-h mappings. Electron. Commun. Jpn. (Part III: Fundam. Electron. Sci.) 79(9), 98–113 (1996)
Sussmann, H.J.: Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Netw. 5(4), 589–593 (1992)
Tomikawa, Y., Nakayama, K.: Approximating many valued mappings using a recurrent neural network. In: Proceedings of the 1998 IEEE International Joint Conference on Neural Networks, IEEE World Congress on Computational Intelligence, vol. 2, pp. 1494–1497. IEEE (1998)
Wong, C., Houlsby, N., Lu, Y., Gesmundo, A.: Transfer learning with neural AutoML. In: Advances in Neural Information Processing Systems, pp. 8356–8365 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Frochte, J., Marsland, S. (2019). A Learning Approach for Ill-Posed Optimisation Problems. In: Le, T., et al. Data Mining. AusDM 2019. Communications in Computer and Information Science, vol 1127. Springer, Singapore. https://doi.org/10.1007/978-981-15-1699-3_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-1699-3_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1698-6
Online ISBN: 978-981-15-1699-3
eBook Packages: Computer ScienceComputer Science (R0)