Abstract
A learning paradigm is proposed, in which one has both classical supervised examples and constraints that cannot be violated, called here “hard constraints”, such as those enforcing the probabilistic normalization of a density function or imposing coherent decisions of the classifiers acting on different views of the same pattern. In contrast, supervised examples can be violated at the cost of some penalization (quantified by the choice of a suitable loss function) and so play the roles of “soft constraints”. Constrained variational calculus is exploited to derive a representation theorem which provides a description of the “optimal body of the agent”, i.e. the functional structure of the solution to the proposed learning problem. It is shown that the solution can be represented in terms of a set of “support constraints”, thus extending the well-known notion of “support vectors”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adams, R.A., Fournier, J.F.: Sobolev spaces. Academic Press (2003)
Diligenti, M., Gori, M., Maggini, M., Rigutini, L.: Bridging logic and kernel machines. Machine Learning 86, 57–88 (2012), 10.1007/s10994-011-5243-x
Gnecco, G., Gori, M., Melacci, S., Sanguineti, M.: Foundations of support constraint machines. Technical report, DII-UNISI (2013)
Gnecco, G., Gori, M., Sanguineti, M.: Learning with boundary conditions. Neural Computation 25, 1029–1106 (2013)
Gori, M., Melacci, S.: Constraint verification with kernel machines. IEEE Trans. Neural Netw. Learning Syst. 24(5), 825–831 (2013)
Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17(1-3), 185–203 (1981)
Kreyszig, E.: Introductory Functional Analysis with Applications. Wiley & Sons (1989)
Kunapuli, G., Bennett, K.P., Shabbeer, A., Maclin, R., Shavlik, J.: Online knowledge-based support vector machines. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010, Part II. LNCS, vol. 6322, pp. 145–161. Springer, Heidelberg (2010)
Melacci, S., Maggini, M., Gori, M.: Semi–supervised learning with constraints for multi–view object recognition. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part II. LNCS, vol. 5769, pp. 653–662. Springer, Heidelberg (2009)
Poggio, T., Girosi, F.: A theory of networks for approximation and learning. Technical report. MIT (1989)
Tikhonov, A.N., Arsenin, V.Y.: Solution of ill-posed problems. W.H. Winston, Washington, D.C. (1977)
Yuille, A.L., Grzywacz, N.M.: A mathematical analysis of the motion coherence theory. Int. J. of Computer Vision 3, 155–175 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gnecco, G., Gori, M., Melacci, S., Sanguineti, M. (2013). Learning with Hard Constraints. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds) Artificial Neural Networks and Machine Learning – ICANN 2013. ICANN 2013. Lecture Notes in Computer Science, vol 8131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40728-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-40728-4_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40727-7
Online ISBN: 978-3-642-40728-4
eBook Packages: Computer ScienceComputer Science (R0)