IDA 2003: Advances in Intelligent Data Analysis V pp 13-24 | Cite as
Regularized Learning with Flexible Constraints
Abstract
By its very nature, inductive inference performed by machine learning methods is mainly data-driven. Still, the consideration of background knowledge – if available – can help to make inductive inference more efficient and to improve the quality of induced models. Fuzzy set-based modeling techniques provide a convenient tool for making expert knowledge accessible to computational methods. In this paper, we exploit such techniques within the context of the regularization (penalization) framework of inductive learning. The basic idea is to express knowledge about an underlying data-generating model in terms of flexible constraints and to penalize those models violating these constraints. Within this framework, an optimal model is one that achieves an optimal trade-off between fitting the data and satisfying the constraints.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Bernardo, J., Smith, A.: Bayesian Theory. J. Wiley & Sons, Chichester (1994)MATHCrossRefGoogle Scholar
- 2.Bouchon-Meunier, B., Dubois, D., Godo, L., Prade, H.: Fuzzy sets and possibility theory in approximate reasoning and plausible reasoning. In: Bezdek, J.C., Dubois, D., Prade, H. (eds.) Fuzzy Sets in Approximate Reasoning and Information Systems, pp. 15–190. Kluwer, Dordrecht (1999)Google Scholar
- 3.Dubois, D., Fargier, H., Prade, H.: The calculus of fuzzy restrictions as a basis for flexible constraint satisfaction. In: Second IEEE International Conference on Fuzzy Systems, pp. 1131–1136 (1993)Google Scholar
- 4.Dubois, D., Fargier, H., Prade, H.: Propagation and satisfaction of flexible constraints. In: Yager, R.R., Zadeh, L. (eds.) Fuzzy Sets, Neural Networks, and Soft Computing, pp. 166–187. Van Nostrand Reinhold, New York (1994)Google Scholar
- 5.Dubois, D., Hüllermeier, E., Prade, H.: Fuzzy set-based methods in instancebased reasoning. IEEE Transactions on Fuzzy Systems 10(3), 322–332 (2002)CrossRefGoogle Scholar
- 6.Dubois, D., Prade, H.: The three semantics of fuzzy sets. Fuzzy Sets and Systems 90(2), 141–150 (1997)MathSciNetMATHCrossRefGoogle Scholar
- 7.Guesgen, H.W.: A formal framework for weak constraint satisfaction based on fuzzy sets. Technical Report TR-94-026, ICSI Berkeley (June 1994)Google Scholar
- 8.Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(3), 55–67 (1970)MATHCrossRefGoogle Scholar
- 9.Mitchell, T.M.: Machine Learning. McGraw-Hill, Boston (1997)MATHGoogle Scholar
- 10.Rissanen, J.: Modeling by shortest data description. Automatica 14, 465–471 (1978)MATHCrossRefGoogle Scholar
- 11.Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)MATHGoogle Scholar
- 12.Zadeh, L.A.: Fuzzy sets. Information and Control 8, 338–353 (1965)MathSciNetMATHCrossRefGoogle Scholar