W-operator window design by minimization of mean conditional entropy
- 147 Downloads
This paper presents a technique that gives a minimal window W for the estimation of a W-operator from training data. The idea is to choose a subset of variables W that maximizes the information observed in a training set. The task is formalized as a combinatorial optimization problem, where the search space is the powerset of the candidate variables and the measure to be minimized is the mean entropy of the estimated conditional probabilities. As a full exploration of the search space requires prohibitive computational effort, some heuristics of the feature selection literature are applied. The proposed technique is mathematically sound and experimental results including binary image filtering and gray-scale texture recognition show its successful performance in practice.
KeywordsFeature selection Information theory Image restoration Image texture analysis Classification
The authors are grateful to FAPESP (99/12765-2, 01/094 01-0, 04/03967-0 and 05/00587-5), CNPq (300722/98-2, 468 413/00-6, 521097/01-0 474596/04-4 and 491323/05-0) and CAPES for financial support. This work was partially supported by grant 1 D43 TW07015-01 from the National Institutes of Health, USA. We also thank Daniel O. Dantas by his complementing post-processing idea for texture recognition (mode filter applied more than once).
- 1.Barrera J, Terada R, Hirata R Jr, Hirata NST (2000) Automatic programming of morphological machines by pac learning. Fundamenta Informaticae, pp 229–258Google Scholar
- 12.Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, 623–656Google Scholar
- 13.Cover TM, Thomas JA (1991) Elements of information theory. In: Wiley series in telecommunications. Wiley, New YorkGoogle Scholar
- 17.Hall MA, Smith LA (1999) Feature selection for machine learning: comparing a correlation-based filter approach to the wrapper. In: Proceedings of the FLAIRS conference, AAAI Press, pp 235–239Google Scholar
- 18.Lewis DD (1992) Feature selection and feature extraction for text categorization. In: Proceedings of speech and natural language workshop, Morgan Kaufmann, San Mateo, pp 212–217Google Scholar
- 19.Bonnlander BV, Weigend AS (1994) Selecting input variables using mutual information and nonparametric density estimation. In: Proceedings of the 1994 international symposium on artificial neural networks, Tainan, pp 42–50Google Scholar
- 21.Zaffalon M, Hutter M (2002) Robust feature selection by mutual information distributions. In: 18th international conference on uncertainty in artificial intelligence (UAI), pp 577–584Google Scholar
- 22.Costa LF, Cesar RM Jr (2001) Shape analysis and classification: theory and practice. CRC Press, Boca RatonGoogle Scholar
- 24.Vaquero DA, Barrera J, Hirata R Jr (2005) A maximum-likelihood approach for multiresolution W-operator design. In: Proceedings of XVIII Brazilian symposium on computer graphics and image processing (SIBGRAPI). IEEE Computer Society Press, pp 71–78Google Scholar