Pattern Analysis and Applications

, Volume 9, Issue 2–3, pp 139–153 | Cite as

W-operator window design by minimization of mean conditional entropy

  • David C. MartinsJrEmail author
  • Roberto M. CesarJr
  • Junior Barrera
Theoretical Advances


This paper presents a technique that gives a minimal window W for the estimation of a W-operator from training data. The idea is to choose a subset of variables W that maximizes the information observed in a training set. The task is formalized as a combinatorial optimization problem, where the search space is the powerset of the candidate variables and the measure to be minimized is the mean entropy of the estimated conditional probabilities. As a full exploration of the search space requires prohibitive computational effort, some heuristics of the feature selection literature are applied. The proposed technique is mathematically sound and experimental results including binary image filtering and gray-scale texture recognition show its successful performance in practice.


Feature selection Information theory Image restoration Image texture analysis Classification 



The authors are grateful to FAPESP (99/12765-2, 01/094 01-0, 04/03967-0 and 05/00587-5), CNPq (300722/98-2, 468 413/00-6, 521097/01-0 474596/04-4 and 491323/05-0) and CAPES for financial support. This work was partially supported by grant 1 D43 TW07015-01 from the National Institutes of Health, USA. We also thank Daniel O. Dantas by his complementing post-processing idea for texture recognition (mode filter applied more than once).


  1. 1.
    Barrera J, Terada R, Hirata R Jr, Hirata NST (2000) Automatic programming of morphological machines by pac learning. Fundamenta Informaticae, pp 229–258Google Scholar
  2. 2.
    Di Ruberto C, Dempster A, Khan S, Jarra B (2002) Analysis of infected blood cell images using morphological operators. Image Vis Comput 20(2):133–146CrossRefGoogle Scholar
  3. 3.
    Dougherty ER (2004) Automatic design of morphological operators. J Electron Imaging 13(3):486–491CrossRefGoogle Scholar
  4. 4.
    Green AC, Marshall S, Greenhalgh D, Dougherty ER (2003) Design of multi-mask aperture filters. Signal Processing 83(9):1961–1971CrossRefGoogle Scholar
  5. 5.
    Pitas I, Venetsanopoulos AN (1990) Nonlinear digital filters. Kluwer, DordrechtzbMATHGoogle Scholar
  6. 6.
    Dougherty ER, Barrera J (2002) Pattern recognition theory in nonlinear signal processing. J Math Imaging Vis 16(3):181–197zbMATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Hirata NST, Dougherty ER, Barrera J (2000) A switching algorithm for design of optimal increasing binary filters over large windows. Pattern Recognit 33:1059–1081CrossRefGoogle Scholar
  8. 8.
    Soille P (2002) On morphological operators based on rank filters. Pattern Recognit 35(2):527–535zbMATHCrossRefGoogle Scholar
  9. 9.
    Jain A, Zongker D (1997) Feature selection—evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 19(2):153–158CrossRefGoogle Scholar
  10. 10.
    Theodoridis S, Koutroumbas K (1999) Pattern recognition, 1st edn. Academic, New YorkzbMATHGoogle Scholar
  11. 11.
    Pudil P, Novovicová J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125CrossRefGoogle Scholar
  12. 12.
    Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, 623–656Google Scholar
  13. 13.
    Cover TM, Thomas JA (1991) Elements of information theory. In: Wiley series in telecommunications. Wiley, New YorkGoogle Scholar
  14. 14.
    Kullback S (1968) Information theory and statistics. Dover, New YorkzbMATHGoogle Scholar
  15. 15.
    Soofi ES (2000) Principal information theoretic approaches. J Am Stat Assoc 95:1349–1353zbMATHMathSciNetCrossRefGoogle Scholar
  16. 16.
    Duda RO, Hart PE, Stork D (2000) Pattern classification. Wiley, New YorkzbMATHGoogle Scholar
  17. 17.
    Hall MA, Smith LA (1999) Feature selection for machine learning: comparing a correlation-based filter approach to the wrapper. In: Proceedings of the FLAIRS conference, AAAI Press, pp 235–239Google Scholar
  18. 18.
    Lewis DD (1992) Feature selection and feature extraction for text categorization. In: Proceedings of speech and natural language workshop, Morgan Kaufmann, San Mateo, pp 212–217Google Scholar
  19. 19.
    Bonnlander BV, Weigend AS (1994) Selecting input variables using mutual information and nonparametric density estimation. In: Proceedings of the 1994 international symposium on artificial neural networks, Tainan, pp 42–50Google Scholar
  20. 20.
    Viola P, Wells WM III (1997) Alignment by maximization of mutual information. Int J Comput Vision 24(2):137–154CrossRefGoogle Scholar
  21. 21.
    Zaffalon M, Hutter M (2002) Robust feature selection by mutual information distributions. In: 18th international conference on uncertainty in artificial intelligence (UAI), pp 577–584Google Scholar
  22. 22.
    Costa LF, Cesar RM Jr (2001) Shape analysis and classification: theory and practice. CRC Press, Boca RatonGoogle Scholar
  23. 23.
    Dougherty ER, Kim S, Chen Y (2000) Coefficient of determination in nonlinear signal processing. Signal Processing 80:2219–2235zbMATHCrossRefGoogle Scholar
  24. 24.
    Vaquero DA, Barrera J, Hirata R Jr (2005) A maximum-likelihood approach for multiresolution W-operator design. In: Proceedings of XVIII Brazilian symposium on computer graphics and image processing (SIBGRAPI). IEEE Computer Society Press, pp 71–78Google Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • David C. MartinsJr
    • 1
    Email author
  • Roberto M. CesarJr
    • 1
  • Junior Barrera
    • 1
  1. 1.Departamento de Ciência da Computação, Instituto de Matemática e EstatísticaUniversidade de São Paulo, Rua do MatãoSão PauloBrasil

Personalised recommendations