Design Choices and Theoretical Issues for Relative Feature Importance, a Metric for Nonparametric Discriminatory Power

  • Hilary J. Holz
  • Murray H. Loew
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)

Abstract

We have developed relative feature importance (RFI), a metric for the classifier-independent ranking of features. Previously, we have shown the metric to rank accurately features for a wide variety of artificial and natural problems, for both two-class and multi-class problems. In this paper, we present the design of the metric, including both theoretical considerations and statistical analysis of the possible components.

Keywords

discriminatory power feature selection feature extraction feature analysis non-parametric classifier-independent relative feature importance multi-class 

References

  1. 1.
    I. Chang and M.H. Loew, “Pattern Recognition with New Class Discovery,” Proceedings of the IEEE Computer Society Conference on Pattern Recognition and Computer Vision, pp. 438–443., 1991.Google Scholar
  2. 2.
    M. Ben-Bassat, “f-Entropies, Probability of Error, and Feature Selection,” Information and Control, vol. 39, pp. 227–242, 1978.MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    T.M. Cover, “The Best Two Independent Measurements are Not the Two Best,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 4, no. 1, pp. 116–117, Jan. 1974.MATHGoogle Scholar
  4. 4.
    K. Fukunaga and J.M. Mantock, “Nonparametric Discriminant Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 5, no. 6, pp. 671–678, Nov. 1983.MATHCrossRefGoogle Scholar
  5. 5.
    G.F. Hughes, “On the mean accuracy of statistical pattern recognizers,” IEEE Transactions on Information Theory, vol. IT-14, pp. 55–63, Jan. 1968.Google Scholar
  6. 6.
    R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis, New York: Wiley & Sons, 1973.MATHGoogle Scholar
  7. 7.
    R.D. Short and K. Fukunaga, “The Optimal Distance Measure for Nearest Neighbor Classification,” IEEE Transactions on Information Theory, vol. IT-27, no. 5, pp. 622–627, Sept. 1981.CrossRefMathSciNetGoogle Scholar
  8. 8.
    H.J. Holz and M.H. Loew, “Multi-class classifier-independent feature analysis,” Pattern Recognition Letters, vol. 18, no. 11–13, pp. 1219–1224, Nov. 1997.CrossRefGoogle Scholar
  9. 9.
    K. Fukunaga, Introduction to Statistical Pattern Recognition: 2nd Edition, Academic Press, Inc. 1990.Google Scholar
  10. 10.
    T.M. Cover and P.E. Hart, “Nearest neighbor pattern classification,” IEEE Transactions on Information Theory, vol. IT-13, pp. 21–27, Jan. 1967.Google Scholar
  11. 11.
    J.M. Van Campenhout, “The Arbitrary Relation Between Probability of Error and Measurement Subset,” Journal of the American Statistical Association, vol. 75, no. 369, pp. 104–109, March 1980.MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    P.M. Narendra and K. Fukunaga, “A branch and bound algorithm for feature subset selection,” IEEE Transactions on Computers, vol. 26, no. 9, pp. 917–922, Sept. 1977.MATHCrossRefGoogle Scholar
  13. 13.
    N.L. Johnson and F.C. Leone, Statistics and Experimental Design in Engineering and Physical Sciences, pp. 614–661, New York: Wiley, 1977.MATHGoogle Scholar
  14. 14.
    H.J. Holz and M.H. Loew, “Relative Feature Importance: A Classifier-Independent Approach to Feature Selection,” Pattern Recognition in Practice IV, E.S. Gelsema and L.N. Kanal, eds, pp. 473–487, Elsevier Science B.V., 1994.Google Scholar
  15. 15.
    -, “Non-Parametric Discriminatory Power,” Proceedings of the 1994 IEEE-IMS Workshop on Information Theory and Statistics, pp. 65, Alexandria, VA, 1994.Google Scholar
  16. 16.
    -, “Validation of Relative Feature Importance Using a Natural Data Set,” 15th International Conference on Pattern Recognition, Barcelona, Spain, September 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Hilary J. Holz
    • 1
  • Murray H. Loew
    • 1
  1. 1.Department of Electrical and Computer EngineeringThe George Washington UniversityWashington, DC

Personalised recommendations