A Novel Supervised Information Feature Compression Algorithm

  • Shifei Ding
  • Zhongzhi Shi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3991)


In this paper, a novel supervised information feature compression algorithm is set up. Firstly, according to the information theories, we carried out analysis for the concept and its properties of the cross entropy, then put forward a kind of lately concept of symmetry cross entropy (SCE), and point out that the SCE is a kind of distance measure, which can be used to measure the difference of two random variables. Secondly, We make the SCE separability criterion of the classes for information feature compression, and design a novel algorithm for information feature compression. At last, the experimental results demonstrate that the algorithm here is valid and reliable, and provides a new research approach for feature compression, data mining and pattern recognition.


Feature Selection Feature Extraction Probability Vector Feature Selection Algorithm Separability Criterion 


  1. Duda, R.O., Hart, P.E. (eds.): Pattern Classification and Scene Analysis. Wiley, New York (1973)MATHGoogle Scholar
  2. Fukunaga, K. (ed.): Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, New York (1990)MATHGoogle Scholar
  3. Hand, D.J. (ed.): Discrimination and Classification. Wiley, New York (1981)MATHGoogle Scholar
  4. Nadler, M., Smith, E.P. (eds.): Pattern Recognition Engineering. Wiley, New York (1993)MATHGoogle Scholar
  5. Yang, J., Yang, J.Y.: A Generalized K-L Expansion Method That Can Deal With Small Sample Size and High-dimensional Problems. Pattern Analysis Applications 6(6), 47–54 (2003)MATHCrossRefGoogle Scholar
  6. Shannon, C.E.: A Mathematical Theory of Communication. Bell Syst. Tech. J. 27, 379–423 (1948)MATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shifei Ding
    • 1
    • 2
  • Zhongzhi Shi
    • 2
  1. 1.College of Information Science and EngineeringShandong Agricultural UniversityTaianP.R. China
  2. 2.Key Laboratory of Intelligent Information Processing, Institute of Computing TechnologyChinese Academy of SciencesBeijingP.R.China

Personalised recommendations