Advertisement

Construction of Decision Trees by Using Feature Importance Value for Improved Learning Performance

  • Md. Ridwan Al Iqbal
  • Mohammad Saiedur Rahaman
  • Syed Irfan Nabil
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7664)

Abstract

Decision Tree algorithms cannot learn accurately with a small training set. This is because, decision tree algorithms recursively partition the data set that leaves very few instances in the lower levels of the tree. Additional domain knowledge has been shown to enhance the performance of learners. We present an algorithm named Importance Aided Decision Tree (IADT) that takes Feature Importance as an additional domain knowledge. Decision Tree algorithm always finds the most important attributes in each node. Thus, Feature Importance can be useful to Decision Tree learning. Our algorithm uses a novel approach to incorporate this feature importance score into decision tree learning. This approach makes decision trees more accurate and robust. We demonstrated theoretical and empirical performance analysis to show that IADT is superior to standard decision tree learning algorithms.

Keywords

Supervised Learning Decision Tree Domain Knowledge 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Scott, A.C., Clayton, J.E., Gibson, E.L.: A practical guide to knowledge acquisition. Addison-Wesley (1991)Google Scholar
  2. 2.
    Alon, N., Spencer, J.: The Probabilistic Method. Wiley, New York (1992)zbMATHGoogle Scholar
  3. 3.
    Breiman, L., Freidman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression treesGoogle Scholar
  4. 4.
    Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 476–491 (1997)CrossRefGoogle Scholar
  5. 5.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc., 13–30Google Scholar
  6. 6.
    Iqbal, R.A.: Empirical learning aided by weak knowledge in the form of feature importance. In: CMSP 2011. IEEE (2011)Google Scholar
  7. 7.
    Mitchell, T.M.: Artificial neural networks, pp. 81–126. McGraw-Hill Science/Engineering/Math. (1997)Google Scholar
  8. 8.
    Mitchell, T.M.: Machine Learning. McGraw-Hill (1997a)Google Scholar
  9. 9.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
  10. 10.
    Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C 35(4), 476–487 (2005)CrossRefGoogle Scholar
  11. 11.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  12. 12.
    Zhang, L., Wang, Z.: Ontology-based clustering algorithm with feature weights. Journal of Computational Information Systems 6(9) (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Md. Ridwan Al Iqbal
    • 1
  • Mohammad Saiedur Rahaman
    • 1
  • Syed Irfan Nabil
    • 1
  1. 1.Department of Computer ScienceAmerican International University-Bangladesh (AIUB)BananiBangladesh

Personalised recommendations