Skip to main content
SpringerLink
Log in

Hybrid Bayesian estimation tree learning with discrete and fuzzy labels

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Classical decision tree model is one of the classical machine learning models for its simplicity and effectiveness in applications. However, compared to the DT model, probability estimation trees (PETs) give a better estimation on class probability. In order to get a good probability estimation, we usually need large trees which are not desirable with respect to model transparency. Linguistic decision tree (LDT) is a PET model based on label semantics. Fuzzy labels are used for building the tree and each branch is associated with a probability distribution over classes. If there is no overlap between neighboring fuzzy labels, these fuzzy labels then become discrete labels and a LDT with discrete labels becomes a special case of the PET model. In this paper, two hybrid models by combining the naive Bayes classifier and PETs are proposed in order to build a model with good performance without losing too much transparency. The first model uses naive Bayes estimation given a PET, and the second model uses a set of small-sized PETs as estimators by assuming the independence between these trees. Empirical studies on discrete and fuzzy labels show that the first model outperforms the PET model at shallow depth, and the second model is equivalent to the naive Bayes and PET.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Quinlan J R. Induction of decision trees. Machine Learning, 1986, 1(1): 81–106

    Google Scholar 

  2. Olaru C, Wehenkel L. A complete fuzzy decision tree technique. Fuzzy Sets and Systems, 2003, 138(2): 221–254

    Article  MathSciNet  Google Scholar 

  3. Quinlan J R. C4. 5: programs for machine learning. Morgan Kaufmann, 1993

    Google Scholar 

  4. Baldwin J, Lawry J, Martin T. Mass assignment fuzzy ID3 with applications. In: Proceedings of the Unicom Workshop on Fuzzy Logic: Applications and Future Directions. 1997, 278–294

    Google Scholar 

  5. Janikow C Z. Fuzzy decision trees: issues and methods. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 1998, 28(1): 1–14

    Article  Google Scholar 

  6. Huang Z, Gedeon T D, Nikravesh M. Pattern trees induction: a new machine learning method. IEEE Transactions on Fuzzy Systems, 2008, 16(4): 958–970

    Article  Google Scholar 

  7. Qin B, Xia Y, Li F. Dtu: a decision tree for uncertain data. Advances in Knowledge Discovery and Data Mining, 2009: 4–15

    Chapter  Google Scholar 

  8. Provost F, Domingos P. Tree induction for probability-based ranking. Machine Learning, 2003, 52(3): 199–215

    Article  MATH  Google Scholar 

  9. Qin Z, Lawry J. Decision tree learning with fuzzy labels. Information Sciences, 2005, 172(1): 91–129

    Article  MathSciNet  MATH  Google Scholar 

  10. Qin Z, Lawry J. Prediction trees using linguistic modelling. In: Proceedings of World Congress of International Fuzzy Systems Association (IFSA-05), 2005

    Google Scholar 

  11. Qin Z, Lawry J. Prediction and query evaluation using linguistic decision trees. Applied Soft Computing, 2011, 11(5): 3916–3928

    Article  Google Scholar 

  12. Lawry J. A framework for linguistic modelling. Artificial Intelligence, 2004, 155(1): 1–39

    Article  MathSciNet  MATH  Google Scholar 

  13. Elkan C. Naive bayesian learning. Technical Report CS97-557, Dept. of Computer Science and Engineering, UCSD, 1997

    Google Scholar 

  14. Blake C, Merz C J. UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html

  15. Zadeh L A. Fuzzy logic= computing with words. IEEE Transactions on Fuzzy Systems, 1996, 4(2): 103–111

    Article  MathSciNet  Google Scholar 

  16. Zadeh L A. The concept of a linguistic variable and its application to approximate reasoning-I. Information Sciences, 1975, 8(3): 199–249

    Article  MathSciNet  MATH  Google Scholar 

  17. Sufyan Beg M, Thint M, Qin Z. Pnl-enhanced restricted domain question answering system. In: Proceedings of the 2007 IEEE International Fuzzy Systems Conference. 2007, 1–7

    Google Scholar 

  18. Qin Z, Thint M, Beg M S. Deduction engine design for pnl-based question answering system. In: Proceedings of the 12th International Fuzzy Systems Association World Congress. 2007, 253–262

    Google Scholar 

  19. Lawry J. Modeling and reasoning with vague concepts. Springer, 2006

    Google Scholar 

  20. Lawry J, Shanahan J G, Ralescu A. Modelling with words: learning, fusion, and reasoning within a formal linguistic representation framework. Volume 2873. Springer, 2003

    Book  Google Scholar 

  21. Qin Z, Lawry J. Lfoil: linguistic rule induction in the label semantics framework. Fuzzy Sets and Systems, 2008, 159(4): 435–448

    Article  MathSciNet  MATH  Google Scholar 

  22. Baldwin J F, Martin T P, Pilsworth B W. Fril-fuzzy and evidential reasoning in artificial intelligence. John Wiley & Sons, Inc., 1995

    Google Scholar 

  23. Zhang W, Qin Z. Dissimilarity measure of logical expressions. In: Proceedings of the 2010 International Conference on Machine Learning and Cybernetics (ICMLC). 2010, 199–203

    Chapter  Google Scholar 

  24. Zhang W, Qin Z. Clustering data and imprecise concepts. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ). 2011, 603–608

    Google Scholar 

  25. Jeffrey R C. The logic of decision. University of Chicago Press, 1990

    Google Scholar 

  26. Qin Z, Lawry J. Fuzziness and performance: an empirical study with linguistic decision trees. In: Proceedings of the 12th International Fuzzy Systems Association World Congress on Foundations of Fuzzy Logic and Soft Computing. 2007, 407–416

    Chapter  Google Scholar 

  27. Randon N J, Lawry J. Classification and query evaluation using modelling with words. Information Sciences, 2006, 176(4): 438–464

    Article  MathSciNet  MATH  Google Scholar 

  28. Qin Z. Naive bayes classification given probability estimation trees. In: Proceedings of the 5th International Conference on Machine Learning and Applications, ICMLA’06. 2006, 34–42

    Google Scholar 

  29. Qin Z, Lawry J. Hybrid bayesian estimation trees based on label semantics. Lecture Notes in Computer Science, 2005, 896–907

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intelligent Computing and Machine Learning Lab, School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China

    Zengchang Qin

  2. School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China

    Tao Wan

  3. Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA

    Tao Wan

Authors
  1. Zengchang Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zengchang Qin.

Additional information

Zengchang Qin obtained his MSc in Computer Science and PhD in Artificial Intelligence from the University of Bristol, UK, in 2002 and 2005, respectively. He worked as a lecturer in the same university before joining Lotfi Zadeh’s BISC group at the EECS Department of UC Berkeley as the BT postdoctoral fellow in 2006. He has been working in Beihang University as an associate professor in the School of Automation Science and Electrical Engineering from 2009. He was also a visiting scholar at Robotics Institute, Carnegie Mellon University, from November 2010 to June 2011. His research interests are uncertainty modeling, machine learning, multimedia retrieval and agent-based modeling.

Tao Wan is a research associate at the Case Western Reserve University, USA. She was a postdoctoral associate in School of Medicine at the Boston University. She received her MS in Global Computing and Multimedia from the University of Bristol, UK in 2004 and her PhD in Computer Science from the same university in 2009. She spent one year working as a senior researcher in the Samsung Advanced Institute of Technology (SAIT) China before becoming a visiting scholar in the Visualization and Image Analysis Lab in the Robotics Institute, Carnegie Mellon University. Her research interests are statistical models for image segmentation, fusion, and denoising, machine learning, computer-aided diagnosis system, medical image analysis on prostate and breast cancer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Qin, Z., Wan, T. Hybrid Bayesian estimation tree learning with discrete and fuzzy labels. Front. Comput. Sci. 7, 852–863 (2013). https://doi.org/10.1007/s11704-013-3007-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-013-3007-4

Keywords

Search

Navigation