Possibilistic classifiers for numerical data
 Myriam Bounhas,
 Khaled Mellouli,
 Henri Prade,
 Mathieu Serrurier
 … show all 4 hide
Purchase on Springer.com
$39.95 / €34.95 / £29.95*
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Abstract
Naive Bayesian Classifiers, which rely on independence hypotheses, together with a normality assumption to estimate densities for numerical data, are known for their simplicity and their effectiveness. However, estimating densities, even under the normality assumption, may be problematic in case of poor data. In such a situation, possibility distributions may provide a more faithful representation of these data. Naive Possibilistic Classifiers (NPC), based on possibility theory, have been recently proposed as a counterpart of Bayesian classifiers to deal with classification tasks. There are only few works that treat possibilistic classification and most of existing NPC deal only with categorical attributes. This work focuses on the estimation of possibility distributions for continuous data. In this paper we investigate two kinds of possibilistic classifiers. The first one is derived from classical or flexible Bayesian classifiers by applying a probability–possibility transformation to Gaussian distributions, which introduces some further tolerance in the description of classes. The second one is based on a direct interpretation of data in possibilistic formats that exploit an idea of proximity between data values in different ways, which provides a less constrained representation of them. We show that possibilistic classifiers have a better capability to detect new instances for which the classification is ambiguous than Bayesian classifiers, where probabilities may be poorly estimated and illusorily precise. Moreover, we propose, in this case, an hybrid possibilistic classification approach based on a nearestneighbour heuristics to improve the accuracy of the proposed possibilistic classifiers when the available information is insufficient to choose between classes. Possibilistic classifiers are compared with classical or flexible Bayesian classifiers on a collection of benchmarks databases. The experiments reported show the interest of possibilistic classifiers. In particular, flexible possibilistic classifiers perform well for data agreeing with the normality assumption, while proximitybased possibilistic classifiers outperform others in the other cases. The hybrid possibilistic classification exhibits a good ability for improving accuracy.
Inside
Within this Article
 Introduction
 General setting of possibilistic classification
 Elicitation of the possibility distributions
 Detecting ambiguities in possibilistic classifiers as a basis for improvement
 Related works
 Experiments and discussion
 Conclusion and discussion
 References
 References
Other actions
 Ben Amor N, Mellouli K, Benferhat S, Dubois D, Prade H (2002) A theoretical framework for possibilistic independence in a weakly ordered setting. Int J Uncertain Fuzziness KnowledgeBased Syst 10:117–155
 Ben Amor N, Benferhat S, Elouedi Z (2004) Qualitative classification and evaluation in possibilistic decision trees. In: FUZZIEEE’04, vol 1, pp 653–657
 Benferhat S, Tabia K (2008) An efficient algorithm for naive possibilistic classifiers with uncertain inputs. In: Proceedings of 2nd international conference on scalable uncertainty management (SUM’08). LNAI, vol 5291. Springer, Berlin, pp 63–77
 Beringer J, Hüllermeier E (2008) Casebased learning in a bipolar possibilistic framework. Int J Intell Syst 23:1119–1134 CrossRef
 Bishop CM (1996) Neural networks for pattern recognition. Oxford University Press, New York
 Bishop CM (1999) Latent variable models. In: Learning in graphical models, pp 371–403
 Borgelt C, Gebhardt J (1999) A naïve bayes style possibilistic classifier. In: Proceedings of 7th European congress on intelligent techniques and soft computing, pp 556–565
 Borgelt C, Kruse R (1988) Efficient maximum projection of databaseinduced multivariate possibility distributions. In: Proceedings of 7th IEEE international conference on fuzzy systems, pp 663–668
 Bounhas M, Mellouli K (2010) A possibilistic classification approach to handle continuous data. In: Proceedings of the eighth ACS/IEEE international conference on computer systems and applications (AICCSA10), pp 1–8
 Bounhas M, Mellouli K, Prade H, Serrurier M (2010) From bayesian classifiers to possibilistic classifiers for numerical data. In: Proceedings of the fourth international conference on scalable uncertainty management, pp 112–125
 Bounhas M, Prade H, Serrurier M, Mellouli K (2011) Possibilistic classifiers for uncertain numerical data. In: Proceedings of 11th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU’11), Belfast, UK, June 29–July 1. LNCS, vol 6717. Springer, Berlin, pp 434–446
 Cheng J, Greiner R (1999) Comparing bayesian network classifiers. In: Proceedings of the 15th conference on uncertainty in artificial intelligence, pp 101–107
 Cover TM, Hart PE (1967) Nearest neighbour pattern classification. IEEE Trans Inf Theory 13:21–27 CrossRef
 De Cooman G (1997) Possibility theory. Part I: measure and integraltheoretic ground work. Part II: conditional possibility; Part III: possibilistic independence. Int J Gen Syst 25:291–371
 Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
 Denton A, Perrizo W (2004) A kernelbased seminaive Bayesian classifier using ptrees. In: Proceedings of the 4th SIAM international conference on data mining
 Devroye L (1983) The equivalence of weak, strong, and complete convergence in l1 for kernel density estimates. Ann Stat 11:896–904
 Domingos P, Pazzani M (2002) Beyond independence: conditions for the optimality of the simple bayesian classifier. Mach Learn 29:102–130
 Dubois D (2006) Possibility theory and statistical reasoning. Comput Stat Data Anal 51:47–69 CrossRef
 Dubois D, Prade H (1988) Possibility theory: an approach to computerized processing of uncertainty
 Dubois D, Prade H (1990) Aggregation of possibility measures. In: Multiperson decision making using fuzzy sets and possibility theory, pp 55–63
 Dubois D, Prade H (1990) The logical view of conditioning and its application to possibility and evidence theories. Int J Approx Reason 4:23–46 CrossRef
 Dubois D, Prade H (1992) When upper probabilities are possibility measures. Fuzzy Sets Syst 49:65–74 CrossRef
 Dubois D, Prade H (1993) On data summarization with fuzzy sets. In: Proceedings of the 5th international fuzzy systems association. World Congress (IFSA’93)
 Dubois D, Prade H (1998) Possibility theory: qualitative and quantitative aspects. In: Gabbay D, Smets P (eds) Handbook on defeasible reasoning and uncertainty management systems, vol 1, pp 169–226
 Dubois D, Prade H (2000) An overview of ordinal and numerical approaches to causal diagnostic problem solving. In: Gabbay DM, Kruse R (eds) Abductive reasoning and learning, handbooks of defeasible reasoning and uncertainty management systems, drums handbooks, vol 4, pp 231–280
 Dubois D, Prade H (2009) Formal representations of uncertainty. In: Bouyssou D, Dubois D, Pirlot M, Prade H (eds) Decisionmaking—concepts and methods, pp 85–156
 Dubois D, Prade H, Sandri S (1993) On possibility/probability transformations. Fuzzy Logic, pp 103–112
 Dubois D, Laurent F, Gilles M, Prade H (2004) Probabilitypossibility transformations, triangular fuzzy sets, and probabilistic inequalities. Reliable Comput 10:273–297 CrossRef
 Figueiredo M, Leitao JMN (1999) On fitting mixture models. In: Energy minimization methods in computer vision and pattern recognition, vol 1654, pp 732–749
 Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29:131–161 CrossRef
 Geiger D, Heckerman D. (1994) Learning gaussian networks. Technical report, Microsoft Research, Advanced Technology Division
 Grossman D, Dominigos P (2004) Learning Bayesian maximizing conditional likelihood. In: Proceedings on machine learning, pp 46–57
 Haouari B, Ben Amor N, Elouadi Z, Mellouli K (2009) Naive possibilistic network classifiers. Fuzzy Sets Syst 160(22):3224–3238 CrossRef
 Hüllermeier E (2003) Possibilistic instancebased learning. Artif Intell 148(1–2):335–383 CrossRef
 Hüllermeier E (2005) Fuzzy methods in machine learning and data mining: status and prospects. Fuzzy Sets Syst 156(3):387–406 CrossRef
 Jenhani I, Ben Amor N, Elouedi Z (2008) Decision trees as possibilistic classifiers. Int J Approx Reason 48(3):784–807 CrossRef
 John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the 11th conference on uncertainty in artificial intelligence
 Kononenko I (1991) Seminaive bayesian classifier. In: Proceedings of the European working session on machine learning, pp 206–219
 Kotsiantis SB (2007) Supervised machine learning: a review of classification techniques. Informatica 31:249–268
 Langley P, Sage S (1994) Induction of selective bayesian classifiers. In: Proceedings of 10th conference on uncertainty in artificial intelligence (UAI94), pp 399–406
 Langley P, Iba W, Thompson K (1992) An analysis of bayesian classifiers. In: Proceedings of AAAI92, vol 7, pp 223–228
 McLachlan GJ, Peel D (2000) Finite mixture models. Probability and mathematical statistics. Wiley, New York
 Mertz J, Murphy PM (2000) Uci repository of machine learning databases. ftp://ftp.ics.uci.edu/pub/machinelearningdatabases
 Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmman, San Francisco
 Pérez A, Larraoaga P, Inza I (2009) Bayesian classifiers based on kernel density estimation: flexible classifiers. Int J Approx Reason 50:341–362 CrossRef
 Quinlan JR (1986) Induction of decision trees. Mach Learn 1:81–106
 Sahami M (1996) Learning limited dependence bayesian classifiers. In: Proceedings of the 2nd international conference on knowledge discovery and data mining, pp 335–338
 Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton
 Solomonoff R (1964) A formal theory of inductive inference. Inf Control 7:224–254 CrossRef
 Strauss O, Comby F, Aldon MJ (2000) Rough histograms for robust statistics. In: Proceedings of international conference on pattern recognition (ICPR’00), vol II, Barcelona. IEEE Computer Society, pp 2684–2687
 Sudkamp T (2000) Similarity as a foundation for possibility. In: Proceedings of 9th IEEE international conference on fuzzy systems, San Antonio, pp 735–740
 Yamada K (2001) Probabilitypossibility transformation based on evidence theory. In: Joint 9th IFSA World Congress and 20th NAFIPS international conference 2001, pp 70–75
 Yang Y, Webb GI (2003) Discretization for naivebayes learning: managing discretization bias and variance. Technical Report 2003131
 Zadeh LA (1978) Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst 1:3–28 CrossRef
 Zhang H (2004) The optimality of naive bayes. In: Proceedings of 17th international FLAIRS conference (FLAIRS2004)
 Title
 Possibilistic classifiers for numerical data
 Journal

Soft Computing
Volume 17, Issue 5 , pp 733751
 Cover Date
 20130501
 DOI
 10.1007/s0050001209479
 Print ISSN
 14327643
 Online ISSN
 14337479
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 Naive Possibilistic Classifier
 Possibility theory
 Proximity
 Gaussian distribution
 Naive Bayesian Classifier
 Numerical data
 Industry Sectors
 Authors

 Myriam Bounhas ^{(1)}
 Khaled Mellouli ^{(1)}
 Henri Prade ^{(2)}
 Mathieu Serrurier ^{(2)}
 Author Affiliations

 1. Laboratoire LARODEC, ISG de Tunis, 41 rue de la liberté, 2000, Le Bardo, Tunisia
 2. Institut de Recherche en Informatique de Toulouse (IRIT), UPSCNRS, 118 route de Narbonne, 31062, Toulouse Cedex, France