Discriminant analysis for discrete variables derived from a tree-structured graphical model

Abstract

The purpose of this paper is to illustrate the potential use of discriminant analysis for discrete variables whose dependence structure is assumed to follow, or can be approximated by, a tree-structured graphical model. This is done by comparing its empirical performance, using estimated error rates for real and simulated data, with the well-known Naive Bayes classification rule and with linear logistic regression, both of which do not consider any interaction between variables, and with models that consider interactions like a decomposable and the saturated model. The results show that discriminant analysis based on tree-structured graphical models, a simple nonlinear method including only some of the pairwise interactions between variables, is competitive with, and sometimes superior to, other methods which assume no interactions, and has the advantage over more complex decomposable models of finding the graph structure in a fast way and exact form.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Abreu GCG, Edwards D, Labouriau R (2010) High-dimensional graphical model search with the gRapHD R package. J Stat Softw 37(1):1–18

    Article  Google Scholar 

  2. Asparoukhov OK, Krzanowski WJ (2001) A comparison of discriminant procedures for binary variables. Comput Stat Data Anal 38:139–160

    MathSciNet  Article  Google Scholar 

  3. Cheng J, Li T, Levina E, Zhu J (2017) High-dimensional mixed graphical models. J Comput Graph Stat 26(2):367–378

    MathSciNet  Article  Google Scholar 

  4. Chow CK, Liu CN (1966) An approach to structure adaptation in pattern recognition. IEEE Trans Syst Sci Cybern 2:73–80

    Article  Google Scholar 

  5. Chow CK, Liu CN (1968) Approximating discrete probability distributions with dependence trees. IEEE Trans Inf Theory 14:462–467

    Article  Google Scholar 

  6. Chow CK, Wagner TJ (1973) Consistency of an estimate of tree-dependent probability distributions. IEEE Trans Inf Theory 19:369–371

    Article  Google Scholar 

  7. Edwards D, Abreu G, Labouriau R (2010) Selecting high-dimensional mixed graphical models using minimal AIC or BIC forests. BMC Bioinform 11:18

    Article  Google Scholar 

  8. Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29:131–163

    Article  Google Scholar 

  9. Goldstein M, Dillon WR (1978) Discrete discriminant analysis. Wiley, New York

    Google Scholar 

  10. Gou J, Levina E, Michailidis G, Zhu L (2015) Graphical models for ordinal data. J Comput Graph Stat 24(1):183–204

    MathSciNet  Article  Google Scholar 

  11. Hand DJ (1981) Discrimination and classification. Wiley, Chichester

    Google Scholar 

  12. Hastie T, Tibshirani R, Wainwright M (2015) Statistical learning with sparsity. CRC Press, Boca Raton, The lasso and generalizations

    Google Scholar 

  13. Højsgaard S (2012) Graphical independence networks with the gRain package for R. J Stat Softw 46(10):1–26

    Google Scholar 

  14. Højsgaard S, Lauritzen SL, Edwards D (2012) Graphical models with R. Springer, New York

    Google Scholar 

  15. Kim J (2009) Estimating classification error rate: repeated cross-validation, repeated hold-out and bootstrap. Comput Stat Data Anal 53:3735–3745

    MathSciNet  Article  Google Scholar 

  16. Krzanowski WJ, Marriott FHC (1995) Multivariate analysis Part 2: classification, covariance, structures and repeated measurements. Arnold, London

    Google Scholar 

  17. Kruskal JB (1956) On the shortest spanning subtree of a graph and the traveling salesman problem. Proc Am Math Soc 7:48–50

    MathSciNet  Article  Google Scholar 

  18. Lauritzen SL (1996) Graphical models. Oxford University Press, New York

    Google Scholar 

  19. Lee JD, Hastie TJ (2015) Learning the structure of mixed graphical models. J Comput Graph Stat 24(1):230–253

    MathSciNet  Article  Google Scholar 

  20. Loh PL, Wainwright MJ (2013) Structure estimation for discrete graphical models: generalized covariance matrices and their inverses. Ann Stat 41:3022–3049

    MathSciNet  Article  Google Scholar 

  21. Perez-de-la-Cruz G, Eslava-Gomez G (2016) Discriminant analysis with Gaussian graphical tree models. AStA Adv Stat Anal 100:161–187

    MathSciNet  Article  Google Scholar 

  22. Prim RC (1957) Shortest connection networks and some generalizations. Bell Syst Tech J 36:1389–1401

    Article  Google Scholar 

  23. R Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria http://www.R-project.org/

  24. Tan VYF, Sanghavi S, Fisher JW, Willsky AS (2010) Learning graphical models for hypothesis testing and classification. IEEE Trans Signal Proces 58:5481–5495

    MathSciNet  Article  Google Scholar 

  25. Welch BL (1939) Note on discriminant functions. Biometrika 31:218–220

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was written while GEG was at the Department of Applied Mathematics and Computer Science, Technical University of Denmark, on Sabbatical leave from the Faculty of Sciences at the National Autonomous University of Mexico (UNAM), and gratefully acknowledges a six months grant from the program PASPA, DGAPA, UNAM. GPC was a postdoctoral researcher at the Department of Applied Mathematics and Computer Science, Technical University of Denmark, and received a postdoctoral Grant (252737) by the National Council of Science and Technology (CONACYT) of Mexico. We are very grateful to Drs. H. Avila Rosas and L. D. Sánchez Velázquez for providing the ICU data, and for helpful discussions concerning the codification and selection of variables.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gonzalo Perez-de-la-Cruz.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Perez-de-la-Cruz, G., Eslava-Gomez, G. Discriminant analysis for discrete variables derived from a tree-structured graphical model. Adv Data Anal Classif 13, 855–876 (2019). https://doi.org/10.1007/s11634-019-00352-z

Download citation

Keywords

  • Discrete variables
  • Discriminant analysis
  • Error rates
  • Minimum weight spanning tree
  • Multinomial distribution
  • Sparseness
  • Structure estimation
  • Tree-structured graphical models

Mathematics Subject Classification

  • 62H30
  • 68T10