Bulletin of Mathematical Biology

, Volume 81, Issue 2, pp 316–336 | Cite as

Dimensions of Group-Based Phylogenetic Mixtures

  • Hector Baños
  • Nathaniel Bushek
  • Ruth Davidson
  • Elizabeth GrossEmail author
  • Pamela E. Harris
  • Robert Krone
  • Colby Long
  • Allen Stewart
  • Robert Walker
Special Issue: Algebraic Methods in Phylogenetics


Mixtures of group-based Markov models of evolution correspond to joins of toric varieties. In this paper, we establish a large number of cases for which these phylogenetic join varieties realize their expected dimension, meaning that they are nondefective. Nondefectiveness is not only interesting from a geometric point-of-view, but has been used to establish combinatorial identifiability for several classes of phylogenetic mixture models. Our focus is on group-based models where the equivalence classes of identified parameters are orbits of a subgroup of the automorphism group of the abelian group defining the model. In particular, we show that for these group-based models, the variety corresponding to the mixture of r trees with n leaves is nondefective when \(n \ge 2r+5\). We also give improved bounds for claw trees and give computational evidence that 2-tree and 3-tree mixtures are nondefective for small n.



This work began at the 2016 AMS Mathematics Research Community on “Algebraic Statistics,” which was supported by the National Science Foundation under Grant number DMS-1321794. RD was supported by NSF DMS-1401591. EG was supported by NSF DMS-1620109. RW was supported by a NSF GRF under Grant number PGF-031543, NSF RTG Grant 0943832, and a Ford Foundation Dissertation Fellowship. HB was supported in part by a research assistantship, funded by the National Institutes of Health Grant R01 GM117590. PEH was partially supported by NSF Grant DMS-1620202.


  1. Abo H, Brambilla MC (2012) New examples of defective secant varieties of Segre–Veronese varieties. Collectanea Mathematica 63(3):287–297MathSciNetCrossRefzbMATHGoogle Scholar
  2. Abo H, Brambilla MC (2013) On the dimensions of secant varieties of Segre-Veronese varieties. Annali di Matematica Pura ed Applicata 192(1):1–32MathSciNetCrossRefzbMATHGoogle Scholar
  3. Alexander J, Hirschowitz A (1995) Polynomial interpolation in several variables. J Algebraic Geom 4(2):201–222MathSciNetzbMATHGoogle Scholar
  4. Allman ES, Petrović S, Rhodes JA, Sullivant S (2011) Identifiability of 2-tree mixtures for group-based models. IEEE/ACM Trans Comput Biol Bioinf 8(3):710–722CrossRefGoogle Scholar
  5. Baños H, Bushek N, Davidson R, Gross E, Harris PE, Krone R, Long C, Stewart A, Walker R (2016) Phylogenetic trees. arXiv preprint arXiv:1611.05805 (2016)
  6. Buczynska W, Wisniewski JA (2007) On the geometry of binary symmetric models of phylogenetic trees. J Eur Mathe Soc 9(3):609–635MathSciNetCrossRefzbMATHGoogle Scholar
  7. Casanellas M (2012) Algebraic tools for evolutionary biology. Math Soc 12–17Google Scholar
  8. Daskalakis C, Mossel E, Roch S (2011) Evolutionary trees and the ising model on the Bethe lattice: a proof of steel’s conjecture. Probab Theory Rel Fields 149(1):149–189Google Scholar
  9. Draisma J (2008) A tropical approach to secant dimensions. J Pure Appl Algebra 212(2):349–363MathSciNetCrossRefzbMATHGoogle Scholar
  10. Evans SN, Speed TP (1993) Invariants of some probability models used in phylogenetic inference. Ann Stat 21(1):355–377MathSciNetCrossRefzbMATHGoogle Scholar
  11. Grayson DR, Stillman ME (2002) Macaulay 2, a software system for research in algebraic geometryGoogle Scholar
  12. Hendy MD, Penny D, Steel MA (1994) A discrete fourier analysis for evolutionary trees. Proc Natl Acad Sci 91(8):3339–3343CrossRefzbMATHGoogle Scholar
  13. Jukes TH, Cantor CR (1969) Evolution of protein molecules. Mamm Protein Metab 3(21):132Google Scholar
  14. Long C, Sullivant S (2015) Identifiability of 3-class Jukes–Cantor mixtures. Adv Appl Math 64:89–110, 3MathSciNetCrossRefzbMATHGoogle Scholar
  15. Mauhar M, Rusinko J, Vernon Z (2017) H-representation of the kimura-3 polytope for the m-claw tree. SIAM J Discret Math 31(2):783–795MathSciNetCrossRefzbMATHGoogle Scholar
  16. Michałek M (2011) Geometry of phylogenetic group-based models. J Algebra 339(1):339–356MathSciNetCrossRefzbMATHGoogle Scholar
  17. Neyman J (1971) Molecular studies of evolution: a source of novel statistical problems. Stat Decis Theory Rel Top 1:1–27MathSciNetzbMATHGoogle Scholar
  18. Pagel M, Meade A (2004) A phylogenetic mixture model for detecting pattern-heterogeneity in gene sequence or character-state data. Syst Biol 53(4):571–581CrossRefGoogle Scholar
  19. Sturmfels B (1996) Gröbner bases and Convex Polytopes, vol 8. American Mathematical Society, ProvidencezbMATHGoogle Scholar
  20. Sturmfels B, Sullivant S (2005) Toric ideals of phylogenetic invariants. J Comput Biol 12(2):204–228CrossRefzbMATHGoogle Scholar
  21. Sullivant S (2018) Algebraic statistics. Graduate studies in mathematics. American Mathematical Society, ProvidencezbMATHGoogle Scholar

Copyright information

© Society for Mathematical Biology 2018

Authors and Affiliations

  • Hector Baños
    • 1
  • Nathaniel Bushek
    • 2
  • Ruth Davidson
    • 3
  • Elizabeth Gross
    • 4
    Email author
  • Pamela E. Harris
    • 5
  • Robert Krone
    • 6
  • Colby Long
    • 7
  • Allen Stewart
    • 8
  • Robert Walker
    • 9
  1. 1.Department of Mathematics and StatisticsUniversity of Alaska FairbanksFairbanksUS
  2. 2.Department of Mathematics and StatisticsUniversity of Minnesota DuluthDuluthUSA
  3. 3.Department of MathematicsUniversity of Illinois Urbana-ChampaignUrbanaUSA
  4. 4.Department of MathematicsUniversity of Hawai’i at MānoaHonoluluUSA
  5. 5.Department of Mathematics and StatisticsWilliams CollegeWilliamstownUSA
  6. 6.Department of MathematicsUniversity of California DavisDavisUSA
  7. 7.Mathematical Biosciences InstituteThe Ohio State UniversityColumbusUSA
  8. 8.Department of MathematicsSeattle UniversitySeattleUSA
  9. 9.Department of MathematicsUniversity of MichiganAnn ArborUSA

Personalised recommendations