# Developing a statistically powerful measure for quartet tree inference using phylogenetic identities and Markov invariants

## Abstract

Recently there has been renewed interest in phylogenetic inference methods based on phylogenetic invariants, alongside the related Markov invariants. Broadly speaking, both these approaches give rise to polynomial functions of sequence site patterns that, in expectation value, either vanish for particular evolutionary trees (in the case of phylogenetic invariants) or have well understood transformation properties (in the case of Markov invariants). While both approaches have been valued for their intrinsic mathematical interest, it is not clear how they relate to each other, and to what extent they can be used as practical tools for inference of phylogenetic trees. In this paper, by focusing on the special case of binary sequence data and quartets of taxa, we are able to view these two different polynomial-based approaches within a common framework. To motivate the discussion, we present three desirable statistical properties that we argue any invariant-based phylogenetic method should satisfy: (1) sensible behaviour under reordering of input sequences; (2) stability as the taxa evolve independently according to a Markov process; and (3) explicit dependence on the assumption of a continuous-time process. Motivated by these statistical properties, we develop and explore several new phylogenetic inference methods. In particular, we develop a statistically bias-corrected version of the Markov invariants approach which satisfies all three properties. We also extend previous work by showing that the phylogenetic invariants can be implemented in such a way as to satisfy property (3). A simulation study shows that, in comparison to other methods, our new proposed approach based on bias-corrected Markov invariants is extremely powerful for phylogenetic inference. The binary case is of particular theoretical interest as—in this case only—the Markov invariants can be expressed as linear combinations of the phylogenetic invariants. A wider implication of this is that, for models with more than two states—for example DNA sequence alignments with four-state models—we find that methods which rely on phylogenetic invariants are incapable of satisfying all three of the stated statistical properties. This is because in these cases the relevant Markov invariants belong to a class of polynomials independent from the phylogenetic invariants.

## Keywords

Phylogenetic invariants Quartets Markov chains Representation theory## Mathematics Subject Classification

92B10 20G05 16W22## Notes

### Acknowledgements

We would like to thank the two anonymous reviewers whose thoughtful and careful reading of our manuscript led to a greatly improved final version.

### Compliance with ethical standards

### Funding

This work was supported by the Australian Research Council Discovery Early Career Fellowship DE130100423 (JGS) and the University of Tasmania Visiting Scholars Program (AT).

## Supplementary material

## References

- Allman ES, Rhodes JA (2003) Phylogenetic invariants of the general Markov model of sequence mutation. Math Biosci 186:113–144MathSciNetCrossRefMATHGoogle Scholar
- Allman ES, Rhodes JA (2008) Phylogenetic ideals and varieties for the general Markov model. Adv Appl Math 40:127–148MathSciNetCrossRefMATHGoogle Scholar
- Allman ES, Rhodes JA, Taylor A (2014) A semialgebraic description of the general markov model on phylogenetic trees. SIAM J Discrete Math 28(2):736–755MathSciNetCrossRefMATHGoogle Scholar
- Bates DJ, Oeding L (2011) Toward a salmon conjecture. Exp Math 20(3):358–370MathSciNetCrossRefMATHGoogle Scholar
- Casanellas M, Fernández-Sánchez J (2010) Relevant phylogenetic invariants of evolutionary models. J Math Pures Appl 96:207–229MathSciNetCrossRefMATHGoogle Scholar
- Cavender JA, Felsenstein J (1987) Invariants of phylogenies in a simple case with discrete states. J Classif 4:57–71CrossRefMATHGoogle Scholar
- Chor B, Hendy MD, Holland BR, Penny D (2000) Multiple maxima of likelihood in phylogenetic trees: an analytic approach. Mol Biol Evol 17:1529–1541CrossRefGoogle Scholar
- Davey JW, Hohenlohe PA, Etter PD, Boone JQ, Catchen JM, Blaxter ML (2011) Genome-wide genetic marker discovery and genotyping using next-generation sequencing. Nat Rev Genet 12(7):499–510CrossRefGoogle Scholar
- Draisma J, Kuttler J (2008) On the ideals of equivariant tree models. Math Ann 344:619–644MathSciNetCrossRefMATHGoogle Scholar
- Eriksson N (2008) Using invariants for phylogenetic tree construction. In: Putinar M, Sullivant S (eds) Emerging applications of algebraic geometry. Springer, BerlinGoogle Scholar
- Felsenstein J (1978) Cases in which parsimony or compatibility methods will be positively misleading. Syst Zool 27:401–410CrossRefGoogle Scholar
- Felsenstein J (1981) Evolutionary trees from DNA sequences: a maximum likelihood approach. J Mol Evol 17:368–376CrossRefGoogle Scholar
- Felsenstein J (2004) Inferring phylogenies. Sinauer Associates, SunderlandGoogle Scholar
- Fernández-Sánchez J, Casanellas M (2015) Invariant versus classical quartet inference when evolution is heterogeneous across sites and lineages. Syst. Biol. 65(2):280–291. http://sysbio.oxfordjournals.org/content/early/2015/11/11/sysbio.syv086.abstract
- Friedland S (2013) On tensors of border rank l in \(\mathbb{C}^{m\times n\times l}\). Linear Algebra Appl 438(2):713–737MathSciNetCrossRefMATHGoogle Scholar
- Friedland S, Gross E (2012) A proof of the set-theoretic version of the salmon conjecture. J Algebra 356(1):374–379MathSciNetCrossRefMATHGoogle Scholar
- Hillis D, Huelsenbeck J, Swofford D (1994) Hobgoblin of phylogenetics? Nature 369:363–364CrossRefGoogle Scholar
- Holland BR, Sumner JG, Jarvis PD (2013) Low-parameter phylogenetic inference under the general Markov model. Syst Biol 62:78–92CrossRefGoogle Scholar
- Huelsenbeck JP, Hillis DM (1993) Success of phylogenetic methods in the four-taxon case. Syst Biol 42(3):247–264CrossRefGoogle Scholar
- Jarvis PD, Sumner JG (2014) Adventures in invariant theory. ANZIAM J 56(02):105–115MathSciNetCrossRefMATHGoogle Scholar
- Lake JA (1987) A rate-independent technique for analysis of nucleic acid sequences: evolutionary parsimony. Mol Biol Evol 4:167–191Google Scholar
- Lemmon EM, Lemmon AR (2013) High-throughput genomic data in systematics and phylogenetics. Annu Rev Ecol Evol Syst 44:99–121CrossRefGoogle Scholar
- Olver PJ (2003) Classical invariant theory. Cambridge University Press, CambridgeGoogle Scholar
- Rusinko JP, Hipp B (2012) Invariant based quartet puzzling. Algorithms Mol Biol 7(1):1CrossRefGoogle Scholar
- Saitou N, Nei M (1987) The neighbor-joining method: a new method for reconstructing phylogenetic trees. Mol Biol Evol 4(4):406–425Google Scholar
- Sumner JG (2017) Dimensional reduction for the general Markov Model on phylogenetic trees. Bull Math Biol 79(3):619–634. doi: 10.1007/s11538-017-0249-6 MathSciNetCrossRefMATHGoogle Scholar
- Sumner JG, Charleston MA, Jermiin LS, Jarvis PD (2008) Markov invariants, plethysms, and phylogenetics. J Theor Biol 253:601–615MathSciNetCrossRefGoogle Scholar
- Sumner JG, Fernández-Sánchez J, Jarvis PD (2012) Lie Markov models. J Theor Biol 298:16–31MathSciNetCrossRefGoogle Scholar
- Sumner JG, Jarvis PD (2009) Markov invariants and the isotropy subgroup of a quartet tree. J Theor Biol 258:302–310MathSciNetCrossRefGoogle Scholar
- Swofford DL, Waddell PJ, Huelsenbeck JP, Foster PG, Lewis PO, Rogers JS (2001) Bias in phylogenetic estimation and its relevance to the choice between parsimony and likelihood methods. Syst Biol 50(4):525–539CrossRefGoogle Scholar
- Wolfram Research Inc (2010) Mathematica 8. Wolfram Research Inc, Champaign, ILGoogle Scholar