On the unranked topology of maximally probable ranked gene tree topologies

Abstract

A ranked tree topology is a tree topology with a temporal ordering of its coalescence events. Under the multispecies coalescent model, we consider ranked gene tree topologies realized along the branches of ranked species trees, where one gene copy is sampled for each species. Previous results have demonstrated that for almost all ranked species tree topologies with at least five species, there exists a set of branch lengths such that the maximally probable ranked gene tree topologies—those generated with the highest probability under the model—do not match the species tree ranked topology. Here, we focus on the agreement of a ranked species tree with its maximally probable ranked gene tree topologies in terms of their unranked topology, that is, disregarding the ordering of the coalescence events. We show that although the set of maximally probable ranked gene tree topologies for a ranked species tree can contain ranked trees with different unranked topologies, at least one of these maximal ranked gene tree topologies must have the same unranked topology as the species tree. Our results contribute to the study of the relationships between gene trees and species trees.

Appendix 1. Proof of the existence of \(\beta \)

We demonstrate the existence of a point $\beta$ of G as defined in step (ii) of the construction described in Sect. 3.1.1. In particular, for a fixed realization of G in S, we seek a point $\beta \ne \gamma _{i^*}$ contemporary with $\gamma _{i^*}$, i.e. at the same height of $\gamma _{i^*}$ in S, lying on a lineage of G that has all its descending taxa below the branch $\alpha$ of S determined in step (i) of the same construction.

First, observe that the node $\gamma _{i^*}$ of G cannot have any of its descending taxa placed below $\alpha$. Indeed, from step (i), $\alpha$ is a branch of $(S_{|G_{i^*}})_{\ell }$ external to $S_{|(G_{i^*})_{\ell }}$—in fact, $\alpha$ coalesces with the root branch of $S_{|(G_{i^*})_{\ell }}$ (Fig. 4). Hence, none of the lineages of $G_{i^*}$ passes through $\alpha$.

Second, with respect to the considered realization of G in S, take any lineage (branch) $\rho$ of G containing a point p contemporary with $\gamma _{i^*}$ and such that at least one of the taxa descending from $\rho$ is placed below the branch $\alpha$. Such a lineage must clearly exists and, as shown above, it cannot contain the node $\gamma _{i^*}$ of G, i.e. $p \ne \gamma _{i^*}$. We claim that p has all its descending taxa below $\alpha$, and thus it satisfies the definition of $\beta$. Suppose a contrario that from p descends a taxon of G that is not placed below $\alpha$. Since p has also a descending taxon below $\alpha$, there must be at least one internal node of G descending from p. Let $\gamma _v$ be the most ancient internal node of G descending from p, where $v > i^*$. As for the point p, the node $\gamma _v$ has some descending taxa below $\alpha$ and some others that are not placed below $\alpha$. Hence, since $\alpha$ is the branch of S that coalesces with the root branch of $S_{| ( G_{i^*} )_{\ell } }$ (Fig. 4), $[S_{|G_v}]$ strictly contains $[S_{| ( G_{i^*} )_{\ell } }]$. Note that $[S_{|G_v}] = [G_v]$, as $v > i^*$, and $[S_{| ( G_{i^*} )_{\ell } }] = [(G_{i^*} )_{\ell }]$. Therefore, the set of taxa of $G_v$ strictly contains the set of taxa of $(G_{i^*} )_{\ell }$, which implies that $\gamma _v$ must be equal or ancestral to $\gamma _{i^*}$ in G. This is in contrast with $v > i^*$. $\Box$