Tree languages and branched groups

We study the portraits of isometries of rooted trees—the labelling of the tree, at each vertex, by the permutation of its descendants—in terms of languages. We characterize regularly branched self-similar groups in terms of ω\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\omega $$\end{document}-regular languages. We deduce the algorithmic decidability of some problems, such as the comparison of regularly branched contracting groups, and their orbit structure on the boundary of the rooted tree.


Introduction
There is a rich interplay between group theory and theoretical computer science, and more precisely the theory of formal languages.We develop it in the context of self-similar groups acting on rooted trees.
1.1.Regular languages.Let A be a finite set called alphabet.A language is a subset of A * , where A * denotes the set of all finite words in A. Some languages can be described by a finite state automaton: a device M with a finite amount of memory that reads letter-by-letter words over the alphabet A as input and decides at the end whether to accept the word.An automaton M accepts the word w if there exists a run q 0 , . . ., q n on w such that q n ∈ Q f .The language L(M) is the set of words accepted by the automaton M. A language K is regular if there exists a finite state automaton M such that K = L(M).For more information we refer to [11].
Consider a group G generated (as a monoid) by a set A. The set of words in A * that represent the identity in G is a language W (G), called the word problem of G, and determines G.It is a still active area of research to relate the complexity of W (G), as a language, with structural properties of G.The first result in this direction is due to Anisimov, who in 1970 proved that W (G) is a regular language if and only if the group G is finite [2].
The group G = A may also be understood via a normal form for its elements: a language N (G) ⊆ A * that is in bijection with G under the natural evaluation map; for example, the free Abelian group Z 2 can be represented by the regular language ({x} * ∪ {x −1 } * )({y} * ∪ {y −1 } * ).
Automata may also be used to represent relations: adding an extra padding symbol ⋄ to A, one may represent a relation R ⊆ A * × A * by its language {(u⋄ max(0,|v|−|u|) , v⋄ max(0,|u|−|v|) : (u, v) ∈ R}, and the relation R is called regular if its language is regular as a subset of ((A⊔{⋄})×(A⊔ {⋄})) * .The rich class of automatic groups are those groups G = A admitting a regular normal form N ⊆ A * and for which the multiplication relations {(u, v) ∈ N 2 : ua = G v} are regular for all a ∈ A; see [9].Kharlampovich, Khoussainov, and Miasnikov introduced Cayley-automatic groups in [12] as groups admitting a regular normal form (not necessarily coming from words over a generating set) for their elements and the group operation.They prove that this class of groups strictly extends that of automatic groups, while retaining some of the most important algorithmic properties (such as fast solvability of the word problem).In this paper, the groups we are interested in shall not admit regular normal forms in the above sense; but rather as "tree languages", see below.
The theory of regular languages has been extended to infinite words by Büchi, see [6].In contrast to classical finite automata, a Büchi machine is a finite state automaton that takes infinite words as input and and accepts a run if the set of recurring states intersects the set of accepting states.An ω-regular language is a language recognized by a Büchi machine.The class of ω-regular languages is closed under all boolean operations; this was used to prove decidability of Pressburger arithmetic [5].
The theory was further extended to tree languages.Finite words are seen as A-labels on the vertices of a finite path, and ω-words are A-labelled infinite paths.In this vein, tree words are A-labelled rooted trees (infinite in our case), and a tree automaton is a finite machine accepting certain tree words.It reads trees starting from the root, one node at a time, and clones itself to process independently and in parallel all descendant subtrees.These tools are at the heart of Rabin's proof [15] that the monadic secondorder logic of two successor functions (S2S) is decidable.
1.2.Self-similar groups.Finite state automata also appear in group theory as representations of the actual elements of a group.For an alphabet X, the set of words X * naturally forms the vertex set of a regular rooted tree T , rooted at the empty word and with an edge between u and ux for all u ∈ X * , x ∈ X.Consider a group G acting by tree isometries on T , and g ∈ G. Then the action of g on T is entirely characterized by its action σ g on X, the neighbours of the root, and for each x ∈ X the induced bijection between the subtrees xX * and σ g (x)X * .Identifying these subtrees with T , we obtain a permutation σ g and a collection (g@x) x∈X of tree isometries called the states of g.If g@x ∈ G for all x ∈ X, we call the group G self-similar.In this case, the action of G on T is conveniently recorded by a homomorphism ψ : G → G ≀ X, defined by g → ((g@x), σ g ), where G ≀ X stands for the permutational wreath product G X ⋊ Sym(X).
Furthermore, the operations g → g@x may be composed; for v = ux ∈ X * we write g@v = (g@u)@x.If g is such that {g@v : v ∈ X * } is finite, then g may be represented by a finite automaton with alphabet X × X: its stateset is Q = {g@v : v ∈ X * } and it has for all h ∈ Q, x ∈ X a transition from h to h@x with label (x, σ h (x)).As an example, consider X = {0, 1}, the stateset Q = {a, b, c, d, e}, and the transitions Every choice of initial state q ∈ Q yields a relation R q ⊂ X * × X * which is the graph of a permutation of X * ; the group generated by these permutations is known as the (first) Grigorchuk group [10] and possesses a wealth of striking properties: it is a finitely generated torsion group which is infinite but all its proper quotients are finite; it is a group of intermediate word growth; it is amenable but not elementarily amenable, . . . .The Grigorchuk group and the Gupta-Sidki p-groups are instances of GGS-groups, and provide a large family of counterexamples to the General Burnside Problem.
Important subclasses of groups acting on trees have emerged: recurrent groups are those for which the map 'g → g@v' is surjective from the stabilizer of v ∈ T onto G. Level-transitive groups are those that act transitively on X n for all n ∈ N. Loosely speaking, a branched groups is a group whose lattice of subnormal subgroups is similar to the corresponding structure in the full isometry group Aut T of the tree T .For a self-similar group G, there is a stronger notion: G is regularly branched if there is a finite-index subgroup K such that G contains, for all v ∈ T , a copy of K acting only on the subtree vX * .The aforementioned examples of the Grigorchuk and Gupta-Sikdi torsion groups are recurrent, level-transitive and regularly branched.
Every f ∈ Aut T can be described by providing, at every vertex v of the tree, a permutation σ f @v ∈ Sym(X) that describes how f acts on the children of v.The portrait of f is this Sym(X)-labelling v → σ f @v of T , and there is a one-to-one correspondence between isometries of T and portraits.
The relations between a self-similar group G and the collection of portraits of its elements have already been considered in the literature, in particular by Siegenthaler in [16] and his 2008 doctoral thesis; see also [14,18,19].In that last article, Penland and Šunić prove that the closures of certain self-similar groups of rooted trees that satisfy an algebraic law do not have a regular language of portraits.
1.3.Main results and their consequences.This paper's main results relate tree languages with group-theoretical properties of branched groups.
Theorem A (= Theorem 10).Let G be a level-transitive, recurrent, closed self-similar subgroup of Aut T .Then G is regularly branched if and only if its set of portraits is a tree regular language.
We point out that in the "if" direction of the theorem above, we do not need that G is level-transitive.
Theorem B (= Theorem 12).Let G be a contracting regular branched group.Then its set of portraits is a tree regular language.
Theorems A and B solve some algorithmic decidability problems in regularly branched groups: Corollary C. Let G, H ≤ Aut T be two contracting regularly branched groups; then there exists an algorithm that determines whether For a group G acting on a tree X * and therefore on its boundary X ω , the orbit relation is the equivalence relation Corollary D. Let G ≤ Aut T be a contracting, regularly branched group; then its orbit relation is an ω-regular language, and is computable.
The Hausdorff dimension of profinite groups was first considered by Abercrombie in [1].In our situation of a closed group G acting on a rooted tree X * , its Hausdorff dimension is computed as follows: let G n be the natural quotient of G acting on X n ; then Corollary E. Let G ≤ Aut T be a closed self-similar regularly branched group; then the Hausdorff dimension of G is computable.
The paper is organized as follows: in Section 2 we define groups acting on trees, with a particular emphasis on branched groups.Moreover, we consider branched groups in the more abstract setting of bisets.In Section 3, we give basic notions about tree languages, and in the last Section 4 we prove theorems A and B together with their corollaries.

Branched groups
In this Section we briefly summarize and recall the most important notions in the theory of groups of isometries of rooted trees and, more precisely, of branched groups.For more information on the topic see, for example, [3,4,8].
2.1.Groups of isometries of rooted trees.Let T be a d-regular rooted tree: an oriented tree with a designated vertex called the root, and such that every vertex has d immediate descendants and one immediate ancestor (except the root which has no ancestor).
Let X be a finite alphabet with d elements, X n the set of all words of length n and X * = n∈N X n ; then the vertex set of T may be identified with X * .We denote by ǫ the empty word, namely the root of the tree.The descendants of u ∈ X n are all words ux ∈ X n+1 , for x ∈ X.For every vertex u ∈ X * , the subtree of T hanging from u has vertex set uX * and is naturally identified with T via uw ↔ w.Let Aut T denote the group of bijective maps of the set X * which preserve incidence, so are isometries of the graph T .Let Sym(X) be the set of all permutations over the alphabet X. Considering separately the action of Aut T on X and on each subtree xX * for x ∈ X, we obtain an isomorphism decomposing every isometry into a permutation of X and a tuple of elements of Aut T .For g ∈ Aut T , we write ψ(g) 1 , respectively ψ(g) 2 the elements of Aut T X , respectively Sym(X); then ψ(g) 1 (x) is the map v → xv g → g(xv) = yw → w, and ψ(g) 2 is the map Definition 1 (Portraits).Anticipating §3, a tree word is a map T → A for some alphabet A. The portrait of g ∈ Aut T is the tree word p g : T → Sym(X) defined inductively by p g (ǫ) = ψ(g) 2 and p g (xv) = p ψ(g)1(x) (v).The portrait map induces a homeomorphism between Aut T and the set of all tree words T → Sym(X).
When considering subgroups G ≤ Aut T , the following definitions clarify additional properties of the restriction to G of the map ψ : Aut T → Aut T ≀ X.
Definition 2 (Self-similar and recurrent groups).Consider G ≤ Aut T .It is selfsimilar if ψ restricts to a map G → G ≀ X, and recurrent if furthermore for every g ∈ G, v ∈ X * there exists h ∈ G with h(v) = v and h@v = g.
For g ∈ Aut T and v ∈ X * , the state of g at v, written g@v, is defined inductively by g@ǫ = g and g@(xv) = ψ(g) 1 (x)@v.Then G is self-similar precisely when g@v ∈ G for all g ∈ G, v ∈ X * , and is recurrent if The portrait of g ∈ Aut T may then be defined directly by p g (v) = σ g@v .Note that if G is self-similar then its set of portraits is a "differentially closed" language, namely a language L such that for every w ∈ L and every x ∈ X one has ∂w/∂x ∈ L, where (∂w/∂x)(v) = w(xv).
Definition 3 (Level-transitive groups).A group G acting on T is level-transitive if it acts transitively on X n for all n ∈ N; so the quotient of T by the action is a single ray.Definition 4 (Contracting groups).A self-similar group G ≤ Aut T is contracting if there exists a finite subset F ⊆ G such that for every g ∈ G the state g@v belongs to F for all v ∈ X * except finitely many.There is a minimal such set F called the nucleus of G, and denoted by N .
2.2.Branched groups and branch structure.In this section we describe one of the most important classes of groups of isometries of rooted trees: branched groups.
Definition 5 (Regularly branched groups).A group G acting on T is regularly branched if there exists a finite index subgroup K ≤ G with K X ≤ ψ(K).
This definition can be conveniently expressed as the existence of a branch structure: Indeed, with K be as above assumed without loss of generality normal, set

Branched bisets.
There exists a formalism of groups acting on trees that does not require fixing a model X * for the regular rooted tree: it is that of bisets.A G-G-biset is a set B equipped with two commuting G-actions, written on the left and the right respectively.A covering biset is a biset such that the right action is free, so one may write B = X • G when only considering the right action, by choosing a set of representatives X ⊆ B of the right action.This set X is called a basis of B. The left action is then expressed as a map G × X → X × G.
If a group G acts self-similarly on a tree T = X * , its associated biset is B = X × G with left action given by g • (x, h) = (ψ(g) 2 (x), ψ(g) 1 (x)h).In other words, the map ψ : G → G ≀ X induces naturally a map G → (X × G) X and therefore a map G × X → X × G, which precisely defines the structure of a covering biset.
Conversely, given covering G-G-bisets B and C, define their product B ⊗ G C = (B × C)/{(bg, c) = (b, gc)}, and note that it is also a covering biset.More precisely, if Define the Fock tree of B as T = n≥0 B ⊗n /G, with natural left G-action.Note that if B ∼ = X × G then T ∼ = X * .We stress the point that B determines canonically an action of G on T , but not on X * which requires a choice of identification B ∼ = X × G.In fact, two different choices yield conjugate actions, but conjugate actions need not correspond to isomorphic bisets.

Definition 6 (Regularly branched bisets).
A G-G-biset B is regularly branched if there exists a finite index subgroup K ≤ G and a basis X of B such that, for every (k x ) ∈ K X , there exists k ∈ K with xk x = kx for all x ∈ X. Lemma 7. The following hold: (i) If G acts on X * and is regularly branched, then its associated biset is regularly branched.(ii) If B is regularly branched, then for every identification B ∼ = X × G the action on X * is regularly branched.
Proof.(i) Since G is regularly branched, there exists a finite index subgroup K ≤ G with ψ(K X ) ≤ K. Given now (k x ) ∈ K X , set k = ψ(k x ); we have kx = xk x as required.
(ii) Assume B is regularly branched, for the subgroup K which without loss of generality we suppose normal, and the basis X.Consider another basis Y .Then there exists a bijection π : X → Y and group elements (g x ) ∈ G X such that x = π(x)g x holds for all x ∈ X.Given now (k y ) ∈ K Y , note that (k gx π(x) ) ∈ K X because K is normal.Let k ∈ K be such that xk gx π(x) = kx for all x ∈ X, and compute, for y = π(x), so the "there exists a basis" in the definition of "B is regularly branched" can be changed to a "for every basis".
In this section we have merely noted that the property of being regularly branched is independent of the choice of basis, so may be defined for bisets rather than tree actions.However, it would be more satisfying to express the definition more intrinsically, without any appeal to a basis.It could also be that the definition diverges from that of tree actions in case the group G does not act faithfully.We leave these questions open for further investigation; the general point being that a biset should be branched if and only if the left action has finitely many orbits on bases.

Automata on infinite trees
In this section we define automata on infinite trees, and defer to [7] for a thorough introduction on automata on trees and tree languages.
Recall that an alphabet is a finite set, and a regular tree is a set of the form T = X * for an alphabet X.Its vertices are thus finite sequences (X * ), and its boundary consists of infinite sequences (X ω ) called rays.For example, if X = {t}, the corresponding regular tree is geometrically an infinite half-line, with vertex set {t n : n ∈ N}, and it has a single ray t ω .
A tree word is a map w : T → A, for a regular tree T and an alphabet A. For example, if T = {t} * , a tree word is a usual ω-word.As we saw in Definition 1, the portrait of a tree isometry is a tree word, with A = Sym(X).
A tree language is a set of tree words, all with the same tree and alphabet; so again if a group acts on a tree T = X * then the set of portraits of its elements is a tree language with alphabet Sym(X).

Definition 8 (Müller automata). A Müller tree automaton on the tree
• Q is a finite (non-empty) set of states; • A is an alphabet; A run of M is a tree word w with alphabet δ, such that w(ǫ) 1 ∈ Q i and for every v ∈ X * one has w(v) 3 (x) = w(vx) 1 and for every ray ξ ∈ X ω one has {w(v) 1 : v prefix of ξ} ∈ Q f .In words, the run must start at an initial state, the states match along edges according to some rules in δ, and the set of states seen along any ray is residual.The value of a run w is the tree word w with alphabet A defined by w(v) = w(v) 2 .A tree word T → A is accepted by M if it is the value of a run of M (see [13]).Definition 9 (Regular tree languages).A tree language is regular if it is the set of accepted tree words of some Müller tree automaton.
We note the fundamental (and non-trivial) fact that the class of languages definable by non-deterministic Müller tree automata is closed under all boolean operations.

Main results
Here we prove Theorem A and B together with some applications.
Theorem 10.Let G be a level-transitive, recurrent, closed self-similar subgroup of Aut(X * ).Then G is regularly branched if and only if its set of portraits is a regular language.
Proof.In the first direction, suppose G is regularly branched, and let (π : a branch structure.Consider the following Müller automaton M: • Q (defined above) is the set of states; (q, a, (q x )) ∈ δ whenever ((q x ), a) defines an element q 1 ∈ Q 1 and φ(q 1 ) = q; Given g ∈ G, construct a run w of M by w(v) = (π(g@v), σ(g@v), (π(g@vx)) x∈X ), and note that its value is the portrait of g.Conversely, given a run w of M, note that its value defines a portrait and therefore an isometry t of X * .Furthermore, the run defines an element w(ǫ) 1 ∈ Q so there exists g ∈ G such that g, t have same image in Q.Now the construction of M implies that for every v ∈ X * there exists obtained by assembling q 1,v , (q 1,vx ), and a lift g 2 ∈ G of q 2,ǫ such that g 2 , t have same image in Q 2 .In this way, we obtain a sequence of elements g n ∈ G such that g n , t have same image in Q n .Since the Q n converge to G and G is closed, we get t = lim g n ∈ G.
Note that we did not use the hypotheses that G be level-transitive nor recurrent.
In the other direction: assume that the language of portraits of the self-similar group G is a regular language, and let M be a Müller machine recognizing it, with stateset Q.Without loss of generality, assume M is trim: every state can be reached from an initial state, and leads to a residual set.For every g ∈ G, let Q(g) denote the set of initial states of runs accepting the portrait of g, noting that Q(g) is non-empty for all g ∈ G. Let K 0 denote the subgroup of G generated by {gh −1 : Q We first note that K 0 is a finite index subgroup of G; more precisely, the index of K 0 is at most #Q.Indeed, let g 0 , g 1 , . . ., g #Q be arbitrary elements of G. Then the sets Q(g 0 ), . . ., Q(g #Q ) cannot all be disjoint, since they are non-empty and more numerous than Q.Thus there exists i and j such that Q(g i ) and Q(g j ) intersect, so g i g −1 j ∈ K 0 , and so g 0 , . . ., g #Q cannot be left coset representatives of K 0 .
Consider next k = gh −1 ∈ K 0 , and choose q ∈ Q(g) ∩ Q(h).Since M is trim, there exists an element f ∈ G and a run w accepting f which reaches state q at some vertex v ∈ X * .Construct two runs w g , w h by replacing in w the subtree at v respectively by the runs accepting g, h in initial state q.These runs define respectively elements f g , f h ∈ G, both starting at the same initial state r ∈ Q i .Therefore Q(f g ) ∩ Q(f h ) = ∅, and thus f g f −1 h ∈ K 0 .Furthermore, f g , f h agree everywhere except on the subtree at v, so f g f −1 h acts as k on that subtree and trivially elsewhere.Thus, for every generator k of K 0 , we have constructed a vertex v and an element ℓ ∈ K 0 with ℓ@v = k and fixing the complement of vX * .
We now let K denote the normal closure of K 0 in G, and continue our reasoning with ℓ@v = k.Write |v| = n, and note n ≤ #Q since we may have chosen a simple path from an initial state to q in the construction of the run w.Since G is recurrent and acts transitively on X n , we obtain for arbitrary u ∈ X n , g ∈ G an element ℓ u,k ∈ K acting as k g on the subtree uX * and fixing its complement.
Note that the choice of parameter n depended only on the state q, so there exists a common multiple of all such choices, call it m.We deduce that K X m is a subgroup of K, and G is regularly branched with branching subgroup K, albeit on the tree (X m ) * obtained by coalescing m levels into one.
The following lemma will be useful for the proof of Theorem B. Lemma 11.Let g x ∈ G and a ∈ Sym(X) such that there exists g ∈ G with π(ψ 1 (g)(x)) = π(g x ).Then there exists g ′ ∈ G with ψ 1 (g ′ )(x) = g x .
Theorem 12.If G is a contracting regularly branched group, then its set of portraits is a regular language.
Proof.We modify the construction in the proof of Theorem 10 as follows.Let N be the nucleus of G.As stateset, we use We extend the transitions δ to δ ′ by allowing transitions from Q ′ to N .More precisely, we add to δ for all g ∈ G the transitions (π(g), σ g , (g@x) x∈X ) whenever g@x ∈ N for all x ∈ X, and for all g ∈ N the transitions (g, σ g , (g@x) x∈X ).Thus an accepting run must reach state N on every ray, and so by König's lemma may have only finitely many states in Q ′ \ N .
On the one hand, it is clear that every g ∈ G is accepted, since almost all states of g belong to N .Conversely, consider an accepting run.It yields by restriction a finite subtree of X * with elements of N at its leaves and elements of Q A at its internal vertices.We construct by induction an element of G from it.If the subtree has a single vertex, then it is labelled by an element of N and therefore directly produces an element of G. Otherwise, consider an internal vertex of maximal height, so all its descendants are labelled by elements of G.We may replace the internal vertex's label by an element of G, and proceed by induction, owing to Lemma 11.
Finally we list some consequences of Theorems A and B. These all pertain to decidability questions.
When given a contracting regularly branched group, the data defining it are assumed to be in the form of a finite generating set S, the restriction of ψ to S, and an oracle promising that the action is faithful, contracting and regularly branched.
Corollary 13.Let G, H ≤ Aut T be two contracting regularly branched groups.Then there exists an algorithm that determines whether G ≤ H.As a consequence, it is decidable if G = H.
Proof.Suppose that there exists an oracle that recognises whether a group is contracting and branched.Then one can algorithmically determine its nucleus and its branching subgroup.Now given two contracting regular branched groups G and H, by Theorem 12 their languages of portraits are regular, and the inclusion in regular languages is decidable.This completes the proof.
On the other hand, a closed regularly branched group is determined by the finite data of its branched structure.
Corollary 14.Let G ≤ Aut T be a closed self-similar regularly branched group.Then the Hausdorff dimension of G is computable.
Proof.By Theorem 10, there exists a tree automaton that recognizes elements of the portrait of a closed group G.The Hausdorff dimension of G is the entropy of its language of portraits, so is computable from the automaton.
More directly, a linear system of equations can easily be set up, one variable per state of the automaton, whose solution yields the Hausdorff dimension of G, see [17].
We finally to turn to the orbit structure of contracting, regularly branched groups.Say the group G acts on the regular tree T = X * ; so its acts on its boundary ∂T = X ω .The orbit relation is the subset Corollary 15.Let G ≤ Aut T be a contracting, regularly branched group.Then the equivalence relation O is an ω-regular language, and is computable.
Then every run of M, recognizing an element g ∈ G leads to runs, for all ξ ∈ X ω , of M ′ recognizing (ξ, gξ); thus the language of M ′ is O.
In particular, for every preperiodic ray ξ, the orbit of ξ is computable as an ω-regular language ⊆ X ω , by intersecting O with the ω-regular language {ξ} × X ω and projecting to the second coordinate.