Hecke Triangle Groups, Transfer Operators and Hausdorff Dimension

Abstract

We consider the family of Hecke triangle groups $\Gamma _{w} = \langle S, T_w\rangle$ generated by the Möbius transformations $S : z\mapsto -1/z$ and $T_{w} : z \mapsto z+w$ with $w > 2.$ In this case, the corresponding hyperbolic quotient $\Gamma _{w}\backslash {\mathbb {H}}^2$ is an infinite-area orbifold. Moreover, the limit set of $\Gamma _w$ is a Cantor-like fractal whose Hausdorff dimension we denote by $\delta (w).$ The first result of this paper asserts that the twisted Selberg zeta function $Z_{\Gamma _{ w}}(s, \rho )$, where $\rho : \Gamma _{w} \rightarrow \mathrm {U}(V)$ is an arbitrary finite-dimensional unitary representation, can be realized as the Fredholm determinant of a Mayer-type transfer operator. This result has a number of applications. We study the distribution of the zeros in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$ of the Selberg zeta function of a special family of subgroups $( \Gamma _w^N )_{N\in {\mathbb {N}}}$ of $\Gamma _w$. These zeros correspond to the eigenvalues of the Laplacian on the associated hyperbolic surfaces $X_w^N = \Gamma _w^N \backslash {\mathbb {H}}^2$. We show that the classical Selberg zeta function $Z_{\Gamma _w}(s)$ can be approximated by determinants of finite matrices whose entries are explicitly given in terms of the Riemann zeta function. Moreover, we prove an asymptotic expansion for the Hausdorff dimension $\delta (w)$ as $w\rightarrow \infty$.

1 Introduction

In [16], Hecke introduced the one-parameter family of subgroups $\Gamma _{w} = \langle S, T_w\rangle$ of $\mathrm {PSL}_2({\mathbb {R}})= \mathrm {SL}_{2}({\mathbb {R}})/\{ \pm {{\,\mathrm{id}\,}}\}$ generated the elements

$$\begin{aligned} T_w = \begin{bmatrix} 1&{}\quad w\\ 0&{}\quad 1\end{bmatrix}\quad \text {and}\quad S =\begin{bmatrix} 0&{}\quad 1\\ 1&{}\quad 0\end{bmatrix}, \end{aligned}$$

and their inverses, where w is a positive real number. On the hyperbolic plane

$$\begin{aligned} {\mathbb {H}}^2 = \{ z = x+iy : x\in {\mathbb {R}},\, y\in {\mathbb {R}}_{>0}\}, \end{aligned}$$

these elements act by the Möbius transformations $S : z\mapsto -1/z$ and $T_{w} : z \mapsto z+w.$ The groups $\Gamma _{w}$, which came to be known as the ‘Hecke triangle groups,’ naturally generalize the well-known modular group

$$\begin{aligned} \mathrm {PSL}_{2}({\mathbb {Z}}) = \left\{ \begin{bmatrix} a&{}\quad b\\ c&{}\quad d\end{bmatrix}\in {\mathbb {Z}}^{2 \times 2} : ad-bc=1 \right\} , \end{aligned}$$

which corresponds to the case $w = 1.$ Hecke showed that $\Gamma _w$ is a Fuchsian group, that is, a discrete subgroup of $\mathrm {PSL}_2({\mathbb {R}})$, if and only if $w = 2\cos (\pi /q)$ for integer $q\ge 3$ or $w \ge 2.$ Moreover, the set

$$\begin{aligned} {\mathscr {F}}(w) = \left\{ z\in {\mathbb {H}}^2 : \vert \mathrm {Re}(z)\vert < \frac{w}{2}, \; \vert z\vert > 1 \right\} \end{aligned}$$

(1)

provides a fundamental domain for the action of $\Gamma _{w}$ on ${\mathbb {H}}^2$, see Fig. (1).

Fig. 1

Fundamental domain ${\mathscr {F}}(w)$ for $\Gamma _{w}$ with $w > 2$

In the present paper, we will restrict our attention to the case $w > 2$. In this case, the quotient $\Gamma _{w}\backslash {\mathbb {H}}^2$ is an infinite-area hyperbolic orbifold with one cusp, one funnel and one conical singularity¹. In particular, the limit set $\Lambda (\Gamma _{w})$ of $\Gamma _{w}$ is a Cantor-like fractal whose Hausdorff dimension we denote by $\delta (w)$. The Hausdorff dimension of limit sets of Fuchsian groups plays a profound role in the spectral theory of hyperbolic surfaces. From Patterson’s work [33], we know that $\delta (w)$ is the exponent of convergence of the Poincaré series for $\Gamma _{w}$ and that the base eigenvalue of the Laplacian on $\Gamma _{w} \backslash {\mathbb {H}}^2$ is given by $\delta (w) (1-\delta (w)).$ The properties of $\delta (w)$ have been studied by several authors [4, 33,34,35,36]. It is known from these papers that

$$\begin{aligned} \delta (w) > \frac{1}{2}, \quad \delta (2) = 1, \quad \lim _{w\rightarrow \infty }\delta (w) = \frac{1}{2}, \end{aligned}$$

(2)

that $w\mapsto \delta (w)$ is a strictly decreasing Lipschitz continuous function on $[2, \infty ),$ and that the base eigenvalue $\delta (w)(1-\delta (w))$ is analytic and concave as a function of $w\in [2, \infty )$.

Apart from the Hausdorff dimension, we are interested in the Selberg zeta function of $\Gamma _{w}$ twisted by arbitrary finite-dimensional unitary representations $\rho :\Gamma _{w} \rightarrow \mathrm {U}(V)$. It is defined formally by the infinite Euler product

$$\begin{aligned} Z_{\Gamma _{w}}(s, \rho ) = \prod _{[\gamma ]} \prod _{k=0}^{\infty } \mathrm {det}_{V}\left( 1_V- \rho (\gamma )e^{-(s+k)\ell (\gamma )} \right) , \end{aligned}$$

(3)

where $[\gamma ]$ runs over the conjugacy classes of primitive hyperbolic elements of $\Gamma _{w}$ and $\ell (\gamma )$ is the displacement length of $\gamma$. This product converges absolutely in the half-plane $\mathrm {Re}(s) > \delta (w)$ (see Sect. 2.1). Notice that (3) reduces to the classical Selberg zeta function when $\rho = \mathbf{1 }$ is the trivial one-dimensional representation.

Our first main result asserts that $Z_{\Gamma _{w}}(s, \rho )$ can be realized as the Fredholm determinant of a well-chosen family of transfer operators.

Theorem 1.1

Fix $w > 2$, let $\Gamma _{w} = \langle S, T_w\rangle$ be the corresponding Hecke triangle group, and let $\rho :\Gamma _{ w}\rightarrow \mathrm {U}(V)$ be a unitary representation with finite-dimensional representation space V. Let ${\mathbb {D}}$ be the open unit disk of the complex plane and consider the operator ${\mathscr {L}}_{s,w,\rho }$ acting on functions $f:{\mathbb {D}}\rightarrow V$ via

$$\begin{aligned} {\mathscr {L}}_{s,w,\rho } f(z) = \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1} f\left( \gamma _n(z) \right) , \quad z\in {\mathbb {D}}, \end{aligned}$$

(4)

where $\gamma _n \mathrel {\mathop :}=S T_w^n.$ Then, for all $s\in {\mathbb {C}}$ with $\mathrm {Re}(s) >\frac{1}{2}$, equation (4) defines a trace-class operator

$$\begin{aligned} {\mathscr {L}}_{s,w,\rho } :H^2({\mathbb {D}}; V)\rightarrow H^2( {\mathbb {D}} ; V) \end{aligned}$$

(5)

(see Sect. 2.3 for more details). Moreover, the twisted Selberg zeta function is represented by the Fredholm determinant of (5), that is,

$$\begin{aligned} Z_{\Gamma _{w}}(s, \rho ) = \det \left( 1- {\mathscr {L}}_{s, w,\rho } \right) \end{aligned}$$

(6)

for all $\mathrm {Re}(s)>\delta (w).$

Remark 1.2

Identities such as (6) are well known in the thermodynamic formalism, a subject going back to Ruelle [41]. The relation between the Selberg zeta function and transfer operators has been studied by a number of different authors. For the convex co-compact setting (no cusps), we refer to [14, 39, 40]. In the presence of cusps, the first example of an identity in the spirit of (6) was given by Mayer [25] for the modular group $\Gamma _1 = {{\,\mathrm{PSL}\,}}_2({\mathbb {Z}})$ and for the trivial twist $\rho = \mathbf{1 }$. Hecke triangle groups (cofinite and non-cofinite) have been studied extensively in Pohl [37, 38], where a version of (6) has been proven using Poincaré sections of the geodesic flow and different transfer operators. Our proof relies solely on certain combinatorial features of the group $\Gamma _{w}$ and is reminiscent of the method of Lewis–Zagier [23] for the modular group. Related work includes [10, 11, 26,27,28].

The representation of the Selberg zeta functions in terms of transfer operators has proven to be a powerful tool in the spectral theory of infinite-area hyperbolic surfaces, a subject not yet fully explored. For instance, transfer operator techniques have been implemented in [21] to construct hyperbolic surfaces with arbitrarily small ‘spectral gap.’ In [14, 31], transfer operators have been used to prove fractal Weyl bounds for resonances of the Laplacian on hyperbolic surfaces, analogous to Sjöstrands pioneering work [46] on semi-classical Schrödinger operators. Related works where the thermodynamic formalism plays an essential role include [8, 18, 20, 29, 30, 32]. Another application of Fredholm determinant identities such as (6) is a simple proof of the meromorphic continuation of the twisted Selberg zeta function, which is far from obvious from its definition in (3) as an infinite product over primitive conjugacy classes. Theorem 1.1 gives a new proof of the following result:

Corollary 1.3

Assumptions being as in Theorem 1.1, $Z_{\Gamma _{w}}(s, \rho )$ admits a meromorphic continuation to $s\in {\mathbb {C}}$ and all its poles are contained in $\frac{1}{2}(1-{\mathbb {N}}_0).$

In this paper, we give additional applications of Theorem 1.1. For applications to spectral theory, it is sometimes more convenient to work with torsion-free Fuchsian groups $\Gamma$ in which case the quotient $\Gamma \backslash {\mathbb {H}}^2$ is a smooth surface². Selberg’s lemma [43] says that every finitely generated Fuchsian group has a finite-index, torsion-free subgroup. In the case of Hecke triangle groups, there is a simple way of manufacturing such a subgroup $\Gamma _w^1 \subset \Gamma _w$. Indeed, let $\rho :\Gamma _{w}\rightarrow {\mathbb {C}}^{\times }$ be the one-dimensional representation defined by $\rho (T_{w}) = 1$ and $\rho (S) = -1$, and set $\Gamma _{w}^{1} = \ker (\rho )$. The group $\Gamma _{w}^{1}$ is a normal subgroup of $\Gamma _{w}$ (being the kernel of a homomorphism) and it is freely generated³ by the elements

$$\begin{aligned} T_w^\pm = \begin{bmatrix} 1&{}\quad \pm w\\ 0&{}\quad 1\end{bmatrix}\quad \text {and}\quad R_w^{\pm } \mathrel {\mathop :}=S T_{w}^\pm S =\begin{bmatrix} 1&{}\quad 0\\ \mp w&{}\quad 1\end{bmatrix}. \end{aligned}$$

(7)

In particular, $\Gamma _{w}^{1}$ contains no elliptic elements and it is therefore torsion-free. Moreover, we have

$$\begin{aligned} \Gamma _{w}/\Gamma _{w}^{1} \simeq \{ \mathrm {id}, S \} \simeq {\mathbb {Z}}/2{\mathbb {Z}}, \end{aligned}$$

so the action of $\Gamma _{w}^{1}$ on ${\mathbb {H}}^2$ has the fundamental domain

$$\begin{aligned} {\mathscr {F}}^{1}(w) = {\mathscr {F}}(w) \cup S .{\mathscr {F}}(w), \end{aligned}$$

(8)

where ${\mathscr {F}}(w)$ is the fundamental domain of $\Gamma _w$ given in (1), see Fig. 2.

Fig. 2

Fundamental domain ${\mathscr {F}}^{1}(w)$ for $\Gamma _{w}^1$ with $w > 2$

It follows that the associated hyperbolic quotient $X_w^1= \Gamma _w^1\backslash {\mathbb {H}}^2$ is a smooth 2-cover of $X_w$. More generally, for every positive integer N, we can define a family of torsion-free subgroups

$$\begin{aligned} \Gamma _w^N = \ker ( \rho _N ) \end{aligned}$$

as the kernel of the representation $\rho _N:\Gamma _w \rightarrow {\mathbb {C}}^\times$ given by

$$\begin{aligned} \rho _N (S) = -1 \quad \text {and} \quad \rho _N(T_w) = e^{\frac{2\pi i}{N}}. \end{aligned}$$

The corresponding quotients $X_w^N = \Gamma _w^N\backslash {\mathbb {H}}^2$ are simultaneous covers of both $X_w$ and $X_w^1$. The associated covering groups can be shown to be isomorphic to

$$\begin{aligned} \Gamma _w/\Gamma _w^N \simeq {\mathbb {Z}}/2N{\mathbb {Z}} \quad \text {and} \quad \Gamma _w^1/\Gamma _w^N \simeq {\mathbb {Z}}/N{\mathbb {Z}}, \end{aligned}$$

respectively. In particular, $X_w^N$ is a smooth, abelian 2N-covering of the Hecke orbifold $X_w = \Gamma _w \backslash {\mathbb {H}}^2$. We can now formulate our next theorem.

Theorem 1.4

Let $w>2$. In what follows, we identify the functions $Z_{\Gamma _w}(s)$ and $Z_{\Gamma _w^N}(s)$ with their meromorphic continuations and their zeros are counted with multiplicities. Then, the following statements hold true:

• (i) $Z_{\Gamma _w}(s)$ has exactly one zero in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$, namely at $s=\delta (w)$,

• (ii) for every positive integer N, $Z_{\Gamma ^N_w}(s)$ has at most N zeros in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$. In particular, the number of $L^2$-eigenvalues of the positive Laplacian on $X_w^N$ is at most N, and

• (iii) for every $\varepsilon > 0$ there exists a constant $c=c(\varepsilon ,w)>0$ such that for every N, $Z_{\Gamma _w^N}(s)$ has at least cN zeros in the interval

  $$\begin{aligned} (\delta (w) - \varepsilon , \delta (w)]. \end{aligned}$$

  In particular, for every $\varepsilon ' > 0$ there exists $c'= c'(\varepsilon ',w) > 0$ such that the positive Laplacian on $X_w^N$ has at least $c' N$ $L^2$-eigenvalues in

  $$\begin{aligned} {}[\lambda _0(w), \lambda _0(w)+\varepsilon '), \end{aligned}$$

  where $\lambda _0(w) = \delta (w)(1-\delta (w))$ is the common base eigenvalue of the surfaces $X_w^N.$

Remark 1.5

Part (i) of Theorem 1.4 should be compared with [34, Theorem 6.1], which states that for all $w > 2$ the base eigenvalue $\delta (w)(1-\delta (w))$ is the only Laplace eigenvalue for the Hecke orbifold $X_w$. From Borthwick–Judge–Perry [7] we know that if $\Gamma$ is a finitely generated, torsion-free Fuchsian group, then the zeros of $Z_{\Gamma }(s)$ in the half-plane $\mathrm {Re}(s) >\frac{1}{2}$ correspond to the $L^2$-eigenvalues $s(1-s)$ of the Laplacian on $\Gamma \backslash {\mathbb {H}}^2$. Thus [34, Theorem 6.1] is morally equivalent to Part (i) of Theorem 1.4. Unfortunately, the result of Borthwick–Judge–Perry does not apply directly to any of the groups $\Gamma _{w}$, since they contain the elliptic element S. Nevertheless, one should expect the zeros of $Z_{\Gamma _{w}}(s)$ to have a similar interpretation in terms of eigenvalues of the Laplacian (though the author is not aware of such a result in the literature).

Remark 1.6

Part (iii) of Theorem 1.4 says that on large abelian covers of $X_w$, the Laplacian possesses a large number of eigenvalues arbitrarily close to base eigenvalue $\lambda _0(w).$ Similar results were proven for the modular surface $X_1$ by Selberg [44, paper 33, p. 12] and for compact hyperbolic surfaces by Randol [9], both using completely different methods. More recently, using transfer operator techniques, a similar (and more precise) result was established for convex co-compact surfaces by Jakobson, Naud and the author in [21]. Part (iii) of Theorem 1.4 seems to be the first result of this kind for surfaces which are neither cofinite, nor convex cocompact.

The next result shows that the classical Selberg zeta function $Z_{\Gamma _w}(s)$ can be approximated by determinants of $k\times k$-matrices, up to an error that tends to zero exponentially fast as $k\rightarrow \infty$. More concretely, we have

Theorem 1.7

For every positive integer k let $A_k(s,w) = \left( a_{i,j}(s,w) \right) _{0\le i,j < k}$ be the matrix given by

$$\begin{aligned} a_{i,j}(s,w) = \left( (-1)^{i+j} + 1 \right) \frac{\zeta (2s+i+j)}{w^{2s+i+j}} { 2s+i+j-1 \atopwithdelims ()i}, \end{aligned}$$

where $\zeta$ denotes the Riemann zeta function, and let $D_k(s,w)$ be the determinant

$$\begin{aligned} D_k(s,w) \mathrel {\mathop :}=\det (I_k-A_k(s,w)), \end{aligned}$$

where $I_k$ denotes the identity matrix of size k. Fix $w>2$ and $s\in {\mathbb {C}}$ with $\mathrm {Re}(s) > \frac{1}{2},$ and identify $Z_{\Gamma _w}(s)$ with its meromorphic continuation. Then, we have

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - D_k(s,w) \right| \le C k^{3/2}\left( \frac{w}{2} \right) ^{-k}, \end{aligned}$$

where $C = C(s,w) > 0$ is some constant independent of k. Moreover, for $\varepsilon > 0$ sufficiently small $D_k(s,w)$ has precisely one zero $s_k(w)$ in the half-plane $\mathrm {Re}(s) \ge \frac{1}{2}+\varepsilon$ for all k sufficiently large and we have

$$\begin{aligned} \lim _{k\rightarrow \infty } s_k(w) = \delta (w). \end{aligned}$$

Remark 1.8

Jenkinson–Pollicott [22] proposed an algorithm to numerically compute the Hausdorff dimension for limit sets of certain Kleinian groups, using in a fundamental way transfer operators ${\mathscr {L}}_{s}$ associated to these sets. In the setting of [22], ${\mathscr {L}}_{s}$ is always given by a finite sum of composition operators. The transfer operator of Theorem 1.1 is an infinite sum of composition operators, making the analysis of Jenkinson–Pollicott more complicated for the task of estimating the Hausdorff dimension for Hecke triangle groups. Theorem 1.7 provides a different method to compute $\delta (w).$ For any given $w > 2$ and k sufficiently large, the numbers $s_k(w)$ can be calculated with arbitrary precision using a computer. Since we have made no attempt to precisely estimate the error $\vert \delta (w) - s_k(w)\vert$, the values of $s_k(w)$ yield only empirical estimates for $\delta (w)$. Nevertheless, these values are in perfect agreement with the approximations given by Phillips and Sarnak in [35]:

	Approximations for $\delta (w)$	Approximations for $s_{15}(w)$
w	of Phillips and Sarnak	from Theorem 1.7
3	$0.753 \pm 0.003$	0.752
4	$0.683 \pm 0.005$	0.684
6	$0.621 \pm 0.001$	0.623
8	$0.595 \pm 0.004$	0.594
10	$0.575 \pm 0.007$	0.577
16	$0.550 \pm 0.005$	0.550
40	$0.520 \pm 0.007$	0.522
100	$0.509 \pm 0.002$	0.509

Our next result is the following

Theorem 1.9

As $w\rightarrow \infty$ we have the asymptotic expansion

$$\begin{aligned} \delta (w) = \frac{1}{2} + \frac{1}{w} - \frac{2\log w}{w^2} + \frac{2\gamma _0}{w^2} + \sum _{j=2}^{4} \frac{P_j(\log w)}{w^{j+1}} + O\left( \frac{(\log w)^5}{w^{6}} \right) . \end{aligned}$$

(9)

Here, the implied constant in the error term does not depend on w, $\gamma _0 \approx 0.5772156649$ is the Euler–Mascheroni constant, and each $P_j$ ($j=2,3,4$) is a polynomial of degree j whose coefficients can be computed explicitly in terms of the Stieltjes constants.

Remark 1.10

The Stieltjes constants $\gamma _k$ appear in the Laurent expansion of the Riemann zeta function at $s=1,$

$$\begin{aligned} \zeta (s)=\frac{1}{s} + \sum _{k=0}^\infty \frac{(-1)^k \gamma _k}{k!} (s-1)^{k}, \end{aligned}$$

(10)

see for instance [15]. The 0-th Stieltjes constant $\gamma _0 \approx 0,577216$ is better known as the ‘Euler–Mascheroni constant.’ Numerical approximations for the Stieltjes constants can be found in OEIS [1].

Remark 1.11

It is likely that our proof method can be extended to give an asymptotic expansion with more terms on the right hand side of (9). We expect a full asymptotic expansion in powers of 1/w and $\log (w).$

Remark 1.12

Although Theorem 1.9 is concerned with the asymptotic behavior of $\delta (w)$ as $w\rightarrow \infty$, the methods developed to prove it may also be used to give numerical estimates for small w. As a concrete example, we prove the value $\delta (3)$ to be in the range

$$\begin{aligned} 0.75065< \delta (3) < 0.75322, \end{aligned}$$

see Sect. 4.4. This sharpens the estimate of Phillips–Sarnak in [35] and it answers in the affirmative a question posed by Jakobson–Naud [19] whether the quantity $\delta (3)$ is strictly larger than $\frac{3}{4}.$

Remark 1.13

An asymptotic formula similar to the one in Theorem 1.9 was proved by Hensley [17] for the Hausdorff dimension of the set $E_n$ as $n\rightarrow \infty$, where $E_n$ consists of all reals $x\in (0,1)$ for which the infinite continued fraction

$$\begin{aligned} x = \frac{1}{a_1 + \frac{1}{a_2 + \frac{1}{\cdots }}} \end{aligned}$$

has all its partial quotients $a_j$ in $\{ 1,\dots , n\}$. The set $\Lambda (\Gamma _w) \cap (-1,1)$ has a similar interpretation in terms of continued fractions: it consists of infinite continued fractions of the form

$$\begin{aligned} x = -\frac{1}{a_1 w - \frac{1}{a_2 w - \frac{1}{\cdots }}} \end{aligned}$$

with nonzero integers $a_1, a_2, \dots$. Similar asymptotic formulas for the Julia set related to the quadratic map $f_c(x) = x^2+c$ as $c\rightarrow 0$ and as $c\rightarrow -\infty$ appear in [5, 42].

Notation We write $f(x) = O(g(x))$ as $x\rightarrow a$ to mean $\limsup _{x\rightarrow a} \vert f(x)/g(x)\vert < \infty$. We use the symbol $f(x) \ll g(x)$ to mean $f(x) \le C g(x)$ for some implied constant $C > 0$ not depending on x.

Organization In Sect. 2, we begin by briefly recalling a few facts on hyperbolic geometry and singular values needed in this paper. After having precisely defined the transfer operator in Sect. 2.3 and the function space on which it acts, we prove Theorem 1.1 and Corollary 1.3. In Sect. 3 we prove Theorem 1.4, and in Sect. 4 we prove Theorem 1.7 and Theorem 1.9.

2 Twisted Selberg Zeta Function and Transfer Operators

2.1 Hyperbolic Geometry

For a thorough discussion on hyperbolic surfaces, Fuchsian groups (of finite and infinite covolume) and their spectral theory, we refer to Borthwick’s book [6]. One of the standard models for the hyperbolic plane is the Poincaré half-plane

$$\begin{aligned} {\mathbb {H}}^2 = \{ z = x+iy : x\in {\mathbb {R}},\, y\in {\mathbb {R}}_{>0}\}, \quad ds^{2} = \frac{dx^{2}+dy^{2}}{y^{2}}. \end{aligned}$$

The group of orientation-preserving isometries of $({\mathbb {H}}^2, ds)$ is isomorphic to

$$\begin{aligned} \mathrm {PSL}_{2}({\mathbb {R}}) = \left\{ \begin{bmatrix} a&{}\quad b\\ c&{}\quad d\end{bmatrix}\in {\mathbb {R}}^{2 \times 2} : ad-bc=1 \right\} . \end{aligned}$$

The elements of this group act on ${\mathbb {H}}^2$ by Möbius transformations:

$$\begin{aligned} \gamma = \begin{bmatrix} a&{}\quad b\\ c&{}\quad d\end{bmatrix} \in \mathrm {PSL}_2({\mathbb {R}}),\, z\in {\mathbb {H}}^2 \quad \Longrightarrow \quad \gamma (z) \mathrel {\mathop :}=\frac{az+b}{cz+d}. \end{aligned}$$

This action extends continuously to the boundary $\partial {\mathbb {H}}^2 = {\mathbb {R}}\cup \{ \infty \}$ and to the whole Riemann sphere ${\overline{{\mathbb {C}}}}$. Now let $\Gamma < {{\,\mathrm{PSL}\,}}_2({\mathbb {R}})$ be a discrete⁴ and finitely generated group. The limit set $\Lambda (\Gamma )$ of $\Gamma$ is defined as the set of accumulation points (in the Riemann sphere topology) of all orbits $\Gamma .z = \{ \gamma (z) : \gamma \in \Gamma \}$ where $z\in {\mathbb {H}}^2$ is any point which is not a fixed point of an elliptic element in $\Gamma .$ It turns out that the quotient $\Gamma \backslash {\mathbb {H}}^2$ has infinite hyperbolic volume if and only if $\Lambda (\Gamma )$ is a perfect, nowhere dense subset of $\partial {\mathbb {H}}^2.$

An element $\gamma \in \Gamma$ is said to be primitive if is not a proper power ${\widehat{\gamma }}^k$ of some element ${\widehat{\gamma }} \ne \gamma .$ An element $\gamma \in \Gamma$ is said to be hyperbolic if its action on ${\mathbb {H}}^2$ has two distinct fixed points on $\partial {\mathbb {H}}^2$, or equivalently, if $\vert {{\,\mathrm{tr}\,}}\gamma \vert > 2.$ Every hyperbolic transformation $\gamma$ is conjugate to the map $z\mapsto e^{\ell } z$ where $\ell = \ell (\gamma ) \in {\mathbb {R}}$, called the displacement length, is given by the formula

$$\begin{aligned} 2 \cosh \left( \frac{\ell (\gamma )}{2} \right) = \vert {{\,\mathrm{tr}\,}}\gamma \vert . \end{aligned}$$

(11)

Notice that $\vert {{\,\mathrm{tr}\,}}\gamma \vert$ is well-defined in $\mathrm {PSL}_2({\mathbb {R}}).$ (11) reveals that $\ell (\gamma )$ is invariant under conjugations, since the trace is. In particular the displacement length is constant on each $\Gamma$-conjugacy class

$$\begin{aligned} {[}\gamma ] \mathrel {\mathop :}=\{ g\gamma g^{-1} : g\in \Gamma \}. \end{aligned}$$

We denote by $[\Gamma ]_{h}$ the set of conjugacy classes of hyperbolic elements of $\Gamma$ and we denote by $[\Gamma ]_{p}$ the set of conjugacy classes of primitive hyperbolic elements of $\Gamma$. It is well know that the set of closed primitive geodesics on $\Gamma \backslash {\mathbb {H}}^2$ is bijective to the set $[\Gamma ]_p$. Moreover, given a conjugacy class $[\gamma ]\in [\Gamma ]_p$, the length of the corresponding geodesic is equal to the displacement length $\ell (\gamma ).$

Since $\ell (\gamma )$ is constant on each conjugacy class $[\gamma ]\in [\Gamma ]_h$, the Euler product definition of the twisted Selberg zeta function

$$\begin{aligned} Z_{\Gamma }(s, \rho ) = \prod _{[\gamma ]\in [\Gamma ]_{p}} \prod _{k=0}^{\infty } \mathrm {det}_{V}\left( 1_V- \rho (\gamma )e^{-(s+k)\ell (\gamma )} \right) \end{aligned}$$

(12)

is independent of the choice of the representative of each conjugacy class $[\gamma ].$ Here, $\rho :\Gamma _w \rightarrow \mathrm {U}(V)$ is assumed to be a unitary representation of the group $\Gamma _w$ with finite-dimensional representation space V.

Let us explain why the right hand side of (12) converges in the half-plane $\mathrm {Re}(s) > \delta$, where $\delta$ denotes the Hausdorff dimension of the limit set $\Lambda (\Gamma ).$ In view of the prime geodesic theorem (see [6, Chapter 14] and references therein), we may redefine the quantity $\delta$ as the abscissa of convergence of the series

$$\begin{aligned} \sum _{[\gamma ] \in [\Gamma ]_{p}} e^{ - s \ell (\gamma )}, \end{aligned}$$

that is,

$$\begin{aligned} \sum _{[\gamma ] \in [\Gamma ]_{p}} e^{ -s\ell (\gamma )}\text { is absolutely convergent} \Longleftrightarrow \mathrm {Re}(s) > \delta . \end{aligned}$$

(13)

Since $\rho$ is assumed to be a unitary representation, the eigenvalues of $\rho (\gamma )$ lie on the unit circle for every $\gamma$, showing that

$$\begin{aligned} \mathrm {det}_{V}\left( 1_V- \rho (\gamma )e^{-(s+k)\ell (\gamma )} \right)\le & {} \left( 1+e^{-(s+k)\ell (\gamma )} \right) ^{\dim (\rho )}\nonumber \\\le & {} \exp \left( \dim (\rho ) e^{-(s+k)\ell (\gamma )} \right) \end{aligned}$$

(14)

where $\dim (\rho ) \mathrel {\mathop :}=\dim (V)$ denotes the dimension of $\rho$. Combining (13) and (14) shows that the product on the left-hand side of (12) converges in the half-plane $\mathrm {Re}(s) > \delta$.

2.2 Singular Values and Fredholm Determinants

In this subsection, we collect some preliminaries about singular values which will be used repeatedly in this paper. Good references for the general theory of singular values and Fredholm determinants include [12, 13, 45].

Given two separable Hilbert spaces ${\mathscr {H}}_1$ and ${\mathscr {H}}_2$ and a compact operator ${\mathscr {A}}:{\mathscr {H}}_1\rightarrow {\mathscr {H}}_2$ we let ${\mathscr {A}}^*:{\mathscr {H}}_2\rightarrow {\mathscr {H}}_1$ denote its adjoint operator. Note that ${\mathscr {A}}^*{\mathscr {A}} :{\mathscr {H}}_1\rightarrow {\mathscr {H}}_1$ is a positive and symmetric operator. The absolute value of ${\mathscr {A}}$, denoted by $\vert {\mathscr {A}}\vert$, is the unique positive and symmetric operator ${\mathscr {H}}_1\rightarrow {\mathscr {H}}_1$ satisfying $\vert {\mathscr {A}}\vert ^2 = {\mathscr {A}}^*{\mathscr {A}}$. The singular values of ${\mathscr {A}}$ are the nonzero eigenvalues of $\vert {\mathscr {A}}\vert$, arranged in decreasing order,

$$\begin{aligned} \mu _1(A) \ge \mu _2(A) \ge \cdots . \end{aligned}$$

If necessary, we turn this sequence into an infinite one by filling it up with zeros at the end. We say that ${\mathscr {A}}$ is a trace-class operator if

$$\begin{aligned} \Vert {\mathscr {A}}\Vert _1 \mathrel {\mathop :}=\sum _{k=1}^\infty \mu _k({\mathscr {A}}) < \infty . \end{aligned}$$

It is well known that $\Vert \cdot \Vert _1$ is a norm, called the trace norm. The min-max characterization of singular values says that

$$\begin{aligned} \mu _m(A) = \min _{\begin{array}{c} V \subset {\mathscr {H}} \\ \dim (V) = m-1 \end{array}} \max _{ \psi \in V^{\perp } } \frac{\Vert A \psi \Vert }{\Vert \psi \Vert }, \end{aligned}$$

(15)

where the minimum is taken over all $m-1$-dimensional subspaces of ${\mathscr {H}}$. It follows immediately that the largest singular value is equal to the operator norm:

$$\begin{aligned} \mu _1(A) = \Vert A\Vert . \end{aligned}$$

We can also use the min-max to derive the following estimate: for any given orthonormal basis $\{ \psi _m \}_{m\in {\mathbb {N}}_0}$ of ${\mathscr {H}}$ we have

$$\begin{aligned} \mu _n({\mathscr {A}}) \le \sum _{k\ge n} \Vert {\mathscr {A}}\psi _k\Vert . \end{aligned}$$

(16)

Now, for every trace-class operator ${\mathscr {A}}:{\mathscr {H}} \rightarrow {\mathscr {H}}$ and for every $u\in {\mathbb {C}}$ sufficiently small we have the absolutely convergent expansion for the Fredholm determinant

$$\begin{aligned} \det \left( 1- u {\mathscr {A}} \right) = \exp \left( {{\,\mathrm{tr}\,}}\log \left( 1-u {\mathscr {A}} \right) \right) = \exp \left( - \sum _{N=1}^{\infty } \frac{u^{N}}{N} \mathrm {tr}\left( {\mathscr {A}}^{N} \right) \right) . \end{aligned}$$

(17)

This is a direct consequence of Lidskii’s theorem, see [45, Chapter 3].

Let us conclude this subsection with an estimate for Fredholm determinants which proves extremely useful in this paper: if both ${\mathscr {A}}$ and ${\mathscr {B}}$ are trace-class operators, then

$$\begin{aligned} \left| \det \left( 1-{\mathscr {A}} \right) - \det \left( 1-{\mathscr {B}} \right) \right| \le \Vert {\mathscr {A}} - {\mathscr {B}} \Vert _1 \exp \left( \Vert {\mathscr {A}} \Vert _1 + \Vert {\mathscr {B}} \Vert _1 + 1 \right) , \end{aligned}$$

(18)

see for instance [12, Corollary 4.2].

2.3 Transfer Operator and Function Space

Recall that the Hecke triangle group $\Gamma _{w}$ is defined to be the subgroup of $\mathrm {PSL}_2({\mathbb {R}})$ generated by the two elements

$$\begin{aligned} T_w \mathrel {\mathop :}=\begin{bmatrix} 1&{}\quad w\\ 0&{}\quad 1\end{bmatrix}\quad \text {and}\quad S\mathrel {\mathop :}=\begin{bmatrix} 0&{}\quad 1\\ -1&{}\quad 0\end{bmatrix}. \end{aligned}$$

We will henceforth assume that $w > 2$ in which case $\Gamma _{w}$ is a Fuchsian group with infinite co-volume, i.e., the hyperbolic quotient $\Gamma _w\backslash {\mathbb {H}}^2$ has infinite area.

From now on, V is a finite-dimensional complex vector space endowed with the Hermitian inner product $\langle \cdot , \cdot \rangle _V$ and $\rho :\Gamma _{w} \rightarrow \mathrm {U}(V)$ is a unitary representation of $\Gamma _{w}$.

Let ${\mathbb {D}} = \{ \vert z\vert < 1 \}$ be the open unit disk in the complex plane. The function space of interest is the vector-valued Bergman space

$$\begin{aligned} H^2({\mathbb {D}} ; V) \mathrel {\mathop :}=\left\{ f:{\mathbb {D}} \rightarrow V\text { holomorphic} \ \left| \ \Vert f\Vert < \infty \right. \right\} , \end{aligned}$$

(19)

with $L^{2}$-norm given by

$$\begin{aligned} \Vert f\Vert ^2 \mathrel {\mathop :}=\int _{{\mathbb {D}}} \Vert f(z)\Vert _{V}^{2}{{\,\mathrm{dvol}\,}}(z). \end{aligned}$$

Here ${{\,\mathrm{vol}\,}}$ denotes the Lebesgue measure and $\Vert \cdot \Vert _V$ is the norm on V induced by $\langle \cdot , \cdot \rangle _V$. Endowed with the inner product

$$\begin{aligned} \langle f, g\rangle \mathrel {\mathop :}=\int _{{\mathbb {D}}} \langle f(z), g(z)\rangle _V{{\,\mathrm{dvol}\,}}(z), \end{aligned}$$

the space $H^{2}({\mathbb {D}} ; V)$ is a Hilbert space. Notice that $H^{2}( {\mathbb {D}} ; {\mathbb {C}}) = H^{2}({\mathbb {D}} )$ is the classical Bergman space over ${\mathbb {D}}$.

Now for every $n\in {\mathbb {Z}}$, we define the element

$$\begin{aligned} \gamma _{n} \mathrel {\mathop :}=S T_{w}^{n} = \begin{bmatrix} 0&{}\quad -1\\ 1&{}\quad nw\end{bmatrix}. \end{aligned}$$

Note that $\gamma _n$ is hyperbolic for all $n\in {\mathbb {Z}}\smallsetminus \{ 0\}$ since $w>2$. Finally, we define the (initially only formal) transfer operator

$$\begin{aligned} {\mathscr {L}}_{s, w, \rho } f(z) = \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1} f\left( \gamma _n(z) \right) , \quad z\in {\mathbb {D}}, \end{aligned}$$

(20)

acting on functions $f\in H^2({\mathbb {D}} ; V).$ Note that the Möbius transformation $\gamma _n$ and its derivative are given by

$$\begin{aligned} \gamma _n(z) = -\frac{1}{z+nw} \quad \text {and}\quad \gamma _n'(z) = \frac{1}{(z+nw)^2}. \end{aligned}$$

It is not difficult to verify that for every nonzero integer n,  $\gamma _n(z)$ belongs to the open unit disk ${\mathbb {D}}$ whenever z does. Indeed, a straightforward calculation shows that $\gamma _n$ ($n\ne 0$) maps ${\mathbb {D}}$ to

$$\begin{aligned} \gamma _n({\mathbb {D}}) = \left\{ z\in {\mathbb {C}} : \left| z - \frac{1}{nw} \right|< \frac{1}{\vert n\vert w} \right\} \subset \left\{ z\in {\mathbb {C}} : \vert z\vert < r \right\} , \end{aligned}$$

(21)

where $r=\frac{2}{w}.$ Since $w>2,$ this implies

$$\begin{aligned} \overline{\gamma _n({\mathbb {D}})} \subset {\mathbb {D}}. \end{aligned}$$

Now let us show that the complex powers $\gamma _n'(z)^s$ appearing in (20) make sense. Note that for all $n\ne 0$ the derivative $\gamma _n'$ is positive on the interval $[-1, 1]$ and nonzero in the disk ${\mathbb {D}}$. For $z\in {\mathbb {D}}$, we can write

$$\begin{aligned} \gamma _n'(z)^{s} = \left( \vert n\vert w \right) ^{-2s} e^{-2s \log \left( 1+\frac{z}{nw} \right) } \end{aligned}$$

(22)

Here, the logarithm is given by the usual Taylor expansion

$$\begin{aligned} \log (1+u) = u - \frac{u^2}{2} + \frac{u^3}{3}\pm \cdots , \end{aligned}$$

which is valid for all $\vert u\vert < 1$. This shows that (22) is well defined whenever $z\in {\mathbb {D}}$ and $n\in {\mathbb {Z}}\smallsetminus \{0\}.$

For our subsequent analysis, it is useful to rewrite the operator ${\mathscr {L}}_{s,w,\rho }$ as an infinite sum

$$\begin{aligned} {\mathscr {L}}_{s, w,\rho } = \sum _{n\in {\mathbb {Z}}\smallsetminus \{0\}} \nu _{s,\rho }(\gamma _n^{-1}) = \sum _{n\in {\mathbb {Z}}\smallsetminus \{0\}} \nu _{s,\rho }(T_w^{-n}S) = \sum _{n\in {\mathbb {Z}}\smallsetminus \{0\}} \nu _{s,\rho }(T_{w}^{n}S),\nonumber \\ \end{aligned}$$

(23)

where for every element $\gamma$, the $\nu _{s,\rho }(\gamma )$’s are composition operators of the form

$$\begin{aligned} \nu _{s,\rho }(\gamma ):H^2( {\mathbb {D}}; V )\rightarrow H^2({\mathbb {D}}; V), \quad \nu _{s,\rho }(\gamma )f(z) := \left[ (\gamma ^{-1})'(z)\right] ^s \rho (\gamma ) f(\gamma ^{-1}(z)).\nonumber \\ \end{aligned}$$

(24)

These operators are well defined provided $\gamma ^{-1}( {\mathbb {D}} ) \subset {\mathbb {D}}.$ The following result shows that ${\mathscr {L}}_{s,w,\rho }$ is a trace-class operator.

Proposition 2.1

Let assumptions and notations be as above. Then for every $s\in {\mathbb {C}}$, there exists a constant $C = C(s,w) > 0$ such that for every integer $n\ne 0$

$$\begin{aligned} \Vert \nu _{s,\rho }(\gamma _n^{-1}) \Vert _1 \le C \frac{\dim ( \rho ) }{\vert n\vert ^{2\sigma }}, \end{aligned}$$

where $\sigma \mathrel {\mathop :}=\mathrm {Re}(s)$. In particular, (20) defines a bounded trace-class operator

$$\begin{aligned} {\mathscr {L}}_{s, w, \rho }:H^2({\mathbb {D}} ; V) \rightarrow H^2( {\mathbb {D}} ; V), \end{aligned}$$

(25)

provided $\sigma > \frac{1}{2}.$

Proof

The family of functions $\{ \psi _j\}_{j\in {\mathbb {N}}_0}$ given by

$$\begin{aligned} \psi _j(z) = \sqrt{\frac{j+1}{\pi }} z^j \end{aligned}$$

(26)

provides an orthonormal basis for the (classical) Bergman space $H^2({\mathbb {D}})$. Let $\mathbf{e }_1, \dots , \mathbf{e }_d$ be a orthonormal basis for the representation space V, where $d = \dim (V).$ Then, the family of functions

$$\begin{aligned} \psi _j\cdot \mathbf{e }_k \end{aligned}$$

(27)

with $j \in {\mathbb {N}}_0$ and $1\le k \le d$ forms a basis for $H^2({\mathbb {D}}; V)$. Using the singular value estimate in (16), we can estimate the singular values of $\nu _{s,\rho }(\gamma _n^{-1})$ as

$$\begin{aligned} \mu _m( \nu _{s,\rho }(\gamma _n^{-1}) ) \le \sum _{k=1}^d \sum _{j \ge m} \Vert \nu _{s,\rho }(\gamma _n^{-1}) \psi _j \mathbf{e }_k \Vert . \end{aligned}$$

(28)

Notice that since $\rho$ is a unitary representation, the operator norm of the endomorphism $\rho (\gamma _n)$ satisfies

$$\begin{aligned} \Vert \rho (\gamma _n) \Vert _{\mathrm {End}(V)} = 1. \end{aligned}$$

(29)

Hence, for $j \in {\mathbb {N}}_0$ and $1\le k \le d$ we have

$$\begin{aligned} \Vert \nu _{s,\rho }(\gamma _n^{-1}) \psi _j \mathbf{e }_k \Vert ^2&= \int _{{\mathbb {D}}} \left| \gamma _n'(z)^s \psi _j(\gamma _n(z)) \right| ^2 {{\,\mathrm{dvol}\,}}(z) \nonumber \\&= \frac{j+1}{\pi } \int _{{\mathbb {D}}} \left| \gamma _n'(z)^s \gamma _n(z)^j\right| ^2 {{\,\mathrm{dvol}\,}}(z). \end{aligned}$$

(30)

The goal is to estimate the integral on the right hand side of (30). For the rest of this proof, we allow the implied constants to depend only on s and w. From (21), it follows directly that for all $n\ne 0$ and all $z\in {\mathbb {D}},$ we have

$$\begin{aligned} \left| \gamma _n(z)\right| \le r, \end{aligned}$$

(31)

where $r=\frac{2}{w}<1.$ In light of (22), the complex derivatives can be estimated as

$$\begin{aligned} \left| \gamma _n'(z)^s \right| \ll \frac{1}{\vert n\vert ^{2\sigma }}. \end{aligned}$$

(32)

Using (31) and (32) in (30) gives

$$\begin{aligned} \Vert \nu _{s,\rho }(\gamma _n^{-1}) \psi _j \mathbf{e }_k \Vert ^2 \ll \frac{(j+1)}{\vert n\vert ^{4\sigma }} r^{2j}. \end{aligned}$$

Going back to (28) and recalling that $0< r < 1$, we estimate

$$\begin{aligned} \mu _m( \nu _{s,\rho }(\gamma _n^{-1}) ) \ll \frac{\dim ( \rho ) }{\vert n\vert ^{2\sigma }} \sum _{j=m}^\infty \sqrt{j+1} \, r^{j} \ll \frac{\dim ( \rho ) }{\vert n\vert ^{2\sigma }} \sqrt{m+1} \, r^m. \end{aligned}$$

We deduce that

$$\begin{aligned} \Vert \nu _{s,\rho }(\gamma _n^{-1})\Vert _1 = \sum _{m=1}^\infty \mu _m( \nu _{s,\rho }(\gamma _n^{-1}) ) \ll \frac{\dim ( \rho ) }{\vert n\vert ^{2\sigma }} \sum _{m=1}^\infty \sqrt{m+1} \, r^m \ll \frac{\dim ( \rho ) }{\vert n\vert ^{2\sigma }}, \end{aligned}$$

which is what we claimed. In particular, this bound shows the trace norms $\Vert \nu _{s,\rho }(\gamma _n^{-1})\Vert _1$ are summable in n provided $\sigma > \frac{1}{2}$. It follows that the operator ${\mathscr {L}}_{s,w,\rho }$ is trace-class for $\sigma > \frac{1}{2}$. This completes the proof. $\square$

2.4 Proof of Theorem 1.1

Combining Proposition 2.1 with the fact that ${\mathscr {L}}_{s, w,\rho }$ depends holomorphically on s, we deduce that the Fredholm determinant

$$\begin{aligned} \det \left( 1- {\mathscr {L}}_{s, w,\rho } \right) \end{aligned}$$

is a holomorphic function in the half-plane $\mathrm {Re}(s) > \frac{1}{2}.$ Our goal is to show that it coincides with the function $Z_{\Gamma _{ w}}(s,\rho )$ in this half-plane. To that effect, fix $s\in {\mathbb {C}}$ with $\mathrm {Re}(s) > \frac{1}{2}$ and consider the entire function

$$\begin{aligned} u\mapsto \det \left( 1-u {\mathscr {L}}_{s, w, \rho } \right) . \end{aligned}$$

Recall from (17) that we have the absolutely convergent expansion

$$\begin{aligned} \det \left( 1- u {\mathscr {L}}_{s,w, \rho } \right) = \exp \left( - \sum _{N=1}^{\infty } \frac{u^{N}}{N} \mathrm {tr}\left( {\mathscr {L}}_{s, w,\rho }^{N} \right) \right) , \end{aligned}$$

(33)

whenever $\vert u\vert$ is small enough. In view of (33), Theorem 1.1 amounts to finding a suitable expression for the traces of the iterates ${\mathscr {L}}_{s, w,\rho }^N$. To do so, notice that the operators in (24) satisfy the composition rule

$$\begin{aligned} \nu _{s,\rho }(g_1)\nu _{s,\rho }(g_2) = \nu _{s,\rho }(g_1 g_2), \end{aligned}$$

which in turn implies that

$$\begin{aligned} {\mathscr {L}}_{s, w,\rho }^{N} = \left( \sum _{n\in {\mathbb {Z}}\smallsetminus \{0\}} \nu _{s,\rho }(T_{w}^{n} S) \right) ^N = \sum _{n_1,\ldots , n_N\in {\mathbb {Z}}\smallsetminus \{0\}} \nu _{s,\rho }(T_{w}^{n_{1}}S T_{w}^{n_{2}}S \cdots T_{w}^{n_{N}}S) \end{aligned}$$

for every positive integer N. We can rewrite this more conveniently as

$$\begin{aligned} {\mathscr {L}}_{s, w,\rho }^{N} = \sum _{\gamma \in P_{N}} \nu _{s,\rho }(\gamma ), \end{aligned}$$

(34)

where $P_{N}\subset \Gamma _w$ is the set

$$\begin{aligned} P_N \mathrel {\mathop :}=\left\{ T_{w}^{n_{1}}S T_{w}^{n_{2}}S \cdots T_{w}^{n_{N}}S : n_1,\ldots , n_N\in {\mathbb {Z}}\smallsetminus \{0\}\right\} . \end{aligned}$$

Now let $P \subset \Gamma _w$ be the union of all the $P_N$’s,

$$\begin{aligned} P \mathrel {\mathop :}=\bigcup _{N\in {\mathbb {N}}} P_{N}. \end{aligned}$$

Recall that $[\Gamma _{w}]_{h}$ and $[\Gamma _{w}]_p$ denote the set of conjugacy classes of hyperbolic elements in $\Gamma _w$ and the set of conjugacy classes of primitive hyperbolic elements in $\Gamma _w$ respectively. Given a conjugacy class $[\gamma ]$ represented by a hyperbolic element $\gamma \in \Gamma _{w}$, we denote by $m(\gamma )$ the unique positive integer m satisfying $\gamma = {\widehat{\gamma }}^{m}$ with ${\widehat{\gamma }}\in \Gamma$ primitive (i.e., $[{\widehat{\gamma }}]\in [\Gamma _{w}]_{p}$).

The following properties can be checked easily:

• (1) Every element in P is hyperbolic. This is true because the only non-hyperbolic elements are those which are conjugated to powers of either S or $T_w$ (since $w > 2$), none of which appear in the set P.

• (2) Every hyperbolic conjugacy class $[\gamma ]\in [\Gamma _{w}]_{h}$ has a representative in P, say in $P_{N}$, and $N = N(\gamma )$ is unique with this property.

• (3) Every hyperbolic conjugacy class $[\gamma ]\in [\Gamma _{ w}]_{h}$ has precisely $N(\gamma )/m(\gamma )$ distinct representatives in $P_{N(\gamma )}$. Indeed, after conjugation we can represent $\gamma$ by a word of the form $\gamma _{n_1} \cdots \gamma _{n_{N(\gamma )}}$ where $\gamma _{n_i} = S T_{w}^{n_{i}}$. This word has precisely $N(\gamma )/m(\gamma )$ distinct cyclic permutations.

The following lemma is crucial to make the connection between Selberg zeta functions and transfer operators.

Lemma 2.2

For every hyperbolic Möbius transformation $\gamma \in \Gamma _{w}$ with $\overline{\gamma ^{-1}({\mathbb {D}})} \subset {\mathbb {D}}$, we have

$$\begin{aligned} \mathrm {tr}(\nu _{s,\rho }(\gamma )) = \chi (\gamma ) \frac{e^{-s \ell (\gamma )}}{1-e^{-\ell (\gamma )}}, \end{aligned}$$

(35)

where $\chi = {{\,\mathrm{tr}\,}}_V \rho$ is the character associated to the representation $\rho$ and $\ell (\gamma )$ is the displacement length of $\gamma$ given by (11).

Results similar to Lemma 2.2 are widely known in the literature, at least for the trivial representation $\rho = \mathbf{1 }$, in which case it can be seen as a special case of the holomorphic Lefschetz fixed point formula, see for instance Lemma 15.9 in [6] and the references given therein. We will give a proof of Lemma 2.2 at the end of this section for the sake of keeping the proof of Theorem 1.1 self-contained.

Taking traces on both sides of (34), using Lemma 2.2 and a geometric series expansion, we obtain

$$\begin{aligned} \mathrm {tr}\left( {\mathscr {L}}_{s, w,\rho }^{N}\right) = \sum _{\gamma \in P_{N}} \chi (\gamma ) \frac{e^{-s \ell (\gamma )}}{1-e^{-\ell (\gamma )}} = \sum _{k=0}^{\infty } \sum _{\gamma \in P_{N}} \chi (\gamma ) e^{-(s+k)\ell (\gamma )}. \end{aligned}$$

Using the properties (1), (2) and (3) above, we can rewrite the inner sum on the right as a sum over primitive hyperbolic conjugacy classes:

$$\begin{aligned} \sum _{\gamma \in P_{N}} e^{-(s+k)\ell (\gamma )} \chi (\gamma )&= \sum _{m=1}^{\infty }\sum _{\begin{array}{c} \gamma \in P_{N} \\ \gamma = {\widehat{\gamma }}^{m}, \; [{\widehat{\gamma }}]\in [\Gamma _{ w}]_{p} \end{array}} \chi ({\widehat{\gamma }}^{m}) e^{-m(s+k)\ell ({\widehat{\gamma }})}\\&= \sum _{m=1}^{\infty } \sum _{\begin{array}{c} [{\widehat{\gamma }}]\in [\Gamma _{ w}]_{p}\\ N({\widehat{\gamma }})\cdot m = N \end{array}} \frac{N}{m} \chi ({\widehat{\gamma }}^{m}) e^{-m(s+k)\ell ({\widehat{\gamma }})}. \end{aligned}$$

Hence, going back to (33), we obtain

$$\begin{aligned} \log \det \left( 1- u {\mathscr {L}}_{s, w,\rho } \right)&= - \sum _{N=1}^{\infty } \frac{u^{N}}{N}\mathrm {tr}\left( {\mathscr {L}}_{s, w,\rho }^{N}\right) \\&= - \sum _{N=1}^{\infty } \frac{u^{N}}{N} \sum _{k=0}^{\infty }\sum _{m=1}^{\infty } \sum _{\begin{array}{c} [{\widehat{\gamma }}]\in [\Gamma _{ w}]_{p}\\ N({\widehat{\gamma }})\cdot m = N \end{array}} \frac{N}{m} \chi ({\widehat{\gamma }}^{m}) e^{-(s+k)\ell (\gamma )}. \end{aligned}$$

Rearranging the order of summation (which is justified for $\mathrm {Re}(s)$ large enough by absolute convergence) leads to

$$\begin{aligned} \log \det \left( 1- u {\mathscr {L}}_{s, w,\rho } \right)&= - \sum _{k=0}^{\infty } \sum _{[{\widehat{\gamma }}]\in [\Gamma _{ w}]_{p}} \sum _{m=1}^{\infty } \frac{u^{N({\widehat{\gamma }})\cdot m}}{m} \chi ({\widehat{\gamma }}^{m}) e^{-m(s+k)\ell ({\widehat{\gamma }})}\\&= \sum _{k=0}^{\infty } \sum _{[{\widehat{\gamma }}]\in [\Gamma _{ w}]_{p}} \log \det \left( 1- u^{N({\widehat{\gamma }})} \rho ({\widehat{\gamma }}) e^{-(s+k)\ell ({\widehat{\gamma }})}\right) \\&= \log \prod _{k=0}^{\infty } \prod _{[{\widehat{\gamma }}]\in [\Gamma _{w}]_{p}} \det \left( 1- u^{N({\widehat{\gamma }})} \rho ({\widehat{\gamma }}) e^{-(s+k)\ell ({\widehat{\gamma }})}\right) . \end{aligned}$$

Recall from the discussion at the end of Sect. 2.1 that the expression in the last line converges at $u=1$, provided $\mathrm {Re}(s)$ is large enough. Thus, we obtain the identity

$$\begin{aligned} Z_{\Gamma _{w}}(s,\rho ) = \det \left( 1- {\mathscr {L}}_{s, w, \rho } \right) , \end{aligned}$$

(36)

completing the proof of Theorem 1.1, provided $\mathrm {Re}(s)$ is large enough. Since both sides of (36) are holomorphic functions in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$, the validity of this identity extends to $\mathrm {Re}(s) > \frac{1}{2},$ by uniqueness of analytic continuation.

Proof of Lemma 2.2

Let $\{ \psi _j\}_{j\ge 0}$ be the orthonormal basis of the space $H^2({\mathbb {D}})$ given by (26). We can then explicitly compute the associated Bergman kernel

$$\begin{aligned} B_{{\mathbb {D}}}(z,z')=\sum _{j=0}^\infty \psi _j(z)\overline{\psi _j(z')} = \frac{1}{\pi (1-z \overline{z'})^2}. \end{aligned}$$

(37)

Recall from the proof of Proposition 2.1 that after having fixed a basis $\mathbf{e }_1, \dots , \mathbf{e }_d$ for the representation space V, the family

$$\begin{aligned} \{ \psi _j\cdot \mathbf{e }_k \}_{\begin{array}{c} j \ge 0\\ 1\le k\le d \end{array}} \end{aligned}$$

provides an orthonormal basis for $H^2({\mathbb {D}}, V)$. Using this basis, we compute the trace as

$$\begin{aligned} \mathrm {tr}(\nu _{s,\rho }(\gamma ))&= \sum _{j=0}^{\infty } \sum _{k=1}^{d} \left\langle \nu _{s,\rho }(\gamma )(\psi _j\cdot \mathbf{e }_k), \psi _j\cdot \mathbf{e }_k \right\rangle \\&= \sum _{m=0}^{\infty } \sum _{j=1}^{d} \int _{{\mathbb {D}}} \langle \rho (\gamma ) \mathbf{e }_k, \mathbf{e }_k \rangle _V \left[ (\gamma ^{-1})'(z)\right] ^s \psi _j(\gamma ^{-1}(z)) \overline{\psi _j(z)} {{\,\mathrm{dvol}\,}}(z)\\&= \left( \sum _{j=1}^{d} \langle \rho (\gamma ) \mathbf{e }_k, \mathbf{e }_k \rangle _V \right) \int _{{\mathbb {D}}} \left[ (\gamma ^{-1})'(z)\right] ^s B_{{\mathbb {D}}}(\gamma ^{-1}(z),z) {{\,\mathrm{dvol}\,}}(z). \end{aligned}$$

The parenthetical sum in the previous line is equal to ${{\,\mathrm{tr}\,}}_V(\rho (\gamma )) = \chi (\gamma )$, so it remains to calculate the integral. Using the explicit formula for the Bergman kernel in (37) we can write

$$\begin{aligned} \int _{{\mathbb {D}}} \left[ (\gamma ^{-1})'(z)\right] ^s B_{{\mathbb {D}}}(\gamma ^{-1}(z),z) {{\,\mathrm{dvol}\,}}(z) = \frac{1}{\pi }\int _{{\mathbb {D}}} \frac{\left[ (\gamma ^{-1})'(z)\right] ^s}{\left( 1 - {\overline{z}} \gamma ^{-1}(z) \right) ^2} {{\,\mathrm{dvol}\,}}(z). \end{aligned}$$

Now we apply the complex form of Stokes’ formula

$$\begin{aligned} \int _{{\mathbb {D}}} \frac{\partial F}{\partial {\overline{z}}} {{\,\mathrm{dvol}\,}}(z) = \frac{1}{2i}\int _{\vert z\vert = 1} F dz, \end{aligned}$$

valid for any $F\in C^1(\overline{{\mathbb {D}}})$, to the function

$$\begin{aligned} F(z,{\overline{z}}) = \frac{ {\overline{z}} \left[ (\gamma ^{-1})'(z)\right] ^s}{1- {\overline{z}} \gamma ^{-1}(z)}. \end{aligned}$$

This yields

$$\begin{aligned}&\int _{{\mathbb {D}}} \left[ (\gamma ^{-1})'(z)\right] ^s B_{{\mathbb {D}}}(\gamma ^{-1}(z),z) {{\,\mathrm{dvol}\,}}(z) \nonumber \\&\quad = \frac{1}{2 \pi i} \int _{\vert z\vert = 1} \frac{ {\overline{z}} \left[ (\gamma ^{-1})'(z)\right] ^s}{1- {\overline{z}} \gamma ^{-1}(z)} dz = \frac{1}{2 \pi i} \int _{\vert z\vert = 1} \frac{ \left[ (\gamma ^{-1})'(z)\right] ^s}{z - \gamma ^{-1}(z)} dz, \end{aligned}$$

(38)

where in the last equation we used the fact that the integration on the center is restricted to $\vert z\vert ^2 = z {\overline{z}} = 1$. Using the Cauchy integral formula, the integral on the right hand side of (38) can be evaluated to be equal to

$$\begin{aligned} \frac{ \left[ (\gamma ^{-1})'(z_0)\right] ^s}{1 -(\gamma ^{-1})'(z_0)}, \end{aligned}$$

where $z_0$ is the (unique) fixed point of the map $\gamma ^{-1}:{\mathbb {D}} \rightarrow {\mathbb {D}}.$ Finally, one can show by an elementary calculation that $(\gamma ^{-1})'(z_0) = e^{-\ell (\gamma )}$, completing the proof of Lemma 2.2. $\square$

2.5 Proof of Corollary 1.3

The goal of this subsection is to prove Corollary 1.3. In view of Theorem 1.1, it suffices to show that $s\mapsto {\mathscr {L}}_{s, w,\rho }$ (which is only defined for $\mathrm {Re}(s) > \frac{1}{2}$) admits a meromorphic continuation to $s\in {\mathbb {C}}$ with poles contained in $\frac{1}{2}(1-{\mathbb {N}}_0)$. We will use ideas of Mayer [24] and Pohl [38].

Every $f\in H^{2}({\mathbb {D}}; V)$, being holomorphic, can be Taylor expanded around $z=0$ as

$$\begin{aligned} f(z) = \sum _{i=0}^\infty c_i z^i \end{aligned}$$

for some suitable coefficients $c_i \in V$. Hence, we can write

$$\begin{aligned} f(z) = f(0) + z {\widetilde{f}}(z), \quad z\in {\mathbb {D}}, \end{aligned}$$

where ${\widetilde{f}} \in H^{2}({\mathbb {D}}; V)$ is given by

$$\begin{aligned} {\widetilde{f}}(z) = \sum _{i=0}^\infty c_{i+1} z^i. \end{aligned}$$

We can then write for all $z\in {\mathbb {D}}$

$$\begin{aligned} {\mathscr {L}}_{s, w, \rho } f(z)&= \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1} f\left( \gamma _n(z) \right) \\ {}&= \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1} f(0) + \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \gamma _{n}(z) \rho (\gamma _n)^{-1} {\widetilde{f}}\left( \gamma _{n}(z) \right) . \end{aligned}$$

Using the relation

$$\begin{aligned} \gamma _n'(z)^{s} \gamma _{n}(z) = - \gamma _n'(z)^{s+1/2} \end{aligned}$$

this gives

$$\begin{aligned} {\mathscr {L}}_{s, w, \rho } f(z)&= \left( \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1} \right) f\left( 0 \right) \nonumber \\&- \underbrace{\sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s+1/2} \rho (\gamma _n)^{-1} {\widetilde{f}} \left( \gamma _n(z) \right) }_{ = {\mathscr {L}}_{s+1/2, w, \rho } {\widetilde{f}}(z)}. \end{aligned}$$

(39)

Now let us introduce the (bounded) operator

$$\begin{aligned} \Psi :H^{2}({\mathbb {D}}; V) \rightarrow H^{2}({\mathbb {D}}; V), \quad f \mapsto -{\widetilde{f}} \end{aligned}$$

(40)

and the (finite-rank) operator

$$\begin{aligned} {\mathscr {F}}_{s,w,\rho ,1} :H^{2}({\mathbb {D}}; V) \rightarrow H^{2}({\mathbb {D}}; V), \quad f \mapsto \xi (s,w,\rho ; \cdot ) f\left( 0 \right) , \end{aligned}$$

(41)

where

$$\begin{aligned} \xi (s,w,\rho ; z) \mathrel {\mathop :}=\sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (\gamma _n)^{-1}. \end{aligned}$$

(42)

We can then rewrite (39) more conveniently as

$$\begin{aligned} {\mathscr {L}}_{s, w,\rho } = {\mathscr {F}}_{s,w,\rho ,1} + {\mathscr {L}}_{s+\frac{1}{2}, w,\rho } \Psi . \end{aligned}$$

(43)

The second term on the right hand side of (43) is obviously defined for all $\mathrm {Re}(s)>0$, while the first term is defined a priori only in the range $\mathrm {Re}(s) > \frac{1}{2}.$ To pass beyond $\mathrm {Re}(s) = \frac{1}{2}$ we have to study the operator in (41). Recalling that $\gamma _n = S T_w^n$, we can rewrite (42) as

$$\begin{aligned} \xi (s,w,\rho ; z) = \left( \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} \rho (T_w^{-1})^{n} \right) \rho (S). \end{aligned}$$

(44)

Furthermore, since $\rho$ is unitary, we can find real numbers $\mu _1, \dots , \mu _d \in [0, 1)$ and a basis $\mathbf{e }_1, \dots , \mathbf{e }_d$ of the representation space V, with respect to which $\rho (T_w^{-1})$ acts by the diagonal matrix

$$\begin{aligned} \rho (T_w^{-1}) = {{\,\mathrm{diag}\,}}\left( e^{2\pi i \mu _1}, \cdots , e^{2\pi i \mu _d} \right) . \end{aligned}$$

(45)

Inserting (45) into (44), we obtain the expression

$$\begin{aligned} \xi (s,w,\rho ; z) = {{\,\mathrm{diag}\,}}\left( \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} e^{2\pi i \mu _1 n}, \cdots , \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} e^{2\pi i \mu _d n} \right) \cdot \rho (S).\nonumber \\ \end{aligned}$$

(46)

Let us now inspect the diagonal entries on the right of (46) individually. Recalling the definition of the complex powers $\gamma _n'(z)^{s}$ given in (22), we can write for each $j\in \{ 1, \dots , d\}$:

$$\begin{aligned} \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} e^{2\pi i \mu _j n}&= \sum _{n=1}^\infty \frac{e^{2\pi i \mu _j n}}{(nw+z)^{2s}} + \sum _{n=1}^\infty \frac{ e^{-2\pi i \mu _j n}}{(nw-z)^{2s}}\\&= w^{-2s} H\left( \frac{z}{w}, 2s, \mu _j \right) + w^{-2s} H\left( -\frac{z}{w}, 2s, -\mu _j \right) , \end{aligned}$$

where

$$\begin{aligned} H(z,s,\mu ) \mathrel {\mathop :}=\sum _{n=1}^\infty \frac{e^{2\pi i \mu n}}{(n+z)^{s}} \end{aligned}$$

(47)

is the Lerch zeta function. The analytical properties of $H(z,s,\mu )$ are well known in the literature, see for instance [2]. Given $0< \mu < 1$ and $0< r < 1$, the Lerch zeta function defines a holomorphic map

$$\begin{aligned} \{ \vert z\vert < r\} \times \left( {\mathbb {C}}\smallsetminus {\mathscr {P}} \right) \rightarrow {\mathbb {C}}, \quad (z,s) \mapsto H(z,s,\mu ), \end{aligned}$$

where ${\mathscr {P}} = 1-{\mathbb {N}}_0$ is the set of (potential) poles. Consequently,

$$\begin{aligned} {\mathbb {D}} \times \left( {\mathbb {C}}\smallsetminus \frac{1}{2}(1-{\mathbb {N}}_0) \right) \rightarrow \mathrm {End}(V), \quad (z,s) \mapsto \xi (s,w,\rho ; z) \end{aligned}$$

is a holomorphic map with values in the endomorphism ring of V. This in turn shows that

$$\begin{aligned} s \mapsto \left( {\mathscr {F}}_{s,w,\rho ,1} :H^{2}({\mathbb {D}}; V)\rightarrow H^{2}({\mathbb {D}}; V) \right) \end{aligned}$$

is a family of operators depending meromorphically on s with poles contained in $\frac{1}{2}\left( 1-{\mathbb {N}}_0 \right) .$ Going back to (43) we have thus shown that ${\mathscr {L}}_{s, w,\rho }$ admits a meromorphic continuation to the half-plane $\mathrm {Re}(s) > 0 .$ To extend ${\mathscr {L}}_{s, w,\rho }$ further to the left, we take an arbitrary positive integer $k\in {\mathbb {N}}$ and iterate equation (43) k times, where in each iteration step the ‘current’ variable s gets replaced by $s+\frac{1}{2}$. This procedure yields

$$\begin{aligned} {\mathscr {L}}_{s, w, \rho } = {\mathscr {F}}_{s,w,\rho ,k} + {\mathscr {L}}_{s+\frac{k}{2}, w, \rho } \Psi ^{k}, \end{aligned}$$

(48)

where

$$\begin{aligned} {\mathscr {F}}_{s,w,\rho ,k} = \sum _{j=0}^{k-1} {\mathscr {F}}_{s+\frac{j}{2},w,\rho ,1}\Psi ^j. \end{aligned}$$

(49)

From the already established analytic properties of ${\mathscr {F}}_{s,w,\rho ,1},$ we infer from the right hand side of (49) that

$$\begin{aligned} s \mapsto \left( {\mathscr {F}}_{s,w,\rho ,k} :H^{2}({\mathbb {D}}; V)\rightarrow H^{2}({\mathbb {D}}; V) \right) \end{aligned}$$

is a meromorphic family of operators with poles in $\frac{1}{2}\left( 1-{\mathbb {N}}_0 \right)$ for all $k\in {\mathbb {N}}$. Hence, (48) shows meromorphic continuability of ${\mathscr {L}}_{s, w, \rho }$ on the half-plane $\mathrm {Re}(s) > \frac{1-k}{2}$ for arbitrary $k\in {\mathbb {N}}.$ This settles the proof of Corollary 1.3.

3 Proof of Theorem 1.4

The goal of this section is to prove Theorem 1.4. Recall from the introduction that for every positive integer n we define the subgroup $\Gamma _w^N \subset \Gamma _w$ as the kernel

$$\begin{aligned} \Gamma _w^N = \ker ( \rho _N ) \end{aligned}$$

of the one-dimensional character $\rho _N:\Gamma _w \rightarrow {\mathbb {C}}^\times$ given by

$$\begin{aligned} \rho _N (S) = -1 \quad \text {and} \quad \rho _N(T_w) = e^{\frac{2\pi i}{N}}. \end{aligned}$$

The groups $\Gamma _w^N$ are finite-index, normal subgroups of $\Gamma _w$. One important feature of these groups is that we can provide a complete set of representatives in $\Gamma _w$ of the left cosets in $\Gamma _w/\Gamma _w^N$, namely

$$\begin{aligned} \Gamma _w/\Gamma _w^N \simeq \{ T_w^a, T_w^a S : 0\le a \le N-1 \}. \end{aligned}$$

Note that this set forms an abelian group, namely

$$\begin{aligned} \Gamma _w/\Gamma _w^N \simeq {\mathbb {Z}}/N{\mathbb {Z}} \times {\mathbb {Z}}/2{\mathbb {Z}} \simeq {\mathbb {Z}}/2N{\mathbb {Z}}. \end{aligned}$$

Similarly, the groups $\Gamma _w^N$ are finite-index, normal subgroup of $\Gamma _w^1$ with a complete set of representatives in $\Gamma _w^1$ of the left cosets in $\Gamma _w^1/\Gamma _w^N$ given by

$$\begin{aligned} \Gamma _w^1/\Gamma _w^N \simeq \{ T_w^a : 0\le a \le N-1 \} \simeq {\mathbb {Z}}/N{\mathbb {Z}}. \end{aligned}$$

(50)

Thus, the action of $\Gamma _w^N$ on ${\mathbb {H}}^2$ has as fundamental domain the following disjoint union of N translates of ${\mathscr {F}}^1(w)$ (ignoring the boundaries), where ${\mathscr {F}}^1(w)$ is the fundamental domain for $\Gamma ^1_w$ in (8):

$$\begin{aligned} {\mathscr {F}}^N(w) = \bigcup _{a = 0}^{N-1} T_w^a.{\mathscr {F}}^1(w). \end{aligned}$$

(51)

Thus, the quotients $X_w^N = \Gamma _w^N\backslash {\mathbb {H}}^2$ are covers of $X_w^1$ of degree N. Observe furthermore that $X_w^1 = \Gamma _w^1\backslash {\mathbb {H}}^2$ is a smooth hyperbolic surface (no conical singularities!) with one funnel ($n_f = 1$), two cusps ($n_c = 2$) and genus zero ($g = 0$). This implies that $X_w^1$ has Euler characteristic

$$\begin{aligned} \chi (X_w^1) = 2-2g-n_c-n_f = -1. \end{aligned}$$

Consequently, $X_w^N$ is a smooth hyperbolic surface with Euler characteristic

$$\begin{aligned} \chi (X_w^N) = N \cdot \chi (X_w^1) = -N. \end{aligned}$$

Part (ii) of Theorem 1.4 is now a straightforward consequence of two well-known results in the spectral theory of hyperbolic surfaces, which we recall here. First, given an arbitrary torsion-free, finitely generated Fuchsian group $\Gamma$, the result of Borthwick–Judge–Perry [7] asserts that the zeros of $Z_\Gamma (s)$ in $\left\{ \mathrm {Re}(s) > \frac{1}{2} \right\}$ correspond, with multiplicities, to the $L^{2}$-eigenvalues $\lambda = s(1-s) \in (0, \frac{1}{4})$ of the Laplacian $\Delta _X$ on $X = \Gamma \backslash {\mathbb {H}}^2$. Second, from Ballmann–Mathiesen–Mondal [3] we know that the number of eigenvalues of $\Delta _X$ in $(0, \frac{1}{4})$ is bounded above by $- \chi (X)$, where $\chi (X)$ denotes the Euler characteristic of the surface X.

Let us now prove Part (i) of Theorem 1.4 which says that $s= \delta (w)$ is the unique zero of $Z_{\Gamma _{w}}(s)$ in the half-plane $\mathrm {Re}(s) > \frac{1}{2}.$ Unfortunately, the result of Borthwick–Judge–Perry mentioned above does not apply directly to the Hecke triangle group $\Gamma _w$, since it is not torsion-free (indeed, $\Gamma _w$ contains the element S which satisfies $S^2 = {{\,\mathrm{id}\,}}$). We need some additional arguments to bypass this issue. We will write $\delta = \delta (w)$ for the remainder of this section.

We can use the product definition of $Z_{\Gamma _w}(s,\rho )$ in (12) to compute its logarithmic derivative in the half-plane $\mathrm {Re}(s) > \delta$ as

$$\begin{aligned} \frac{Z_{\Gamma _w}'(s)}{Z_{\Gamma _w}(s)} = \sum _{[\gamma ]\in [\Gamma _w]_{p}} \sum _{k=0}^{\infty } \sum _{m=1}^\infty \chi (\gamma ^m) \ell (\gamma ) e^{-m(s+k)\ell (\gamma )}, \end{aligned}$$

(52)

where $\chi (\gamma ) \mathrel {\mathop :}={{\,\mathrm{tr}\,}}_V( \rho (\gamma ) )$ is the character of the representation $\rho :\Gamma _w \rightarrow \mathrm {U}(V)$.

Now we can either invoke the Venkov–Zograf factorization formula (see [47, 48] or [10, Theorem 6.1]) or directly prove that the Selberg zeta function of $\Gamma _w^1$ factorizes as

$$\begin{aligned} Z_{\Gamma _{w}^{1}}(s) = Z_{\Gamma _{w}}(s)Z_{\Gamma _{w}}(s,\rho _1). \end{aligned}$$

(53)

Applying Part (ii) to the case $n=1$ shows that $Z_{\Gamma _{w}^{1}}(s)$ has exactly one zero in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$. By Corollary 1.3, both $Z_{\Gamma _{w}}(s)$ and $Z_{\Gamma _{w}}(s,\rho _1)$ are holomorphic in the half-plane $\mathrm {Re}(s) > \frac{1}{2}$. Combining these facts with the factorization in (53) immediately implies that $Z_{\Gamma _{w}}(s)$ has at most one zero in $\mathrm {Re}(s) > \frac{1}{2}$, possibly at $s=\delta$. Let us suppose by contradiction that $Z_{\Gamma _{w}}(s)$ has no zero at $s=\delta$. In that case $Z_{\Gamma _{w}}(s,\rho _1)$ has one zero at $\delta$, and $Z_{\Gamma _{w}}(s)$ has no zeros at all in the half-plane $\mathrm {Re}(s) > \frac{1}{2}.$ Then, using (52) and standard methods of analytic number theory we can use this information on the zeros of $Z_{\Gamma _{w}}(s, \rho _1)$ and $Z_{\Gamma _{w}}(s)$ to obtain the asymptotics

$$\begin{aligned} \sum _{\begin{array}{c} [\gamma ] \in [\Gamma _w]_p \\ \ell (\gamma ) \le x \end{array}} \rho _1(\gamma ) \ell ( \gamma ) \sim e^{\delta x}, \quad x\rightarrow \infty \end{aligned}$$

and

$$\begin{aligned} \sum _{\begin{array}{c} [\gamma ] \in [\Gamma _w]_p \\ \ell (\gamma ) \le x \end{array}} \ell ( \gamma ) = O_\varepsilon (e^{ (\frac{1}{2}+\varepsilon )x }), \quad x \rightarrow \infty \end{aligned}$$

for all $\varepsilon > 0$. We refer to [6, Chapter 14.5] for the details. Now note that since $\rho _1(\gamma ) \in \{ \pm 1 \}$, we can trivially bound

$$\begin{aligned} e^{\delta x} \sim \sum _{\begin{array}{c} [\gamma ] \in [\Gamma ]_p \\ \ell (\gamma ) \le x \end{array}} \rho _1(\gamma )\ell ( \gamma ) \le \sum _{ \begin{array}{c} [\gamma ] \in [\Gamma ]_p \\ \ell (\gamma ) \le x \end{array}}\ell ( \gamma ) = O_\varepsilon (e^{ (\frac{1}{2}+\varepsilon )x }) \end{aligned}$$

(54)

for all $\varepsilon > 0$ as $x\rightarrow \infty$. Comparing the exponents on both sides of (54) forces $\delta \le \frac{1}{2}$, a contradiction to (2). Hence, $s=\delta$ is a zero of $Z_{\Gamma _w}(s)$ and there are no other zeros in $\mathrm {Re}(s) > \frac{1}{2}$, as claimed.

It remains to show Part (iii), the proof of which will occupy the remainder of this section. We will adapt the argument given in [21] (where it was used to prove a similar statement for Schottky groups). Let us introduce the family of operators

$$\begin{aligned} {\mathscr {L}}_{s,w}^{(\theta )}:H^2({\mathbb {D}})\rightarrow H^2({\mathbb {D}}) \end{aligned}$$

defined for all real parameters $\theta \in {\mathbb {R}}$ by

$$\begin{aligned} {\mathscr {L}}_{s,w}^{(\theta )}f(z) \mathrel {\mathop :}=\sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} e^{2\pi i n \theta } \gamma _{n}'(z)^s f(\gamma _n(z)). \end{aligned}$$

(55)

We then have the following result.

Lemma 3.1

We have the factorization

$$\begin{aligned} Z_{\Gamma _w^N}(s) = \prod _{a = 0}^{N-1} \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \cdot \prod _{a = 0}^{N-1} \det \left( 1+{\mathscr {L}}_{s,w}^{(a/N)} \right) \end{aligned}$$

(56)

and every factor $\det \left( 1 \pm {\mathscr {L}}_{s,w}^{(a/N)} \right)$ is holomorphic in the half-plane $\mathrm {Re}(s) > \frac{1}{2}.$

Proof

Let $\xi _N :\Gamma _w \rightarrow {\mathbb {C}}^\times$ be the representation given by

$$\begin{aligned} \xi _N (S) = 1 \quad \text {and} \quad \xi _N(T_w) = e^{\frac{2\pi i}{N}}. \end{aligned}$$

and recall that $\rho _1$ is the representation of $\Gamma _w$ given by

$$\begin{aligned} \rho _1(S) = -1 \quad \text {and} \quad \rho _1(T_w) = 1. \end{aligned}$$

The set of irreducible representations of the (abelian) symmetry group

$$\begin{aligned} \Gamma _w/\Gamma ^N_w \simeq {\mathbb {Z}}/N{\mathbb {Z}} \times {\mathbb {Z}}/2{\mathbb {Z}} \end{aligned}$$

is given by the collection of the 2N characters

$$\begin{aligned} \{ \rho _1 \xi _N^a, \xi _N^a : 0\le a \le N-1 \}. \end{aligned}$$

By the Venkov–Zograf formula (citations as above) we can factorize the Selberg zeta function of the subgroup $\Gamma _w^N \subset \Gamma _w$ into a product of twisted Selberg zeta functions of $\Gamma _w$ as

$$\begin{aligned} Z_{\Gamma _w^N}(s) = \prod _{a = 0}^{N-1} Z_{\Gamma _w}(s, \xi _N^a) \cdot \prod _{a = 0}^{N-1} Z_{\Gamma _w}(s, \rho _1 \xi _N^a). \end{aligned}$$

(Note that this is a straightforward generalization of the factorization in (53).) Applying Theorem 1.1 to each of the factors appearing on the right hand side, we obtain

$$\begin{aligned} Z_{\Gamma _w^N}(s) = \prod _{a = 0}^{N-1}\det \left( 1- {\mathscr {L}}_{s,w, \xi _N^a} \right) \cdot \prod _{a = 0}^{N-1} \det \left( 1- {\mathscr {L}}_{s,w, \rho _1 \xi _N^a} \right) . \end{aligned}$$

Now note that we can use the notation introduced in (55) to write

$$\begin{aligned} {\mathscr {L}}_{s,w, \xi _N^a} = {\mathscr {L}}_{s,w}^{(a/N)} \quad \text {and} \quad {\mathscr {L}}_{s,w, \rho _1 \xi _N^a} = - {\mathscr {L}}_{s,w}^{(a/N)}, \end{aligned}$$

completing the proof. $\square$

In light of Lemma 3.1, the proof of Part (iii) can be explained as follows. For $\theta$ close to zero the operator ${\mathscr {L}}_{s,w}^{(\theta )}$ is ‘close’ to ${\mathscr {L}}_{s,w} = {\mathscr {L}}_{s,w}^{(0)}$. In particular, if $\theta \approx 0$ then the Fredholm determinant $\det \left( 1- {\mathscr {L}}_{s,w}^{(\theta )}\right)$ must have a zero close to $s = \delta$, which is a zero of $\det \left( 1- {\mathscr {L}}_{s,w}\right) = Z_{\Gamma _w}(s)$. This in turn implies that each factor $\det \left( 1- {\mathscr {L}}_{s,w}^{a/N}\right)$ appearing on the right hand side of (56) will produce a zero arbitrarily close to $s=\delta$, provided the ratio a/N is sufficiently small. To materialize this idea, we need the following result.

Lemma 3.2

For all $\sigma \mathrel {\mathop :}=\mathrm {Re}(s) > \frac{1}{2}$, there exists a constant $C>0$ such that

$$\begin{aligned} \Vert {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(\theta )} \Vert _1 \le C \theta ^{2\sigma - 1} \end{aligned}$$

for all $\theta \in [0,1)$.

Proof

If $\theta = 0$ there is nothing to prove, so we assume that $\theta > 0.$ Notice that we can write

$$\begin{aligned} {\mathscr {L}}_{s,w}^{(\theta )} = \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0 \}} e^{ 2 \pi i \theta n } \nu _s(\gamma _n^{-1}) \end{aligned}$$

where $\nu _s(\gamma _n^{-1}) = \nu _{s,\mathbf{1 }}(\gamma _n^{-1})$ is the operator given by (24). Thus, we can write

$$\begin{aligned} {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(\theta )} = \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0 \}} \left( 1- e^{ 2 \pi i \theta n } \right) \nu _s(\gamma _n^{-1}). \end{aligned}$$

Using the identity

$$\begin{aligned} \left| 1- e^{ 2 \pi i \theta n } \right| = 2 \vert \sin (\pi \theta n)\vert \end{aligned}$$

and applying the triangle inequality gives

$$\begin{aligned} \Vert {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(\theta )} \Vert _1 \le 2 \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0 \}} \vert \sin (\pi \theta n)\vert \Vert \nu _s(\gamma _n^{-1}) \Vert _1. \end{aligned}$$

Using the bound for the trace norm for $\nu _s(\gamma _n^{-1})$ in Proposition 2.1, we obtain

$$\begin{aligned} \Vert {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(\theta )} \Vert _1 \le C \sum _{n = 1}^\infty \frac{\vert \sin (\pi \theta n)\vert }{\vert n\vert ^{2\sigma }} \end{aligned}$$

(57)

for some $C > 0$ depending solely on s and w. We split the sum on the right into two parts:

$$\begin{aligned} \sum _{n = 1}^\infty \frac{\vert \sin ( \pi \theta n)\vert }{\vert n\vert ^{2\sigma }} = \sum _{1 \le n < 1/(3\theta )} \frac{\vert \sin ( \pi \theta n)\vert }{\vert n\vert ^{2\sigma }} + \sum _{n \ge 1/(3\theta )} \frac{\vert \sin ( \pi \theta n)\vert }{\vert n\vert ^{2\sigma }}. \end{aligned}$$

(58)

Using the elementary bound $\vert \sin (\pi x) \vert < 2 \vert x\vert$ for all $\vert x\vert < 1/2$, we estimate the first sum on the right as

$$\begin{aligned} \sum _{1 \le n< 1/(3\theta )} \frac{\vert \sin ( \pi \theta n)\vert }{\vert n\vert ^{2\sigma }} \ll \sum _{1 \le n < 1/(3\theta )} \frac{ n \theta }{\vert n\vert ^{2\sigma }} \ll \theta ^{2\sigma -1}. \end{aligned}$$

(59)

The second sum on the right hand side of (58) can be estimated as

$$\begin{aligned} \sum _{n \ge 1/(3\theta )} \frac{\vert \sin ( \pi \theta n)\vert }{\vert n\vert ^{2\sigma }} \le \sum _{n \ge 1/(3\theta )} \frac{1}{\vert n\vert ^{2\sigma }} \ll \int _{1/(3\theta )}^\infty \frac{dx}{x^{2\sigma }} \ll \theta ^{2\sigma - 1}. \end{aligned}$$

(60)

Combining (59) and (60) shows that the sum on the right of (57) is of size at most $O(\theta ^{2\sigma - 1})$. This completes the proof of Lemma 3.2. $\square$

We are now ready to prove Part (iii) of Theorem 1.4. In what follows, we assume that s lies in the half-plane $\sigma \mathrel {\mathop :}=\mathrm {Re}(s) > \frac{1}{2}$. From Part (i) we know that on this half-plane, $Z_{\Gamma _w}(s)$ vanishes only at $s=\delta$. Hence for any fixed $0< \varepsilon < \delta -\frac{1}{2}$, we have

$$\begin{aligned} C(\varepsilon ,w) \mathrel {\mathop :}=\inf _{ \vert s- \delta \vert = \varepsilon } \vert Z_{\Gamma _w}(s) \vert > 0. \end{aligned}$$

On the other hand, Theorem 1.1 and the estimate in (18) show that

$$\begin{aligned}&\left| Z_{\Gamma _w}(s) - \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \right| = \left| \det \left( 1- {\mathscr {L}}_{s,w} \right) - \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \right| \\ {}&\le \Vert {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(a/N)} \Vert _1 \exp \left( \Vert {\mathscr {L}}_{s,w}^{(a/N)} \Vert _1 + \left\| {\mathscr {L}}_{s,w} \right\| _1 + 1 \right) . \end{aligned}$$

By Proposition 2.1, the trace norms $\Vert {\mathscr {L}}_{s,w} \Vert _1$ and $\Vert {\mathscr {L}}_{s,w}^{(a/N)} \Vert _1$ are bounded from above by a constant depending only on s and w, so we have

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \right| \le C_1 \Vert {\mathscr {L}}_{s,w} - {\mathscr {L}}_{s,w}^{(a/N)} \Vert _1 \end{aligned}$$

for some constant $C_1 = C_1(s,w)$ not depending on N. Applying Lemma 3.2 shows furthermore that on the circle $\vert s-\delta \vert = \varepsilon$ we have

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \right| \le C_2 \left( \frac{a}{N} \right) ^{2\sigma - 1} \le C_2 \left( \frac{a}{N} \right) ^{2(\delta - \varepsilon ) - 1}, \end{aligned}$$

where $C_2 = C_2(\varepsilon ,w)$ is independent of N. Now we can choose a constant $c > 0$ sufficiently small such that the bound

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \right| < C(\varepsilon , w) \end{aligned}$$

(61)

holds true whenever $0 \le a \le c N.$ But by Rouché’s theorem for holomorphic functions, this bound forces the (holomorphic) function

$$\begin{aligned} s \mapsto \det \left( 1- {\mathscr {L}}_{s,w}^{(a/N)} \right) \end{aligned}$$

to vanish at some point $s\in {\mathbb {C}}$ with $\vert s-\delta \vert < \varepsilon .$ By Lemma 3.1 this implies that $Z_{\Gamma _w^N}(s)$ has at least cN zeros (counted with multiplicities) in the disk $\vert s-\delta \vert < \varepsilon .$ Finally, the result of Borthwick–Judge–Perry [7] shows that all these zeros lie on the real interval $(\delta - \varepsilon , \delta ]$ and they correspond one to one to the eigenvalues $s(1-s)$ of the Laplacian on $X_w^N$. The proof of Theorem 1.4 is now complete.

4 Hausdorff Dimension of Hecke Triangle Groups

This section is devoted to the proofs of Theorem 1.7 and Theorem 1.9. We will work solely with the trivial one-dimensional representation $\rho = \mathbf{1 }$. Therefore we will drop the representation from the notation of the transfer operator, writing only ${\mathscr {L}}_{s,w}$ instead of ${\mathscr {L}}_{s,\rho , \mathbf{1 }}$. Recall that ${\mathscr {L}}_{s,w}$ acts on the classical Bergman space $H^2({\mathbb {D}})$ consisting of holomorphic functions on the unit disk with bounded $L^2$-norm. Every function $f\in H^2({\mathbb {D}})$ can be Taylor expanded around $z=0$ as

$$\begin{aligned} f(z) = \sum _{i=0}^{\infty } c_{i}z^{i} \end{aligned}$$

(62)

for some suitable coefficients $c_i \in {\mathbb {C}}$. The proofs of Theorems 1.7 and 1.9 are independent but both rely on the following identity.

Proposition 4.1

For all $\mathrm {Re}(s) > \frac{1}{2}$ and for every $f\in H^2({\mathbb {D}})$ with Taylor expansion as in (62), we have the absolutely convergent expression

$$\begin{aligned} {\mathscr {L}}_{s,w} f(z) = \sum _{i=0}^\infty \sum _{j=0}^\infty a_{i,j}(s,w) c_j z^i \end{aligned}$$

(63)

with

$$\begin{aligned} a_{i,j}(s,w) = \left( (-1)^{i+j} + 1 \right) \frac{\zeta (2s+i+j)}{w^{2s+i+j}} { 2s+i+j-1 \atopwithdelims ()i}, \end{aligned}$$

where we use the notation

$$\begin{aligned} { r \atopwithdelims ()k} = \frac{r(r-1)\cdots (r-k+1)}{k!} \end{aligned}$$

for all $r\in {\mathbb {C}}$ and $k\in {\mathbb {N}}_0$.

Proof

Fix a point $z\in {\mathbb {D}}$. By the definition of the transfer operator in (20), we write

$$\begin{aligned} {\mathscr {L}}_{s,w} f(z)&= \sum _{n\in {\mathbb {Z}}\smallsetminus \{ 0\}} \gamma _n'(z)^{s} f(\gamma _n(z))\\&= \sum _{n = 1}^{\infty } \frac{1}{(nw+z)^{2s}} f\left( \frac{-1}{nw+z} \right) + \sum _{n = 1}^{\infty } \frac{1}{(nw-z)^{2s}} f\left( \frac{1}{nw-z} \right) . \end{aligned}$$

Inserting the Taylor expansion for f into the previous line, we obtain

$$\begin{aligned} {\mathscr {L}}_{s,w} f(z) = \sum _{n=1}^{\infty } \sum _{j=0}^\infty c_{j} \left[ \frac{(-1)^j }{(nw+z)^{j+2s}} + \frac{1}{(nw-z)^{j+2s}} \right] . \end{aligned}$$

(64)

We can use the generalized binomial theorem

$$\begin{aligned} (1+z)^{r} = \sum _{i=0}^{\infty } { r \atopwithdelims ()i} z^{i}, \end{aligned}$$

valid for all $r\in {\mathbb {C}}$ and $\vert z\vert < 1$, to rewrite the bracketed expression in (64) as

$$\begin{aligned}&\frac{(-1)^j }{(nw+z)^{j+2s}} + \frac{1}{(nw-z)^{j+2s}} \\&\quad = \frac{(-1)^j }{(nw)^{j+2s}}\left( 1+ \frac{z}{nw} \right) ^{-2s-j} + \frac{1}{(nw)^{j+2s}} \left( 1-\frac{z}{nw} \right) ^{-2s-j}\\&\quad = \sum _{i=0}^\infty \left( (-1)^j + (-1)^i \right) \frac{1}{(nw)^{2s+i+j}}{ -2s-j \atopwithdelims ()i} z^i\\&\quad = \sum _{i=0}^\infty \left( (-1)^{i+j} + 1 \right) \frac{1}{(nw)^{2s+i+j}}{ 2s+i+j-1 \atopwithdelims ()i} z^i, \end{aligned}$$

where in the last line we used the relation

$$\begin{aligned} { -2s-j \atopwithdelims ()i} = (-1)^i { 2s+i+j-1 \atopwithdelims ()i}. \end{aligned}$$

Inserting this into the right hand side of (64), we obtain

$$\begin{aligned} {\mathscr {L}}_{s,w} f(z) = \sum _{n=1}^{\infty } \sum _{j=0}^\infty \sum _{i=0}^\infty \left( (-1)^{i+j} + 1 \right) \frac{1}{(nw)^{2s+i+j}}{ 2s+i+j-1 \atopwithdelims ()i} c_j z^i.\nonumber \\ \end{aligned}$$

(65)

Let us now argue why the sum in (65) is absolutely convergent. Using orthogonality of the functions $\{ z^j\}_{j\in {\mathbb {N}}_0}$ in $H^2({\mathbb {D}})$, we compute the $L^2$-norm of $f\in H^2({\mathbb {D}})$ as

$$\begin{aligned} \Vert f\Vert ^{2} = \sum _{j=0}^{\infty } \vert c_j\vert ^2 \int _{{\mathbb {D}}} \vert z\vert ^{2j} {{\,\mathrm{dvol}\,}}(z) = \pi \sum _{j=0}^{\infty } \frac{\vert c_j\vert ^2}{j+1}. \end{aligned}$$

(66)

From (66), we deduce that

$$\begin{aligned} \vert c_j\vert \le \frac{\sqrt{j+1}}{\sqrt{\pi }} \Vert f\Vert \end{aligned}$$

(67)

for all $j\ge 0.$ The bound for binomial coefficients

$$\begin{aligned} {r \atopwithdelims ()k} \le {\lceil r\rceil \atopwithdelims ()k} \le 2^{\lceil r\rceil }, \end{aligned}$$

valid for all positive reals r and all integers $0\le k\le r$, yields

$$\begin{aligned} \left| { 2 s +i+j-1 \atopwithdelims ()i} \right| \le { 2 \vert s\vert +i+j-1 \atopwithdelims ()i} \ll 2^{ 2 \vert s\vert + i + j }. \end{aligned}$$

(68)

Combining (67) and (68) shows that the absolute value of each term appearing in the sum (65) is bounded from above by

$$\begin{aligned} \left| \left( (-1)^{i+j} + 1 \right) \frac{1}{(nw)^{2s+i+j}}{ 2s+i+j-1 \atopwithdelims ()i} c_j z^i \right| \ll \frac{\sqrt{j+1}}{ n^{2 \mathrm {Re}(s) +i+j}} \left( \frac{2}{w} \right) ^{i+j} \Vert f\Vert , \end{aligned}$$

where the implied constant depends solely on the variable s. Since $w>2$, this clearly shows that the triple sum on the right hand side of (65) is absolutely convergent for $\mathrm {Re}(s) > \frac{1}{2}.$

Finally, recalling the definition of the Riemann zeta function

$$\begin{aligned} \zeta (s) = \sum _{n=1}^{\infty } \frac{1}{n^s} \end{aligned}$$

for $\mathrm {Re}(s) > 1$, we can interchange sums in (65) (allowed by absolute convergence) to write

$$\begin{aligned} {\mathscr {L}}_{s,w} f(z) = \sum _{i=0}^\infty \sum _{j=0}^\infty \left( (-1)^{i+j} + 1 \right) \frac{\zeta (2s+i+j)}{w^{2s+i+j}} { 2s+i+j-1 \atopwithdelims ()i} c_j z^i, \end{aligned}$$

completing the proof of Proposition 4.1. $\square$

4.1 Proof of Theorem 1.7

In this subsection, we fix $w > 2$ and we write $\delta = \delta (w).$ Motivated by Proposition 4.1, we define for every integer $k>1$ the operator

$$\begin{aligned} {\mathscr {A}}_{s,w,k} :H^2({\mathbb {D}}) \rightarrow H^2({\mathbb {D}}) \end{aligned}$$

acting on functions $f(z) = c_0 + c_1 z + c_2 z^2+\cdots$ by

$$\begin{aligned} {\mathscr {A}}_{s,w,k} f(z) \mathrel {\mathop :}=\sum _{i=0}^{k-1} \sum _{j=0}^{k-1} a_{i,j}(s,w) c_j z^i, \end{aligned}$$

(69)

where

$$\begin{aligned} a_{i,j}(s,w) = \left( (-1)^{i+j} + 1 \right) \frac{\zeta (2s+i+j)}{w^{2s+i+j}} { 2s+i+j-1 \atopwithdelims ()i}. \end{aligned}$$

Let $V_k$ denote the subspace of $H^2({\mathbb {D}})$ spanned by the functions $\phi _m(z) = z^m$ with $0\le m < k$. Notice that the operator ${\mathscr {A}}_{s,w,k}$ is a finite-rank operator acting by zero on the orthogonal complement of $V_k$. On the subspace $V_k$, the action of ${\mathscr {A}}_{s,w,k}$ is represented by the $k\times k$-matrix

$$\begin{aligned} A_k(s,w) = \left( a_{i,j}(s,w)\right) _{0\le i,j < k} \end{aligned}$$

with respect to the basis $\{ \phi _0, \dots , \phi _{k-1} \}.$ In particular, the Fredholm determinant of $1 - {\mathscr {A}}_{s,w,k}$ agrees with the usual linear algebra determinant of $I_k-A_k(s,w)$:

$$\begin{aligned} \det \left( 1-{\mathscr {A}}_{s,w,k} \right) = \det (I_k-A_k(s,w)) =: D_k(s,w). \end{aligned}$$

(70)

The next result shows that the sequence of operators ${\mathscr {A}}_{s,w,k}$ converges exponentially fast to ${\mathscr {L}}_{s,w}$ as $k\rightarrow \infty$ with respect to the trace norm.

Lemma 4.2

For all $\mathrm {Re}(s)>\frac{1}{2}$ and all integers $k\ge 1$, we have

$$\begin{aligned} \Vert {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} \Vert _1 \le C k^{3/2} \left( \frac{w}{2} \right) ^{- k}, \end{aligned}$$

where $C = C(s,w) > 0$ is independent of k.

Proof

From formula (63), we can estimate

$$\begin{aligned} \vert a_{i,j}(s,w)\vert \le 2 \frac{\zeta (2\sigma )}{w^{i+j+2\sigma }} { 2 \vert s\vert +i+j-1 \atopwithdelims ()i} \end{aligned}$$

(71)

for all $0\le i,j < \infty$, where we have used

$$\begin{aligned} \vert \zeta (2s+i+j) \vert \le \zeta (2\sigma + i+j) \le \zeta (2\sigma ) \end{aligned}$$

with $\sigma = \mathrm {Re}(s)$. Recall from (68) that we can bound the binomial coefficient as

$$\begin{aligned} { 2 \vert s\vert +i+j-1 \atopwithdelims ()i} \ll 2^{ 2 \vert s\vert + i + j }, \end{aligned}$$

which when inserted into (71) gives

$$\begin{aligned} \vert a_{i,j}(s,w)\vert \ll \left( \frac{w}{2} \right) ^{-(i+j)} \end{aligned}$$

(72)

with implied constant depending only on s and w (but not on i nor on j). Now let $f\in H^2({\mathbb {D}})$ be a function whose Taylor expansion is as in (62). By Proposition 4.1 and the definition of ${\mathscr {A}}_{s,w,k}$, we can write

$$\begin{aligned} ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})f(z) = \sum _{ \begin{array}{c} i,j \in {\mathbb {N}}_0 \\ \max (i,j) \ge k \end{array}} a_{i,j}(s,w) c_j z^i. \end{aligned}$$

(73)

Recall that the functions $\psi _{j} (z) = \sqrt{\frac{j+1}{\pi }} z^j$ with $j\in {\mathbb {N}}_0$ provide an orthonormal basis for $H^2({\mathbb {D}}),$ so it follows from (73) that

$$\begin{aligned} ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j = \sum _{ \begin{array}{c} i \in {\mathbb {N}}_0 \\ \max (i,j) \ge k \end{array}} a_{i,j}(s,w) \sqrt{\frac{j+1}{i+1}} \psi _i \end{aligned}$$

(74)

and noticing that $\Vert \psi _i\Vert = 1$ gives

$$\begin{aligned} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \le \sum _{ \begin{array}{c} i \in {\mathbb {N}}_0 \\ \max (i,j) \ge k \end{array}} \vert a_{i,j}(s,w) \vert \sqrt{\frac{j+1}{i+1}}. \end{aligned}$$

Inserting the bound into (72) yields

$$\begin{aligned} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \ll \sum _{ \begin{array}{c} i \in {\mathbb {N}}_0 \\ \max (i,j) \ge k \end{array}} \left( \frac{w}{2} \right) ^{-(i+j)} \sqrt{\frac{j+1}{i+1}}. \end{aligned}$$

Assuming first that $j\ge k$, we can estimate this as

$$\begin{aligned} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \ll \sum _{i\ge 1} \left( \frac{w}{2} \right) ^{-(i+j)} \sqrt{\frac{j+1}{i+1}} \ll \sqrt{j+1} \left( \frac{w}{2} \right) ^{-j}. \end{aligned}$$

Similarly, assuming that $j<k$, we have

$$\begin{aligned} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \ll \sum _{i\ge k} \left( \frac{w}{2} \right) ^{-(i+j)} \sqrt{\frac{j+1}{i+1}} \ll \left( \frac{w}{2} \right) ^{-k}. \end{aligned}$$

Combining the two previous bounds, we can write

$$\begin{aligned} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \le \sqrt{j+1} \left( \frac{w}{2} \right) ^{-\max (j,k)}. \end{aligned}$$

(75)

Using the singular value estimate in (16) and the estimate in (75), we obtain for all $n\ge 1$ the estimate

$$\begin{aligned} \mu _n( {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} ) \le \sum _{j \ge n} \Vert ({\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k})\psi _j \Vert \ll \sum _{j \ge n} \sqrt{j+1} \left( \frac{w}{2} \right) ^{-\max (j,k)}. \end{aligned}$$

The sum on the right may be estimated by treating the cases $k<n$ and $k\ge n$ separately. We obtain

$$\begin{aligned} \mu _n( {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} ) \ll \sqrt{\max (n,k)} \left( \frac{w}{2} \right) ^{-\max (n,k)}. \end{aligned}$$

With this bound on singular values, we can finally estimate the trace norm as

$$\begin{aligned} \Vert {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} \Vert _1&= \sum _{n=1}^\infty \mu _n( {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} )\\&\ll \sum _{n=1}^\infty \sqrt{\max (n,k)} \left( \frac{w}{2} \right) ^{-\max (n,k)}\\&= \sum _{n=1}^k \sqrt{k} \left( \frac{w}{2} \right) ^{-k} + \sum _{n>k} \sqrt{n} \left( \frac{w}{2} \right) ^{-n}\\&\ll k^{3/2} \left( \frac{w}{2} \right) ^{-k}. \end{aligned}$$

This finishes the proof of Lemma 4.2. $\square$

Using the Fredholm determinant identities from Theorem 1.1 and (70) in conjuction with the bound on Fredholm determinants in (18), we obtain

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - D_k(s,w) \right|&= \left| \det \left( 1-{\mathscr {L}}_{s,w} \right) - \det \left( 1-{\mathscr {A}}_{s,w,k} \right) \right| \\&\le \Vert {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} \Vert _1 \exp \left( \Vert {\mathscr {L}}_{s,w} \Vert _1 + \Vert {\mathscr {A}}_{s,w,k} \Vert _1 + 1 \right) \\&\le \Vert {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} \Vert _1 \exp \left( 2\Vert {\mathscr {L}}_{s,w} \Vert _1 + \Vert {\mathscr {L}}_{s,w} - {\mathscr {A}}_{s,w,k} \Vert _1 + 1 \right) . \end{aligned}$$

Using Lemma 4.2 in the previous line gives

$$\begin{aligned} \left| Z_{\Gamma _w}(s) - D_k(s,w) \right| \le C k^{3/2} \left( \frac{w}{2}\right) ^{-k} \end{aligned}$$

(76)

for some constant $C = C(s,w) > 0$, proving the first part of Theorem 1.7.

To conclude the proof of Theorem 1.7, let $\varepsilon > 0$ be small enough so that $\delta > \frac{1}{2}+\varepsilon .$ Using Rouché’s theorem, the bound in (76), and the fact that $Z_{\Gamma _w}(s)$ has precisely one zero in $\mathrm {Re}(s) \ge \frac{1}{2}+\varepsilon$, we can show that $D_k(s,w)$ has exactly one zero in $\mathrm {Re}(s) \ge \frac{1}{2} +\varepsilon$, provided k is large enough. (In fact, we can use an argument similar to the final argument in our proof of Part (iii) of Theorem 1.4.) Let $s_k(w)$ denote this zero. It is clear from (76) that $s_k(w)$ converges to $\delta$ as k tends to infinity. In fact, we show

Lemma 4.3

Notations being as above, we have for k sufficiently large

$$\begin{aligned} \vert \delta - s_k(w) \vert \le C' k^{3/2} \left( \frac{w}{2} \right) ^{-k} \end{aligned}$$

where $C' > 0$ is some constant depending only on w.

Proof

By the mean value theorem, there exists some $t_k \in {\mathbb {C}}$ in the line segment joining $s_k(w)$ and $\delta$ such that

$$\begin{aligned} Z_{\Gamma _w}'(t_k)(\delta - s_k(w))&= Z_{\Gamma _w}(\delta ) - Z_{\Gamma _w}(s_k(w))\\&= - Z_{\Gamma _w}(s_k(w))\\&= D_k(s_k(w)) - Z_{\Gamma _w}(s_k(w)), \end{aligned}$$

and thus by (76) we get

$$\begin{aligned} \vert Z_{\Gamma _w}'(t_k) \vert \cdot \vert \delta - s_k(w) \vert \le C k^{3/2} \left( \frac{w}{2} \right) ^{-k} \end{aligned}$$

(77)

Now notice that $t_k$ must also converge to $\delta$ as $k\rightarrow \infty$ and in particular we have

$$\begin{aligned} \lim _{k\rightarrow 0} Z_{\Gamma _w}'(t_k) = Z_{\Gamma _w}'(\delta ). \end{aligned}$$

Note that $Z_{\Gamma _w}'(\delta ) \ne 0$, since $s = \delta$ is a simple zero of $Z_{\Gamma _w}(s)$ by Part (i) of Theorem 1.4. Thus, for all k large enough we have

$$\begin{aligned} \vert Z_{\Gamma _w}'(t_k) \vert \ge \frac{1}{2} \vert Z_{\Gamma _w}'(\delta )\vert > 0. \end{aligned}$$

Inserting this into (77) we obtain

$$\begin{aligned} \vert \delta - s_k(w) \vert \le \frac{C}{\vert Z_{\Gamma _w}'(t_k) \vert } k^{3/2} \left( \frac{w}{2} \right) ^{-k} \le \frac{2C}{\vert Z_{\Gamma _w}'(\delta ) \vert } k^{3/2} \left( \frac{w}{2} \right) ^{-k}, \end{aligned}$$

completing the proof. $\square$

4.2 1-Eigenfunctions of the Transfer Operator

The results of this subsection are crucial for the proof of Theorem 1.9. Recall that $Z_{\Gamma _w}(s)$ vanishes at $s = \delta$. We deduce from Theorem 1.1 and the general theory of Fredholm determinants that 1 is an eigenvalue of ${\mathscr {L}}_{\delta ,w} :H^2({\mathbb {D}})\rightarrow H^2({\mathbb {D}})$. That is, there exists a nonzero function $f \in H^{2}({\mathbb {D}})$ satisfying

$$\begin{aligned} f(z) = {\mathscr {L}}_{\delta ,w} f(z), \quad z\in {\mathbb {D}}. \end{aligned}$$

(78)

In this subsection, we investigate the coefficients $c_i$ of the 1-eigenfunctions f of ${\mathscr {L}}_{\delta ,w}$ in the Taylor expansion

$$\begin{aligned} f(z) = \sum _{i=0}^{\infty } c_{i}z^{i}. \end{aligned}$$

(79)

The main result of this subsection is

Proposition 4.4

Assume that $w\ge 3$ and let $f\ne 0$ be a 1-eigenfunction of ${\mathscr {L}}_{\delta ,w}$. Then, the Taylor coefficients of f in (79) have the following properties:

• (i) For all odd i, we have $c_i = 0$. In other words, f is an even function.

• (ii) For all even $i\ge 2$, we have the bound

  $$\begin{aligned} \vert c_i\vert \le \frac{3 \zeta (3) }{\sqrt{\pi }} (i+1)\left( \frac{3}{2w}\right) ^{i+1} \Vert f\Vert . \end{aligned}$$

• (iii) Moreover, the constant term of f satisfies

  $$\begin{aligned} \vert c_0\vert \ge 0.31 \Vert f\Vert . \end{aligned}$$

For the proof of Proposition 4.4, we need some preparatory lemmas. Recall from (2) that $1/2< \delta < 1$. We will occasionally use this estimate below without mention.

Lemma 4.5

Assumptions and notations being as in Proposition 4.4, we have the relation

$$\begin{aligned} c_i = \sum _{j=1}^\infty a_{i,j}(\delta ,w) c_j \end{aligned}$$

(80)

for every positive integer i, where $a_{i,j}(\delta , w)$ is given by Proposition 4.1.

Proof

For every 1-eigenfunction f, we have

$$\begin{aligned} \sum _{i=0}^\infty c_i z^i = f(z) = {\mathscr {L}}_{\delta ,w} f(z) = \sum _{i=0}^\infty \sum _{j=0}^\infty a_{i,j}(\delta ,w) c_j z^i. \end{aligned}$$

(81)

by Proposition 4.1. Comparing the coefficients in (81) yields the relation in (80). $\square$

Also helpful is the following

Lemma 4.6

For all $0< x < 1$ and all positive integers i, we have

$$\begin{aligned} S_{i}^{\mathrm {even}}(x) \mathrel {\mathop :}=\sum _{\begin{array}{c} j=0 \\ j\text { even} \end{array}}^{\infty } (j+1) { i+j+1 \atopwithdelims ()i} x^{j} < \frac{i+1}{(1-x)^{i+2}} \end{aligned}$$

and

$$\begin{aligned} S_{i}^{\mathrm {odd}}(x) \mathrel {\mathop :}=\sum _{\begin{array}{c} j=1 \\ j \text { odd} \end{array}}^{\infty } (j+1) { i+j+1 \atopwithdelims ()i} x^{j} < \frac{i+1}{2(1-x)^{i+2}}. \end{aligned}$$

Proof

It is an exercise to check that

$$\begin{aligned} S_{i}(x) \mathrel {\mathop :}=\sum _{j=0}^{\infty } (j+1) { i+j+1 \atopwithdelims ()i} x^{j} = \frac{i+1}{(1-x)^{i+2}} \end{aligned}$$

for all $\vert x\vert < 1.$ The result then follows from

$$\begin{aligned} S_{i}^{\mathrm {even}}(x) = \frac{1}{2}\left( S_{i}(x) + S_{i}(-x) \right) , \quad S_{i}^{\mathrm {odd}}(x) = \frac{1}{2}\left( S_{i}(x) - S_{i}(-x) \right) \end{aligned}$$

and from the fact that

$$\begin{aligned} S_{i}(x)> S_{i}(-x) > 0, \end{aligned}$$

provided $x>0$. $\square$

The next result will be needed to show that 1-eigenfunctions of ${\mathscr {L}}_{\delta ,w}$ are even functions.

Lemma 4.7

Assume that $w\ge 3$. Fix a 1-eigenfunction f with Taylor expansion as in (79). We call a pair of positive numbers $(\alpha , \eta )$ ‘good’ (for the 1-eigenfunction f) if the bound

$$\begin{aligned} \vert c_{i}\vert < \alpha (i+1) \left( \frac{\eta }{w} \right) ^{i+1}. \end{aligned}$$

(82)

is satisfied for all odd indices $i\ge 1$. Then, the following holds: if $(\alpha , \eta )$ is a good pair with $\eta w^{-2} < 1$, then

$$\begin{aligned} \left( \frac{\alpha \eta }{2(1-\eta w^{-2})}, \frac{1}{1-\eta w^{-2}} \right) \end{aligned}$$

is also a good pair.

Proof

It follows directly from the expression in (63) that $a_{i,j}(\delta ,w) = 0$ whenever i and j have different parity, and that

$$\begin{aligned} \vert a_{i,j}(\delta ,w)\vert \le \frac{2 \zeta (2\delta +i+ j)}{w^{i+j+2\delta }} { 2\delta +i+j-1 \atopwithdelims ()i} \le \frac{2 \zeta (3)}{w^{i+j+2\delta }} { i+j+1 \atopwithdelims ()i} \end{aligned}$$

(83)

when i and $j \ge 1$ have the same parity.

Assume that $i\ge 1$ is an odd integer. Then, using Lemma 4.5 and the estimate in (83), we obtain

$$\begin{aligned} \vert c_{i}\vert&\le \frac{2 \zeta (3)}{w^{i + 2 \delta }} \sum _{\begin{array}{c} j=1 \\ j\text { odd } \end{array}}^{\infty } \vert c_j\vert \frac{1}{w^{j}} { i+j+1 \atopwithdelims ()i} \end{aligned}$$

Now assume that $(\alpha , \eta )\in {\mathbb {R}}_{>0}^2$ is a good pair with $\eta w^{-2} < 1$. Inserting (82) into the previous line and rearranging then gives

$$\begin{aligned} \vert c_{i}\vert&<\frac{2\alpha \zeta (3)}{w^{i+2\delta }} \sum _{\begin{array}{c} j=1 \\ j\text { odd } \end{array}}^{\infty } (j+1)\left( \frac{\eta }{w} \right) ^{j+1} \frac{1}{w^{j}} { i+j+1 \atopwithdelims ()i}\\&= \frac{2\alpha \eta \zeta (3) }{w^{i+2\delta +1}} \sum _{\begin{array}{c} j=1 \\ j\text { odd } \end{array}}^{\infty } (j+1) \left( \eta w^{-2} \right) ^{j} { i+j +1 \atopwithdelims ()i}\\&= \frac{2\alpha \eta \zeta (3)}{w^{i+2\delta +1}} S^{\mathrm {odd}}_i\left( \eta w^{-2} \right) \\&< \frac{\alpha \eta \zeta (3) }{w^{i+2\delta +1}} (i+1) \left( \frac{1}{1-\eta w^{-2}}\right) ^{i+2}, \end{aligned}$$

where in the last line we have used Lemma 4.6. Hence, we have shown

$$\begin{aligned} \vert c_{i}\vert < {\widetilde{\alpha }} (i+1) \left( \frac{{\widetilde{\eta }}}{w} \right) ^{i+1}, \end{aligned}$$

where

$$\begin{aligned} {\widetilde{\alpha }} = \frac{\alpha \eta \zeta (3)}{ w^{2\delta } (1-\eta w^{-2})} \quad \text {and}\quad {\widetilde{\eta }} = \frac{1}{1-\eta w^{-2}}. \end{aligned}$$

(84)

Recall that $w\ge 3$ by assumption and that $\delta > \frac{1}{2}.$ Together with the crude bound $\zeta (3) <\frac{3}{2}$, we get

$$\begin{aligned} \frac{\zeta (3)}{w^{2\delta }}< \frac{\zeta (3)}{3} < \frac{1}{2}, \end{aligned}$$

which when inserted into (84) gives

$$\begin{aligned} {\widetilde{\alpha }} < \frac{ \alpha \eta }{2(1-\eta w^{-2})}, \end{aligned}$$

completing the proof of Lemma 4.7. $\square$

Proof of Proposition 4.4

We may assume without loss of generality that the 1-eigenfunction f is normalized so that $\Vert f\Vert = 1$. Recall from (67) that we have the a-priori bound on coefficients

$$\begin{aligned} \vert c_j\vert \le \frac{\sqrt{j+1}}{\sqrt{\pi }} \le \frac{j+1}{\sqrt{\pi }} \quad \text {for all} \quad j\ge 0. \end{aligned}$$

(85)

Let us first prove Part (ii) and assume that $i\ge 2$ is even. Note that in this case we have $a_{i,j}(\delta , w) = 0$ whenever j is odd. Thus, we can use Lemma 4.5 together with the bounds in (85) and in (83), to estimate

$$\begin{aligned} \vert c_{i}\vert&\le \sum _{\begin{array}{c} j=0 \\ j\text { even} \end{array}}^\infty \vert a_{i,j}(\delta ,w) \vert \vert c_j \vert \\&\le \frac{2 \zeta (3) }{w^{i+2\delta } \sqrt{\pi }} \sum _{\begin{array}{c} j=0 \\ j\text { even} \end{array}}^{\infty } (j+1) { i+j+ 1 \atopwithdelims ()i} \frac{1}{w^j}\\&= \frac{2 \zeta (3)}{w^{i+2\delta } \sqrt{\pi }} S^{\mathrm {even}}_i(w^{-1}). \end{aligned}$$

Recalling that $w \ge 3$ we can use Lemma 4.6 to obtain furthermore

$$\begin{aligned} \vert c_{i}\vert \le \frac{2 \zeta (3) }{w^{i+1} \sqrt{\pi }} (i+1)\left( \frac{3}{2}\right) ^{i+2} = \frac{3 \zeta (3) }{\sqrt{\pi }} (i+1)\left( \frac{3}{2w}\right) ^{i+1}, \end{aligned}$$

(86)

which completes the proof of (ii).

Let us now prove Part (i) and address the case when $i\ge 1$ is an odd integer. By repeating the same steps as above, we obtain an estimate of the type

$$\begin{aligned} \vert c_{i}\vert < \alpha (i+1)\left( \frac{3}{2w} \right) ^{i+1} \end{aligned}$$

for all odd $i\ge 1,$ where $\alpha > 0$ is some absolute constant. In the language of Lemma 4.7 this means that the pair $(\alpha _0, \eta _0):= \left( \alpha , \frac{3}{2}\right)$ is good. By iterating Lemma 4.7, we obtain a sequence of good pairs $(\alpha _\ell , \eta _\ell )$ recursively defined by

$$\begin{aligned} \alpha _{\ell } = \alpha _{\ell - 1} \frac{\eta _{\ell - 1}}{2\left( 1-\eta _{\ell - 1} w^{-2} \right) } \quad \text {and}\quad \eta _{\ell } = \frac{1}{1-\eta _{\ell - 1} w^{-2}}. \end{aligned}$$

One can check that the sequence $\eta _\ell$ is decreasing as $\ell \rightarrow \infty$, so $\eta _\ell \le \eta _0 = \frac{3}{2}.$ Moreover, since

$$\begin{aligned} x \mapsto \frac{x}{2\left( 1-x w^{-2} \right) } \end{aligned}$$

is an increasing function, we get

$$\begin{aligned} \alpha _{\ell } = \alpha _{\ell - 1} \frac{ \eta _{\ell - 1}}{2\left( 1-\eta _{\ell - 1} w^{-2} \right) } \le \alpha _{\ell - 1} \frac{ \eta _{0}}{2\left( 1-\eta _{0} w^{-2} \right) } \le 0.9 \alpha _{\ell - 1}, \end{aligned}$$

where for the last inequality we used the assumption that $w\ge 3$. This implies that

$$\begin{aligned} \alpha _{\ell } \le 0.9^{\ell }\alpha \rightarrow 0 \quad (\ell \rightarrow \infty ), \end{aligned}$$

which in turn implies that for all odd $i\ge 1$ we have

$$\begin{aligned} \vert c_i\vert < \alpha _\ell (i+1) \left( \frac{\eta _\ell }{w}\right) ^{i+1} \le \alpha _\ell (i+1) \left( \frac{\eta _0}{w}\right) ^{i+1} \rightarrow 0 \quad (\ell \rightarrow \infty ). \end{aligned}$$

But this forces $c_i = 0$ for all odd $i \ge 1$, completing the proof of Part (i).

To prove Part (iii) recall from (66) that the norm of f can be expressed in term of its Taylor coefficients as

$$\begin{aligned} \Vert f\Vert ^{2} = \pi \sum _{j=0}^{\infty } \frac{\vert c_j\vert ^2}{j+1}. \end{aligned}$$

(87)

Since $c_j = 0$ for all odd j, we can restrict this sum to the even terms and isolate the 0-th term, writing

$$\begin{aligned} \Vert f\Vert ^{2} = \pi \vert c_0\vert ^2 + \pi \sum _{l=1}^{\infty } \frac{\vert c_{2l}\vert ^2}{2l+1}. \end{aligned}$$

By assumption, we have $\Vert f\Vert = 1$ and $w\ge 3$, so

$$\begin{aligned} 1&\le \pi \vert c_0\vert ^2 + 9 \zeta (3)^{2} \sum _{l=1}^{\infty } (2l+1) \left( \frac{3}{2w}\right) ^{4l+2}\\&\le \pi \vert c_0\vert ^2 + 9 \zeta (3)^{2} \sum _{l=1}^{\infty } (2l+1) \left( \frac{1}{2}\right) ^{4l+2}\\&\le \pi \vert c_0\vert ^2 + 0.68. \end{aligned}$$

Rearranging this inequality, we obtain

$$\begin{aligned} \vert c_0\vert \ge \sqrt{\frac{0.32}{\pi }} > 0.31, \end{aligned}$$

completing the proof of Proposition 4.4. $\square$

4.3 Finishing the Proof of Theorem 1.9

We can now prove Theorem 1.9. Let $f \in H^2({\mathbb {D}})$ be a nonzero 1-eigenfunction of ${\mathscr {L}}_{\delta ,w}$ with Taylor expansion

$$\begin{aligned} f(z) = c_0 + c_1 z + c_2 z^2 + \cdots . \end{aligned}$$

We may assume without loss of generality that f is normalized so that $\Vert f\Vert = 1.$ Applying Lemma 4.5 with $i=0$ gives

$$\begin{aligned} c_{0} = \sum _{j=0}^{\infty } \alpha _{0,j}(\delta , w) c_j = \sum _{j=0}^{\infty } c_j \left( (-1)^j + 1 \right) \frac{1}{w^{2\delta +j}} \zeta (2\delta +j). \end{aligned}$$

Notice that we can restrict this sum to even terms $j=2l$ and isolate the term $l=0$ to write

$$\begin{aligned} c_{0} = 2 \zeta (2\delta ) \frac{1}{w^{2\delta }} c_{0} + 2 \sum _{l=1}^{\infty } c_{2l} \frac{1}{w^{2l+2\delta }} \zeta (2l+2\delta ). \end{aligned}$$

(88)

We are interested in the behavior of $\delta = \delta (w)$ as $w\rightarrow \infty$, so we may assume that $w\ge 3$. Then, by Part (iii) of Proposition 4.4, we have

$$\begin{aligned} c_0 \ne 0. \end{aligned}$$

Thus, we can divide both sides of (88) by $c_0$ to obtain

$$\begin{aligned} 1 = 2 \zeta (2\delta ) \frac{1}{w^{2\delta }} + E(w), \end{aligned}$$

(89)

where we have put

$$\begin{aligned} E(w) = 2c_{0}^{-1} \sum _{l=1}^{\infty } c_{2l} \frac{1}{w^{2l+2\delta }} \zeta (2l+2\delta ). \end{aligned}$$

Invoking the estimates in Parts (ii) and (iii) of Proposition 4.4, and recalling that $\delta > \frac{1}{2}$, we get the bound

$$\begin{aligned} \vert E( w)\vert \le 2 \vert c_{0}\vert ^{-1} \sum _{l=1}^{\infty } \vert c_{2l}\vert \frac{1}{w^{2l+2\delta }} \zeta (2l+2\delta )= O\left( \sum _{l=1}^{\infty } \frac{1}{w^{4l+2\delta +1}}\right) = O\left( \frac{1}{w^6} \right) , \nonumber \\ \end{aligned}$$

(90)

where the implied constant in the error term does not depend on w. Thus, returning to (89), we have

$$\begin{aligned} 1 = 2 \zeta (2\delta ) \frac{1}{w^{2\delta }} + O\left( \frac{1}{w^6} \right) . \end{aligned}$$

(91)

The final step towards the proof of Theorem 1.9 is to ‘solve’ this equation for the unknown variable $\delta$. On introducing a new variable $x>0$ and making the substitution

$$\begin{aligned} \delta = \frac{1+x}{2}, \end{aligned}$$

we can rewrite (91) as

$$\begin{aligned} 1 = 2 \zeta (1 + x) \frac{1}{w^{1+x}} + O\left( \frac{1}{w^6} \right) . \end{aligned}$$

(92)

Recalling from (10) the well-known Laurent expansion of the Riemann zeta function $\zeta (s)$ at $s = 1,$ we write

$$\begin{aligned} \zeta (1+x) = \frac{1}{x} + \sum _{n=0}^\infty \frac{(-1)^n \gamma _n}{n!} x^{n}, \end{aligned}$$

(93)

where $\gamma _n$ is the n-th Stieltjes constant. Notice also that

$$\begin{aligned} \frac{1}{w^{1+x}} = \frac{e^{-x t}}{w} = \frac{1}{w} \sum _{n=0}^{\infty } \frac{(-t)^{n}}{n!}x^{n}, \end{aligned}$$

(94)

where we have set

$$\begin{aligned} t=\log w \end{aligned}$$

for notational convenience. Using the Cauchy product formula, we can multiply the series expansions in (93) and (94) to obtain an expression of the form

$$\begin{aligned} 2 \zeta (1 + x) \frac{1}{w^{1+x}} = \frac{2}{xw} + \frac{2}{w}\sum _{n=0}^\infty Q_{n+1}(t) x^n, \end{aligned}$$

(95)

where each $Q_n$ is a polynomial of degree n whose coefficients can be computed in terms of the Stieltjes constants. Notice in particular that

$$\begin{aligned} Q_1(t) = - t + \gamma _0. \end{aligned}$$

We truncate the series on the right of (95) at $n=4$ to write

$$\begin{aligned} 2 \zeta (1 + x) \frac{1}{w^{1+x}}= & {} \frac{2}{xw} + Q_1(t) \frac{2}{w} + Q_2(t) \frac{2x}{w}+ Q_3(t) \frac{2x^2}{w} + Q_4(t) \frac{2x^3}{w} \nonumber \\&+ Q_4(t) \frac{2x^4}{w} + O \left( \frac{\log (w)^5}{w^5} \right) . \end{aligned}$$

(96)

At first sight, the truncation at $n=4$ may seem arbitrary but when comparing the right hand side of (96) with the right hand of (92), we obtain

$$\begin{aligned} 1= & {} \frac{2}{xw} + Q_1(t) \frac{2}{w} + Q_2(t) \frac{2x}{w}+ Q_3(t) \frac{2x^2}{w} + Q_4(t) \frac{2x^3}{w} \nonumber \\&+ Q_5(t) \frac{2x^4}{w} + O\left( \frac{\log (w)^5}{w^5}\right) . \end{aligned}$$

(97)

Notice that the term $O( \frac{1}{w^6})$ in (92) gets absorbed by the term $O( \frac{(\log w)^5}{w^5})$ on the right hand side of (96). Multiplying both sides of (97) by x yields

$$\begin{aligned} x= & {} \frac{2}{w} + Q_1(t) \frac{2x}{w} + Q_2(t) \frac{2x^2}{w}+ Q_3(t) \frac{2x^3}{w} + Q_4(t) \frac{2x^4}{w} \nonumber \\&+ Q_5(t) \frac{2x^5}{w} + O\left( x\frac{\log (w)^5}{w^5}\right) . \end{aligned}$$

(98)

Recall from (2) that $\delta = \delta (w) \rightarrow \frac{1}{2}^+$ which implies that $x\rightarrow 0^+$. Thus, (98) immediately implies the a-priori bound

$$\begin{aligned} x = O\left( \frac{1}{w} \right) \end{aligned}$$

as $w\rightarrow \infty$. Inserting this bound into the error term in (98) gives

$$\begin{aligned} x = \frac{2}{w} + Q_1(t) \frac{2x}{w} + Q_2(t) \frac{2x^2}{w}+ Q_3(t) \frac{2x^3}{w} + Q_4(t) \frac{2x^4}{w} + O\left( \frac{\log (w)^5}{w^6}\right) .\nonumber \\ \end{aligned}$$

(99)

We can now repeatedly substitute every occurrence of x on the right hand side of this expression by the expression itself, leading to an expression of the form

$$\begin{aligned} x = \frac{2}{w} + P_1(t) \frac{2}{w^2} + P_2(t) \frac{2}{w^3}+ P_3(t) \frac{2}{w^4} + P_4(t) \frac{2}{w^5} + O\left( \frac{\log (w)^5}{w^6}\right) ,\nonumber \\ \end{aligned}$$

(100)

where $P_1, P_2, P_3, P_4$ can be determined explicitly from the polynomials $Q_1, Q_2, Q_3, Q_4$ (this procedure, if carried out in detail, shows that each $P_i$ is a polynomial of degree at most i and that its coefficients can be computed in terms of the Stieltjes constants). In particular, this procedure yields (after the first substitution)

$$\begin{aligned} P_1(t) = 2 Q_1(t) = -2 t + 2\gamma _0. \end{aligned}$$

By re-substituting the variables, we obtain

$$\begin{aligned} \delta= & {} \frac{1+x}{2} \\ {}= & {} \frac{1}{2} + \frac{1}{w} + P_1(t) \frac{1}{w^2} + P_2(t) \frac{1}{w^3}+ P_3(t) \frac{1}{w^3} + P_4(t) \frac{1}{w^5} + O\left( \frac{\log (w)^5}{w^6}\right) , \end{aligned}$$

as $w\rightarrow \infty$, completing the proof of Theorem 1.9.

4.4 Sharp Numerical Estimates

In this subsection, we show how to obtain numerical estimates for $\delta (w)$. The case $w = 3$ will be of special interest (due to the question posed by Jakobson–Naud in [19]), but we will initially work with arbitrary $w\ge 3$ and write $\delta =\delta (w)$.

We fix a nonzero 1-eigenfunction $f(z) = c_0 + c_1 z + \cdots$ a of ${\mathscr {L}}_{\delta , w}$ and we assume that f is normalized so that $\Vert f\Vert = 1.$ Specializing Lemma 4.5 to $i=0$ gives

$$\begin{aligned} c_{0} = 2\sum _{l=0}^{\infty } c_{2l} \frac{1}{w^{2l+2\delta }} \zeta (2l+2\delta ) = 2c_{0} \frac{1}{w^{2\delta }} \zeta (2\delta ) + 2c_{2} \frac{1}{w^{2+2\delta }} \zeta (2+2\delta ) + {\mathscr {E}}_1(w),\nonumber \\ \end{aligned}$$

(101)

where

$$\begin{aligned} {\mathscr {E}}_1(w) \mathrel {\mathop :}=2 \sum _{l=2}^{\infty } c_{2l} \frac{1}{w^{2l+2\delta }}\zeta (2l+2\delta ). \end{aligned}$$

Similarly, specializing Lemma 4.5 to $i=2$ yields

$$\begin{aligned} c_{2} = 2 c_{0} \frac{1}{w^{2\delta +2}} { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta ) + 2 c_{2} \frac{1}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )+ {\mathscr {E}}_2(w),\nonumber \\ \end{aligned}$$

(102)

where

$$\begin{aligned} {\mathscr {E}}_2(w)\mathrel {\mathop :}=2 \frac{1}{w^{2}} \sum _{l=2}^{\infty } c_{2l} \frac{1}{w^{2l+2\delta }} { 2l + 2\delta + 1 \atopwithdelims ()2} \zeta (2+2l+2\delta ). \end{aligned}$$

Solving (102) for $c_2$ yields

$$\begin{aligned} c_2 = c_{0} \frac{ \frac{2}{w^{2+2\delta }} { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta )}{1- \frac{2}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} + \frac{{\mathscr {E}}_2(w)}{1- \frac{2}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )}. \end{aligned}$$

(103)

Inserting (103) into (101) gives

$$\begin{aligned} c_0 = c_{0} \left( \frac{2}{w^{2\delta }} \zeta (2\delta ) + \frac{ \frac{4}{w^{4+4\delta }} { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta )^2}{1- \frac{2}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} \right) + {\mathscr {E}}(w) \end{aligned}$$

(104)

with

$$\begin{aligned} {\mathscr {E}}(w) \mathrel {\mathop :}={\mathscr {E}}_1(w) + \frac{\frac{2}{w^{2+2\delta }} \zeta (2+2\delta )}{1- \frac{2}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} {\mathscr {E}}_2(w). \end{aligned}$$

(105)

Recall from Proposition 4.4 that $c_0\ne 0$. Hence, we can divide both sides of (104) by $c_0$ to obtain

$$\begin{aligned} \left| 1- \frac{2}{w^{2\delta }} \zeta (2\delta ) - \frac{\frac{4}{w^{4+4\delta }} { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta )^2}{1- \frac{2}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} \right| = E(w), \end{aligned}$$

(106)

where

$$\begin{aligned} E(w)\mathrel {\mathop :}=\frac{\vert {\mathscr {E}}(w)\vert }{\vert c_0\vert }. \end{aligned}$$

The subsequent goal is to estimate the error E(w). By Part (iii) of Proposition 4.4 we have

$$\begin{aligned} E(w) \le \frac{1}{0.31}\vert {\mathscr {E}}(w)\vert , \end{aligned}$$

(107)

so we have to estimate $\vert {\mathscr {E}}(w)\vert$. Recall that we have the coefficient bound

$$\begin{aligned} \vert c_i\vert < \frac{3 \zeta (3) }{\sqrt{\pi }} (i+1)\left( \frac{3}{2w}\right) ^{i+1} \end{aligned}$$

from Proposition 4.4. Thus

$$\begin{aligned} \vert {\mathscr {E}}_1(w)\vert&\le 2 \sum _{l=2}^{\infty } \vert c_{2l}\vert \frac{1}{w^{2l+2\delta }} \zeta (2l+2\delta )\nonumber \\&< \frac{9 \zeta (3) \zeta (4+2\delta )}{w \sqrt{\pi }} \sum _{l=2}^{\infty } (2l+1)\left( \frac{3}{2w}\right) ^{2l} \frac{1}{w^{2l+2\delta }} \nonumber \\&= \frac{9 \zeta (3) \zeta (4+2\delta ) }{w^{1+2\delta } \sqrt{\pi }} \sum _{l=2}^{\infty } (2l+1)\left( \frac{3}{2w^2}\right) ^{2l}\nonumber \\&= \frac{9\zeta (3) \zeta (4+2\delta ) }{w^{1+2\delta } \sqrt{\pi }} \cdot \frac{(\frac{3}{2w^2})^{4} \left( 5-3 (\frac{3}{2w^2})^{2} \right) }{\left( 1- (\frac{3}{2w^2})^{2} \right) ^2}\nonumber \\&= \frac{729 \, \zeta (3) \zeta (4+2\delta ) \left( 5-3 (\frac{3}{2w^2})^{2} \right) }{16\, w^{9+2\delta } \sqrt{\pi } \left( 1- (\frac{3}{2w^2})^{2} \right) ^2 }. \end{aligned}$$

(108)

In the second last line, we have used the elementary identity

$$\begin{aligned} \sum _{l=2}^{\infty } (2l+1)x^{2l} = \frac{d}{dx} \sum _{l=2}^{\infty } x^{2l+1} = \frac{d}{dx} \left( \frac{x^5}{1-x^2} \right) = \frac{x^4 (5-3x^2)}{(1-x^2)^2} \end{aligned}$$

(109)

for $x = \frac{3}{2w^2}.$ Similarly, we have

$$\begin{aligned} \vert {\mathscr {E}}_2(w)\vert&\le 2 \frac{1}{w^{2}} \sum _{l=2}^{\infty } \vert c_{2l}\vert \frac{1}{w^{2l+2\delta }} { 2l + 2\delta + 1 \atopwithdelims ()2} \zeta (2+2l+2\delta )\\&< \frac{9 \zeta (3) \zeta (6+2\delta ) }{ w^{3+2\delta } \sqrt{\pi }} \sum _{l=2}^{\infty } (2l+1)\left( \frac{3 }{2w^2}\right) ^{2l} { 2l + 2\delta + 1 \atopwithdelims ()2} \\&< \frac{9 \zeta (3) \zeta (7) }{ w^{3+2\delta } \sqrt{\pi }} \sum _{l=2}^{\infty } (2l+1)\left( \frac{3}{2w^2}\right) ^{2l} { 2l + 3 \atopwithdelims ()2}. \end{aligned}$$

To estimate the remaining sum in the last line, we can use the identity

$$\begin{aligned} \sum _{l=2}^{\infty } (2l+1)x^{2l} { 2l + 3 \atopwithdelims ()2} = \frac{1}{2} \frac{d^3}{d x^3} \left( \frac{x^7}{1-x^2} \right) = 3x^{4} \cdot \frac{35-56x^2+39x^4-10x^6}{(1-x^{2})^{4}}. \end{aligned}$$

One can then check that

$$\begin{aligned} \sum _{l=2}^{\infty } (2l+1)x^{2l} { 2l + 3 \atopwithdelims ()2} < 113 x^{4} \quad \text {for all}\quad x\le 1/6. \end{aligned}$$

Inserting this bound above, we obtain for all $w\ge 3$ the somewhat simpler estimate

$$\begin{aligned} \vert {\mathscr {E}}_2(w)\vert< \frac{9 \zeta (3) \zeta (7) }{ w^{3+2\delta } \sqrt{\pi }}\cdot 113 \left( \frac{3 }{2w^2}\right) ^{4} < \frac{3521}{w^{11+2\delta }}. \end{aligned}$$

(110)

Going back to (105) and gathering the estimates in (107), (108), (110), we obtain the following final bound for the error:

$$\begin{aligned} \vert E(w)\vert< & {} \frac{1}{0.31}\left( \frac{729 \, \zeta (3) \zeta (4+2\delta ) \left( 5-3 (\frac{3}{2w^2})^{2} \right) }{16\, w^{9+2\delta } \sqrt{\pi } \left( 1- (\frac{3}{2w^2})^{2} \right) ^2 }\right. \nonumber \\&\left. + \frac{7042\, \zeta (2+2\delta )}{w^{13+4\delta }\left( 1- 2 \frac{1}{w^{4+2\delta }} { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta ) \right) } \right) . \end{aligned}$$

(111)

Let us now specialize to the case $w = 3.$ To estimate the error term, we may use the already established numerical estimates by Phillips–Sarnak in [35]. We will simply use the (weaker) lower bound $\delta = \delta (3) > 0.7$. Note that the right hand side of (111) is decreasing as a function of $\delta$, so we can insert these values to obtain

$$\begin{aligned} \vert E(3)\vert < 0.0066. \end{aligned}$$

Thus, going back to (106), we deduce that $\delta = \delta (3)$ must satisfy

$$\begin{aligned} \left| 1- 2 (1/3)^{2\delta } \zeta (2\delta ) - \frac{ 4 (1/3)^{4+4\delta } { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta )^2}{1- 2 (1/3)^{4+2\delta } { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} \right| < \varepsilon \mathrel {\mathop :}=0.0066.\nonumber \\ \end{aligned}$$

(112)

The function

$$\begin{aligned} F:\left( \frac{1}{2}, \infty \right) \rightarrow {\mathbb {R}}, \quad F(\delta )= & {} 1- 2 (1/3)^{2\delta } \zeta (2\delta )\\&\quad - \frac{ 4 (1/3)^{4+4\delta } { 2\delta + 1 \atopwithdelims ()2} \zeta (2+2\delta )^2}{1- 2 (1/3)^{4+2\delta } { 2\delta + 3 \atopwithdelims ()2} \zeta (4+2\delta )} \end{aligned}$$

is strictly increasing, so (112) forces $\delta (3)$ to lie in the range

$$\begin{aligned} \delta ^{-}< \delta (3) < \delta ^{+}, \end{aligned}$$

where $\delta ^{\pm } \in (\frac{1}{2}, \infty )$ are the unique solutions of

$$\begin{aligned} F( \delta ^{\pm } ) = \pm \varepsilon . \end{aligned}$$

We can now check that

$$\begin{aligned} F( 0.75065 ) < -\varepsilon \quad \text {and} \quad F( 0.75322 ) > \varepsilon , \end{aligned}$$

showing that

$$\begin{aligned} 0.75065< \delta (3) < 0.75322. \end{aligned}$$