Proper holomorphic mappings of balanced domains in \(\mathbb {C}^n\)

The central result of this paper is:

We recall that a domain \(\Omega \) is said to be balanced if whenever \(z \in \Omega \) and \(\lambda \in \mathbb {C}, |\lambda | \le 1\), then we have \(\lambda z \in \Omega \). The above theorem is motivated by the famous result of Alexander [2] showing the non-existence of non-injective proper holomorphic self-maps of the unit ball in \(\mathbb {C}^n, \ n >1\)—which the above theorem generalizes. Since its appearance, Alexander’s theorem has been extended in many different ways. We refer the interested reader to Sect. 3 of the survey article by Forstnerič [18]. One such result is that of Bedford and Bell [3], which extends Alexander’s theorem to bounded pseudoconvex domains having real-analytic boundaries. Recent work relating to Alexander’s theorem has, therefore, focused on domains whose boundaries need not be real-analytic. See, for instance, the work of Berteloot [8] and Coupet et al. [14, 15].

An alternative strategy for deriving a conclusion similar to that of Alexander’s theorem, especially when a domain does not have real-analytic boundary, is to impose a condition on its automorphism group. With a hypothesis on the automorphism group, there are often interesting interactions between the symmetries of the domain and the boundary geometry. These interactions allow one to simplify the structure of the branch locus of a branched proper holomorphic self-map.

However, there is a new ingredient to our proof, which we briefly introduce here. In a relatively recent paper, Opshtein proved the following result:

Refer to Sect. 3 for the definitions of the terms that occur in the above result. In [22], Opshtein suggests that his results could serve as a new set of tools for establishing Alexander-type theorems. We found Opshtein’s viewpoint very useful in the context of Theorem 1.1: Result 1.2 plays a key role in our proof of Theorem 1.1 (which we shall presently explain).

Another influence behind this work is the following result:

Let \(\Omega \) and \(F\) be as in Theorem 1.1. Our proof of Theorem 1.1 and the proof of Result 1.3 share some key ideas. As \(\Omega \) is simply connected, to prove that \(F\) is an automorphism it suffices—in view of the monodromy theorem—to establish that \(F\) is unbranched. Thus, a natural approach to proving Theorem 1.1 is to assume that \(F\) is branched, and use this assumption together with the properties of \(\Omega \) to reach a contradiction. To this end, we prove the following result that establishes an important property of the branch locus of \(F\).

We ought to point out that Proposition 1.5 is a hypothetical statement. If \(F\) as in Theorem 1.1 were branched, then it would have the above structure. The thrust of our proof is that \(F\) can never be branched.

Coupet, Pan and Sukhov have proved a version of Proposition 1.5 for domains in \(\mathbb {C}^2\) in which the domain need not be balanced, but is only required to admit a transverse \(\mathbb {T}\)-action. The point that is worth highlighting here is that by restricting ourselves to balanced domains, we are able to give an almost entirely elementary proof of Proposition 1.5, and that these methods have one significant payoff: we do not have to assume in the above results that the \(\mathbb {T}\)-action is transverse. We bypass the need for transversality by using some results from dimension theory. These results are insensitive to dimension, which allows us to state and prove Theorem 1.1 in \(\mathbb {C}^n\) for all \(n > 1\).

A key object used in the proofs of Proposition 1.5 and Theorem 1.1 is the function \(\tau : \partial \Omega \rightarrow \mathbb {Z}_+ \cup \{ 0 \}\) introduced by Bedford and Bell (see [3, 7]). The number \(\tau (p)\) is the order of vanishing in the tangential directions of the Levi determinant of a smoothly bounded pseudoconvex domain at the point \(p \in \partial \Omega \). The function \(\tau \) has been used to study the branching behaviour of proper holomorphic mappings in many earlier results. It turns out that, owing to the hypothesis that \(\Omega \) is of (D’Angelo) finite type, \(\tau \) is bounded on \(\partial \Omega \). We refer the reader to Sect. 2 for a more precise discussion.

Before, we proceed further, we clarify that, in this paper, whenever we use use the word “smooth”, it will refer to \(\mathcal {C}^{\infty }\)-smoothness unless specified otherwise. By a smoothly bounded domain, we shall mean a bounded domain whose boundary is \(\mathcal {C}^\infty \)-smooth.

In this section, we shall summarize the properties of the function \(\tau \) alluded to above. For the sake of completeness, we first give the definition of D’Angelo finite type.

It is simple to prove that the above definition is independent of the choice of the defining function \(r\). Whenever we say below that a bounded domain is of finite type, we shall mean that each boundary point is of finite type in the sense of the above definition.

We now define a function \(\tau \). This function was introduced by Bedford and Bell [3], and has been used in many results to examine the branching behaviour of proper holomorphic mappings. It is used multiple times in our proof of Theorem 1.1

Observe that \(\tau \) is an upper semi-continuous function on \(\partial D\).

The next two results give an idea why the \(\tau \) function is relevant to our main theorem.

Let \(D \subset \mathbb {C}^n\) be a smoothly bounded pseudoconvex domain of finite type and let \(p\in \partial D\). It can be argued that \(\tau (p)\) is finite. As the D’Angelo 1-type of \(p\) is finite, if we pick a non-zero vector \(V\in T_p(\partial D)\cap iT_p(\partial D)\), then by definition there exists a \(k\in \mathbb {Z}_+\) such that the homogeneous polynomial of degree \(k\) in the Taylor expansion of \(\zeta \mapsto r(p+\zeta V)\) around \(0\in \mathbb {C}\) is not identically zero (\(r\) here is a defining function of \(D\)). However, as this \(k\) depends on the choice of \(V\in T_p(\partial D)\cap iT_p(\partial D)\), it becomes quite technical to produce a single finite-order tangential differential operator \(T\) at \(p\) such that \(T\Lambda _r(p)\ne 0\). We could not find an elementary proof of Result 2.6 (see below) in the literature, although it has been made use of a number of times; see, for instance, Pan [23]. A recent work of Nicoara [21, Main Theorem 1.1] provides an effective upper bound for \(\tau \) in terms of the D’Angelo 1-type. For our purposes, the following consequence of Nicoara’s result suffices:

The next result is by Coupet et al. [15]. Since the statement of the result below is slightly different from that of [15, Lemma 1], and its proof is sufficiently important for our purposes, we provide a proof of it. Here, and elsewhere in this paper, \(F^\nu \) will denote the \(\nu \)th iterate of \(F\).

The extendability of \(F\) as stated in the previous lemma—and, indeed, a stronger conclusion—is guaranteed when \(\Omega \) is as in Theorem 1.1. We shall need this stronger conclusion, presented in the next lemma, in our proof. This lemma is an easy consequence of [6, Theorem 2]; see [9, Lemma 4.2] for a proof.

In this section we summarize some material from the theory of (iterative) dynamics of holomorphic self-maps of a taut manifold. Given complex manifolds \(X\) and \(Y\), \(\mathrm{Hol}(X,Y)\) will denote the space of holomorphic mappings from \(X\) to \(Y\), where the topology on \(\mathrm{Hol}(X,Y)\) is the compact-open topology. We are interested in the set \(\Gamma (f)\) which is defined to be the set of all limit points of the iterates of a holomorphic mapping \(f \in \mathrm{Hol}(X,X)\), where \(X\) is a taut complex manifold. Of course, \(\Gamma (f)\) might be empty. The following result describes the possible behaviours of the iterates.

The behaviour of the iterates of a holomorphic self-map of a taut manifold \(X\) depends on whether \(f\) has a fixed point or not. The following theorem gives a quantitative description of the behaviour of the complex derivative \(f'\) at a fixed point of \(f\); see [1, Theorem 2.1.21] for a proof.

The space \(L_U\) is called the unitary space of \(f\) at \(z_0\).

The next result gives quite precise information about the set \(\Gamma (f)\); see [1, Corollary 2.4.4] for a proof.

We point out that Result 3.1 guarantees that \(f|_M\) is an automorphism of \(M\).

To conclude this section, we reiterate that one of the main results of [22] plays a central role in the final step in our proof of Theorem 1.1. This is Result 1.2 above. All the terms occurring in that result have now been defined in this section.

We begin with some material on complex geodesics. In what follows, given a domain \(D \subset \mathbb {C}^n\), we will denote the Kobayashi pseudo-distance by \(K_D\), and the infinitesimal Kobayashi metric by \(\kappa _D\). We will denote the Poincaré metric and distance on \(\mathbb {D}\) by \(\omega \) and \(\mathsf{p}_{\!{{\scriptstyle {\mathbb {D}}}}}\), respectively.

The notion of complex geodesics was introduced by Vesentini ([25]) and is a useful tool in the study of holomorphic mappings.

We need one more notion before we can state an important uniqueness result for certain geodesics.

The following uniqueness result for \(K_D\)-geodesics illustrates the importance of the above definition; see [20, Proposition 8.3.5] for a proof.

We now prove a proposition that is a simple consequence of the above uniqueness result. This proposition has already been obtained by Vesentini in the more general context of reflexive Banach spaces. We use his techniques to give a simple proof in our special case.

If we assume that the map \(F: \Omega \rightarrow \Omega \), as stated in Theorem 1.1 is branched, then our assumptions on the geometry of \(\Omega \) stated in the first sentence of Theorem 1.1 give us a structural result for the branch locus \(V_F\) of \(F\). We begin with the proof of this result—i.e., Proposition 1.5. We mention here that some of the arguments used in the proof were inspired by the arguments used in [13].

A comment about notation: in what follows, \(\dim _H(S)\) will denote the Hausdorff dimension of the set \(S \subset \mathbb {C}^n\).

Since the extension \(\widetilde{F}\) of \(F\) is uniquely determined by the latter, for the remainder of the proof we shall not use different symbols for \(F:\Omega \rightarrow \Omega \) and \(\widetilde{F}\).

We now have all the tools needed to prove our main theorem. Owing to the fact that the domain \(\Omega \) admits a smooth defining function that is plurisubharmonic in \(\Omega \), it is a fortiori pseudoconvex. We shall use this fact without explicit mention in our proof.