On the dimension of totally geodesic submanifolds in the Prym loci

Abstract

In this paper we give a bound on the dimension of a totally geodesic submanifold of the moduli space of polarised abelian varieties of a given dimension, which is contained in the Prym locus of a (possibly) ramified double cover. This improves the already known bounds. The idea is to adapt the techniques introduced by the authors in collaboration with A. Ghigi and G. P. Pirola for the Torelli map to the case of the Prym maps of (ramified) double covers.

1 Introduction

The purpose of this paper is to improve the estimates obtained in [4, 5] on the maximal dimension of a germ of a totally geodesic submanifold of the moduli space of polarised abelian varieties of a given dimension, which is contained in the Prym locus of a (possibly) ramified double cover. The idea is to adapt to the Prym case the technique developed in [6] and [9] to give a bound on the maximal dimension of a germ of a totally geodesic submanifold contained in the Torelli locus (see also [11]).

Denote by ${{\mathcal {R}}}_{g,r}$ the moduli space of isomorphism classes of triples $[(C, \alpha , R)]$ where C is a smooth complex projective curve of genus g, R is a reduced effective divisor of degree 2r on C and $\alpha$ is a line bundle on C such that $\alpha ^2={{\mathcal {O}}}_{C}(R)$.

A point $[(C, \alpha , R)] \in {{\mathcal {R}}}_{g,r}$ determines a double cover of C, $\pi :{\tilde{C}}\rightarrow C$ branched on R, with ${\tilde{C}}=Spec({{\mathcal {O}}}_{C}\oplus \alpha ^{-1})$.

This defines the Prym variety $P(C,\alpha ,R)$ which is the connected component containing 0 of the kernel of the norm map $Nm_{\pi }: J {\tilde{C}} {\rightarrow }JC$. If $r>0$, $\ker Nm_{\pi }$ is connected. The Prym variety $P(C,\alpha ,R)$ is an abelian variety of dimension $g-1+r$ endowed with a polarization $\Xi$, which is induced by restriction of the principal polarisation on $J{\tilde{C}}$. The polarisation $\Xi$ is of type $\delta =(1,\ldots , 1, \underbrace{2, \ldots ,2}_{g \text { times }} \ )$ for $r>0$, while if $r=0,1$ it is twice a principal polarisation and we endow $P({\tilde{C}}, C)$ with this principal polarisation. Denote by ${{\mathcal {A}}}^{\delta }_{g-1+r}$ the moduli space of abelian varieties of dimension $g-1+r$ with a polarization of type $\delta$. The Prym map $P_{g,r}: {{\mathcal {R}}}_{g,r} \rightarrow {{\mathcal {A}}}^{\delta }_{g-1+r}$ is defined as follows: $P_{g,r}([C, \alpha , R]):= [(P(C,\alpha ,R), \Xi )]$.

The map $P_{g,r}$ is generically finite, if and only if $\dim {{\mathcal {R}}}_{g,r} \le \dim {{\mathcal {A}}}^{\delta }_{g-1+r},$ and this holds if: either $r\ge 3$ and $g\ge 1$, or $r=2$ and $g\ge 3$, $r=1$ and $g\ge 5$, $r=0$ and $g \ge 6$. If $r =0$ the Prym map is generically injective for $g \ge 7$ [10, 13]. If $r>0$, the works of Marcucci and Pirola [16], Marcucci and Naranjo [15] and Naranjo and Ortega [18] show the generic injectivity in all the cases except for $r=2$, $g =3$, which was previously studied in [2, 17] and for which the degree of the Prym map is 3. Recently a global Prym-Torelli theorem was proved for all g and $r\ge 6$ ([12] for $g=1$, [19] for all g).

In [7] an analogous question to the Coleman Oort conjecture for the Prym maps $P_{g,r}$ for $r=0,1$ was formulated. Namely, the authors asked whether there exist Shimura subvarieties (hence totally geodesic subvarieties) of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ ($r=0,1$) generically contained in the Prym loci.

In [7] and in [8] examples of Shimura curves generically contained in the (possibly ramified) Prym loci have been constructed for low values of g and the computations of such examples suggest that they should not exist if g is sufficiently high.

Assume that $\dim {{\mathcal {R}}}_{g,r} \le \dim {{\mathcal {A}}}^{\delta }_{g-1+r}$, so that the differential of the Prym map is generically injective and denote by ${{\mathcal {R}}}^0_{g,r}$ the open subset of ${{\mathcal {R}}}_{g,r}$ where the differential of the Prym map is injective. In this paper we give an estimate on the maximal dimension of a germ of a totally geodesic submanifold generically contained in the Prym loci, passing through a point $[(C, \alpha , R)] \in {{\mathcal {R}}}^0_{g,r}$ in terms of the gonality k of the curve C. The results are summarised the following Theorem and they improve the estimates given in [4, Theorem 3.2] and [5, Theorem 3.2].

Theorem 1.1

(Theorems 5.1, 6.1) Let $[C, \alpha , R] \in {{\mathcal {R}}}^0_{g,r}$, where C is a curve of genus $g>0$. Denote by k its gonality and assume that C has no involutions and that $g+r \ge k+3$. Denote by Y a totally geodesic subvariety contained in $P_{g,r}({{\mathcal {R}}}^0_{g,r})$ and passing through $P_{g,r}([C, \alpha , R])$.

• (1) If $r >k+1$, then

  $$\begin{aligned} \dim (Y) \le \frac{3}{2}g - \frac{1}{2} + r +k . \end{aligned}$$

• (2) If $g +r \ge k+4$, $r \le k+1$, $b+r \ge 5$, then

  $$\begin{aligned} \dim (Y) \le \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2} \le \frac{3}{2}g +2+ r +k. \end{aligned}$$

• (3) If $g+r \ge k+4+m$, $r \le k+1$, $b+r=5-m$, $m \ge 1$, then

  $$\begin{aligned} \dim (Y) \le \frac{3}{2}g + k + 2 - \frac{m}{2} + \frac{r}{2}. \end{aligned}$$

From this, using that the gonality $k \le [\frac{g+3}{2}]$, we deduce the following result which improves the estimates obtained in [4, Theorem 3.4] , [5, Theorem 3.4].

Theorem 1.2

(Corollaries 5.3, 6.3) Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,r})$, with $g \ge 3$. Assume that there exists a point $[C, \alpha , R] \in Y$ such that C has no involutions.

• (1) If $g<2r-5$, then $\dim Y\le 2g+r$ if g even, $\dim Y\le 2g+r+1$ if g is odd.

• (2) If $g \ge 2r-5$, $r \ge 5$, then $\dim Y\le 2g+r+3$.

• (3) If $r=4$, if $g \ge 4$, then $\dim Y \le 2g+4$ if g is even, $\dim Y \le 2g+5$ if g is odd.

• (4) If $r=3$, if $g \ge 8$, then $\dim Y \le 2g+3$ if g is even, $\dim Y \le 2g+4$ if g is odd.

• (5) If $r=2$, if $g \ge 12$, then $\dim Y \le 2g+2$ if g is even, $\dim Y \le 2g+3$ if g is odd.

• (6) If $r=1$, if $g \ge 16$, then $\dim Y \le 2g+1$ if g is even, $\dim Y \le 2g+2$ if g is odd.

• (7) If $r=0$, if $g \ge 20$, then $\dim Y \le 2g$ if g is even, $\dim Y \le 2g+1$ if g is odd.

The improvements in the above estimates are obtained using ideas introduced in [9] that allow to use more than one quadric to compute the second fundamental form of the Torelli map. Here we need to adapt this technique to the more complicated case of the Prym maps.

In the cases in which the assumptions of this theorem do not hold (i.e. for low values of g and r), we still have the estimates done using only one quadric (Theorem 7.2), as in [4, 5], that are improved here thanks to a more careful study of the base locus of the linear system $M:=K_C\otimes \alpha \otimes F^{-1}$, where F is the $g^1_k$ on the curve C.

The result is the following

Theorem 1.3

(Theorem 7.2) Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,r})$.

• (1) If $g=1$, $r =4$, then $\dim Y\le 7$.

• (2) If $g=2$, $r =4$, then $\dim Y\le 9$.

• (3) If $g=3$, $r =4$, then $\dim Y\le 12$.

• (4) If $2 \le g\le 7$, $r=3$, $\dim Y\le \frac{9}{4} g +4$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{17}{4}$, if g is odd.

• (5) If $4 \le g \le 11$, $r =2$, then $\dim Y\le \frac{9}{4}g+4$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{17}{4}$, if g is odd.

• (6) If $6 \le g \le 15$, $r =1$, then $\dim Y\le \frac{9}{4}g+\frac{5}{2}$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{11}{4}$, if g is odd.

• (7) If $8 \le g \le 19$, $r =0$, then $\dim Y\le \frac{9}{4}g+1$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{5}{4}$, if g is odd.

Finally, in the case $r \ge 4$, in Proposition 7.3 we get a better estimate than the one in Theorem 1.1, if the curve C is hyperelliptic, which is possible since a global Prym-Torelli theorem is proved in [12, 19].

The structure of the paper is the following: In Sect. 2 we introduce the notation and recall the results on the second fundamental form of the ramified Prym map obtained in [5].

In Sect. 3 we do a careful study of the linear system $M:=K_C\otimes \alpha \otimes F^{-1}$, where F is the $g^1_k$ on the curve C. This technical part is crucial to get the estimates.

In Sect. 4 we explain the technique introduced in [6] and in [9] and adapt it in the case of the Prym-canonical linear system $K_C \otimes \alpha$, to construct certain quadrics containing the Prym-canonical image of the curve C, where we are able to compute the second fundamental form of the Prym map.

In Sect. 5 we prove Theorems 1.1, 1.2 for $r \ge 5$.

In Sect. 6 we prove Theorems 1.1, 1.2 for $r \le 4$.

In Sect. 7 we do the estimate using only one quadric in the missing cases, that is when $r \le 4$ and for low values of g. Finally we prove Proposition 7.3, in which we allow C to be hyperelliptic if $r \ge 4$.

2 Second fundamental form of the Prym map

In this section we recall the results contained in [5, Section 2]. Denote by C a smooth complex projective curve of genus g, let R be a reduced divisor of degree 2r on C and $\alpha$ a line bundle on C such that $\alpha ^2={{\mathcal {O}}}_{C}(R)$. To such data we can associate a double cover $\pi :{\tilde{C}}\rightarrow C$ branched on R. The Prym variety $P(C,\alpha , R)$ associated with this data is the polarised abelian variety given by the connected component containing the origin of kernel of the norm map $Nm_{\pi }:J{\tilde{C}}\rightarrow JC$. For $r>0$ the kernel of the norm map is connected. For $r>1$ the polarisation $\Xi$ is the restriction to $P(C,\alpha , R)$ of the principal polarisation $\Theta _{{\tilde{C}}}$ on the jacobian of ${\tilde{C}}$. For $r=0,1$ the polarisation $\Theta _{{\tilde{C}}|P(C,\alpha ,R)}$ is twice a principal polarisation $\Xi$ and we endow $P(C,\alpha , R)$ with this principal polarisation.

The Prym map $P_{g,r}: {{\mathcal {R}}}_{g,r} \rightarrow {{\mathcal {A}}}^{\delta }_{g-1+r},$ is the map that associates to a point $[(C,\alpha ,R)] \in {{\mathcal {R}}}_{g,r}$ the isomorphism class of its Prym variety $P(C,\alpha , R)$ with the polarisation $\Xi$.

The dual of the differential of the Prym map $P_{g,r}$ at a generic point $[(C, \alpha , R)]$ is given by the multiplication map

$$\begin{aligned} (dP_{g,r})^* : S^2H^0(C, K_C \otimes \alpha ) {\rightarrow }H^0(C, K_C^2(R)) \end{aligned}$$

(1)

and it is surjective at the generic point if either $r\ge 3$ and $g\ge 1$, or $r=2$ and $g\ge 3$, $r=1$ and $g\ge 5$, $r=0$ and $g \ge 6$ (see [14]).

In these cases we denote by ${{\mathcal {R}}}^0_{g,r}$ the non empty open subset of ${{\mathcal {R}}}_{g,r}$ where the Prym map $P_{g,r}$ is an immersion. Notice that if $r \ge 3$, a global Torelli theorem is proved in [19] (and in [12] for $g=1$), hence ${{\mathcal {R}}}^0_{g,r} = {{\mathcal {R}}}_{g,r}$.

Let us consider the (orbifold) tangent bundle exact sequence of the Prym map

$$\begin{aligned} 0 \rightarrow T_{{{\mathcal {R}}}^0_{g,r}} \rightarrow P_{g,r}^*T_{{{\mathcal {A}}}^{\delta }_{g-1+r}} \rightarrow {{\mathcal {N}}_{{{\mathcal {R}}}^0_{g,r}/{{\mathcal {A}}}^{\delta }_{g-1+r}}} \rightarrow 0 \end{aligned}$$

(2)

We endow ${{\mathcal {A}}}^{\delta }_{g-1+r}$ with the orbifold metric induced by the symmetric metric on the Siegel space ${{\mathcal {H}}}_{g-1+r}$. The dual of the associated second fundamental form with respect to the metric connection of the above exact sequence is a map

$$\begin{aligned} \rho _P: {{\mathcal {N}}^*_{{{\mathcal {R}}}^0_{g,r}/{{\mathcal {A}}}^{\delta }_{g-1+r}}} \rightarrow S^2 \Omega ^1_{{{\mathcal {R}}}^0_{g,r}}. \end{aligned}$$

(3)

In [5] a description of this second fundamental form is given in terms of the second fundamental form of the Torelli map of the covering curves ${\tilde{C}}$. Denote by U the open subset of ${{\mathcal {R}}}^0_{g,r}$ where there is a universal family ${{\tilde{f}}} : \tilde{{\mathcal {C}}} \rightarrow U$ and the differential of the modular map $U {\rightarrow }{{\mathcal {M}}}_{{\tilde{g}}}$, $[{\tilde{C}} {\rightarrow }C] \mapsto [{\tilde{C}}]$ is injective.

At the point $b_0:= [(C,\alpha ,R)] \in U$ corresponding to the 2 : 1 cover $\pi :{\tilde{C}} {\rightarrow }C$, the space $P_{g,r}^*\Omega ^1_{{{{\mathcal {A}}}^{\delta }_{g-1+r}}, b_0}$ is isomorphic to $S^2H^0(K_C \otimes \alpha )$, $\Omega ^1_{{{\mathcal {R}}}^0_{g,r},b_0}$ is isomorphic to $H^{0}(K_C^2(R))$, ${{\mathcal {N}}^*_{{{{\mathcal {R}}}^0_{g,r}/{{\mathcal {A}}}^{\delta }_{g-1+r, b_0}} }}\cong I_2(K_C \otimes \alpha )$, and the dual of the exact sequence (2) at the point $b_0$ becomes

$$\begin{aligned} 0 \rightarrow I_2(K_C\otimes \alpha ) \rightarrow S^2H^0(K_C \otimes \alpha ) {\mathop {\rightarrow }\limits ^{m}}H^0(K_C^2(R)) \rightarrow 0. \end{aligned}$$

The dual of the second fundamental form of the Prym map at the point $b_0$ is a map

$$\begin{aligned} \rho _P: I_2(K_C\otimes \alpha )\rightarrow S^2H^0(K_C^2(R)) \end{aligned}$$

(4)

Denote by $I_2(K_{{{\tilde{C}}}})$ the kernel of the multiplication map $S^2H^0(K_{{{\tilde{C}}}}) {\rightarrow }H^0(2K_{{{\tilde{C}}}})$ and by $I_2(K_{{{\tilde{C}}}})^+$ its invariant subspace by the action of the involution on ${\tilde{C}}$. Observe that we have the inclusion $I_2(K_C\otimes \alpha ) {\mathop {\hookrightarrow }\limits ^{\pi ^*}}I_2(K_{{\tilde{C}}})^+$. Moreover by the projection formula we have $H^1(T_C(-R)) \cong H^1(T_{{\tilde{C}}})^+$. In [5, Thm. 2.1, formula (2.13)] we have shown that $\forall Q \in I_2(K_C\otimes \alpha )$, $\forall v_1, v_2 \in H^1(T_C(-R))$, we have:

$$\begin{aligned} \rho _P(Q)(v_1 \odot v_2) = {\tilde{\rho }}(\pi ^*Q)(v_1 \odot v_2). \end{aligned}$$

(5)

3 Gonality

Let $[C, \alpha , R] \in {{\mathcal {R}}}^0_{g,r}$, where C is a curve of genus $g>0$. Denote by k its gonality. We have $2\le k\le [\frac{g+3}{2}]$. Assume $g+r \ge k+3$. Let F be a line bundle on C of degree k and such that $h^0(F)=2.$ Set $M:=K_C\otimes \alpha \otimes F^{-1}$, where $K_C$ is the canonical bundle. Set $n+1:= h^0(M)$. From Riemann Roch we get

$$\begin{aligned} h^0(M)= & {} h^0(F \otimes \alpha ^{-1})+2g-2+ r-k-(g-1)\nonumber \\= & {} h^0(F \otimes \alpha ^{-1}) +g-1+r-k \ge g+r -(k+1) \end{aligned}$$

(6)

Hence $h^0(M) \ge 2$, since $g+r \ge k+3$. The Clifford index of C is either $k-2$ (computed by F) or $k-3$. Denote by B the base locus of M and set $b:= deg(B)$. The line bundle M has degree equal to $2g-2+r-k$.

Proposition 3.1

Assume $g \ge 1$, $g +r \ge k+3$.

• (1) If $r >k+1$, then $b=0$ and $h^0( F \otimes \alpha ^{-1})=0$.

• (2) If $r >k+2$, then |M| is very ample.

• (3) If $r = k+2$, and D is an effective divisor of degree s such that $h^0(M(-D)) = h^0(M) -1=n$, then $s \le 2$. Moreover $s=2$ if and only if either C is hyperelliptic and $F \cong {{\mathcal {O}}}_C(D)$, or |M| gives a birational map that contracts D.

Proof

Since $r > k+1$, then $deg(M) =2g-2+r-k > 2g-1$, hence $B = \emptyset$. We have $deg(F \otimes \alpha ^{-1}) = k-r<0$, so $h^0( F \otimes \alpha ^{-1})=0$. If $r > k+2$, then $deg(M) =2g-2+r-k > 2g$, therefore |M| is very ample.

Assume $r = k+2$ and let D be an effective divisor of degree s such that $h^0(M(-D)) = h^0(M)-1$.

By Riemann Roch we have $h^0(F(D) \otimes \alpha ^{-1})= h^0(F \otimes \alpha ^{-1}) + s -1 = s-1$, $\deg (F(D) \otimes \alpha ^{-1}) = s-2$, so Clifford’s theorem implies $s \le 2$ and $s=2$ if and only if $F(D) \otimes \alpha ^{-1} \cong {{\mathcal {O}}}_C$. In the case $s=2$, either C is hyperelliptic and $F \cong {{\mathcal {O}}}_C(D)$, or |M| gives a birational map to its image that contracts D. $\square$

Corollary 3.2

If $g \ge 3$, C is not hyperelliptic, $g +r \ge k+3$, and $r >k+1$, then the morphism given by |M| is birational onto its image.

Proof

The proof follows from Proposition 3.1. $\square$

Assume now that $r \le k+1$. We have

$$\begin{aligned} h^1(M(-B)) = b + h^0(F \otimes \alpha ^{-1}). \end{aligned}$$

(7)

We have the following

Lemma 3.3

Assume $g+r \ge k+3$, $r \le k+1$. If either $b \ge 2$, or $b =1$ and $h^0(F\otimes \alpha ^{-1}) = 1$, then

• (1) $b \le 5-r -2h^0(F \otimes \alpha ^{-1})$.

• (2) If $b\ge 2$, $r \le 3 - 2h^0(F \otimes \alpha ^{-1})$.

• (3) If $b=1=h^0(F \otimes \alpha ^{-1})$, $r \le 2$.

Proof

If either $b \ge 2$, or $b =1$ and $h^0(F\otimes \alpha ^{-1}) = 1$, (6) and (7) show that the linear system $|M(-B)|$ contributes to the Clifford index, hence

$$\begin{aligned} cliff(M(-B))= & {} 2g-2 +r-k-b -2(h^0(M(-B)) -1)\\= & {} -r+ k-b+2-2h^0(F \otimes \alpha ^{-1}) \ge k-3. \end{aligned}$$

Therefore $b \le 5-r -2h^0(F \otimes \alpha ^{-1})$. Moreover $r \le 5-b -2h^0(F \otimes \alpha ^{-1}) \le 3- 2h^0(F \otimes \alpha ^{-1})$ if $b \ge 2$. If $b=1 = h^0(F \otimes \alpha ^{-1})$, $r \le 5-b -2h^0(F \otimes \alpha ^{-1}) =2.$ $\square$

Corollary 3.4

Assume $g+r \ge k+3$, $r \le k+1$. Then we have

• (1) $b \le 5$.

• (2) If $r \ge 4$, then $b \le 1$.

• (3) If $r =3$, then $b \le 2$.

• (4) If $r \ge 3$ and $b =1$, then $h^0(F \otimes \alpha ^{-1}) =0$.

Proof

The proof follows immediately from Lemma (3.3). $\square$

Let us now study the morphism given by $|M-B|$. Assume that $g+r \ge k+4$.

Proposition 3.5

Assume $g+r \ge k+ 4$, $r \le k+1$. Let $D_1$ be an effective divisor of degree $s_1 \ge 3$ such that $h^0(M(-B-D_1)) = h^0(M(-B)) -1 = h^0(M)-1$. Then $s_1 \le -r-b-2 h^0(F \otimes \alpha ^{-1})+7$. Therefore $b+r \le 4$.

Proof

By Riemann Roch we have

$$\begin{aligned} h^0(F(B+D_1) \otimes \alpha ^{-1})= & {} h^0(F \otimes \alpha ^{-1}) + b + s_1 -1 \ge s_1-1 \ge 2, \\ h^1(F(B+D_1) \otimes \alpha ^{-1})= & {} h^0(M(-B-D_1)) = h^0(M) -1 = \\= & {} h^0(F \otimes \alpha ^{-1}) + g -2 -k+r\ge g+r - k -2 \ge 2, \end{aligned}$$

since $g+r \ge k+4$. Hence $F(B+D_1) \otimes \alpha ^{-1}$ contributes to the Clifford index and we have $cliff(F(B+D_1) \otimes \alpha ^{-1}) = k-r+b+s_1 - 2( h^0(F(B+D_1) \otimes \alpha ^{-1}) -1) = k-r-b-s_1 -2 h^0(F \otimes \alpha ^{-1}) +4 \ge k-3.$ So we have $s_1 \le -r-b+7 -2 h^0(F \otimes \alpha ^{-1})$. Thus $b+r \le 7-s_1 -2 h^0(F \otimes \alpha ^{-1}) \le 4$. $\square$

Corollary 3.6

If $g+r \ge k+4$, $r \le k+1$, $b+r \ge 5$, then $s_1 \le 2$, hence the morphism induced by $|M(-B)|$ is either birational on its image, or it has degree 2 on its image.

Proof

By Proposition 3.5 we have $s_1 \le -(r +b) +7 -2h^0(F \otimes \alpha ^{-1}) \le 2$, so the morphism induced by $|M(-B)|$ has degree at most 2 on its image. $\square$

Assume now that $b+r \le 4$.

Consider an effective divisor $D_1$ as above of maximal degree $s_1 \ge 3$ with the property that $h^0(M(-B-D_1)) = h^0(M(-B)) -1 = h^0(M) -1 = n$.

Then clearly the linear system $|M(-B-D_1)|$ is base point free and $h^0(M(-B-D_1)) = n \ge 2$, if $g+r \ge k+4$. Under this assumptions, in Proposition 3.5, we have shown that $s_1 \le 7 -(r+b) - 2h^0(F\otimes \alpha ^{-1})$.

We want to study the linear system $|M(-B-D_1)|$ and give sufficient conditions ensuring that the map associated with $|M(-B-D_1)|$ has at most degree 2 on its image. Assume that $g+r \ge k+5$. We have the following

Proposition 3.7

Assume $g+r \ge k+5$, $b+r \le 4$, $r \le k+1$. With the above notation, assuming $s_1 \ge 3$, let $D_2$ be an effective divisor of degree $s_2>0$ such that $h^0(M(-B-D_1-D_2)) = h^0(M(-B-D_1)) -1 = n-1$. Then $s_2\le 9-(r+b+s_1) -2 h^0(F \otimes \alpha ^{-1}) \le 6-(r+b).$

Proof

Assume that $D_2$ is an effective divisor of degree $s_2$ such that $h^0(M(-B-D_1-D_2))= h^0(M(-B-D_1)) -1 = n-1.$ Then we have

$$\begin{aligned} h^1(F(B+D_1+D_2) \otimes \alpha ^{-1})= & {} h^0(M(-B-D_1-D_2)) =n-1\\= & {} h^0(F \otimes \alpha ^{-1}) + g-1 +r-k -2 \ge 2, \end{aligned}$$

since $g+r \ge k+ 5$.

$$\begin{aligned} h^0(F(B+D_1+D_2) \otimes \alpha ^{-1})= & {} h^0(M(-B-D_1-D_2)) + k+s_1+s_2+b-r -g+1\\= & {} h^0(F \otimes \alpha ^{-1}) + s_1+s_2 +b -2 \ge s_2+1, \end{aligned}$$

since $s_1 \ge 3.$ So $F(B+D_1+D_2) \otimes \alpha ^{-1}$ contributes to the Clifford index, hence we have

$$\begin{aligned}&cliff(F(B+D_1+D_2) \otimes \alpha ^{-1}) \\&\quad = k + b + s_1 + s_2 - r - 2(h^0(F(B+D_1+D_2) \otimes \alpha ^{-1}) -1) \\&\quad = k-(r+b+s_1+s_2) + 6 -2 h^0(F \otimes \alpha ^{-1}) \ge k-3, \end{aligned}$$

if and only if

$$\begin{aligned} s_2\le & {} 9-(r+b+s_1) -2 h^0(F \otimes \alpha ^{-1}) \\\le & {} 6-(r+b) -2 h^0(F \otimes \alpha ^{-1}) \le 6-(r+b). \end{aligned}$$

$\square$

Corollary 3.8

If $g+r \ge k+5$, $b +r = 4$, $r \le k+1$, then the morphism induced by $|M(-B)|$ is either birational or it has degree 2 on its image.

Proof

By Proposition 3.7 we know that $s_2 \le 2$, hence the morphism induced by $|M(-B-D_1)|$ is either birational, or it has degree 2 on its image, hence the same holds for the morphism induced by $|M(-B)|$. $\square$

Now we repeat the strategy explained above for the cases $b+r \le 3$. Set $D_l$ an effective divisor of maximal degree $s_l$, $l=1,\ldots ,5$, such that $h^0(M(-B-D_1-\cdots -D_l) )= n+1-l \ge 2$. This holds if $g+r \ge k+3 + l$.

Proposition 3.9

Assume that $g+r \ge k+4 + m$, $b+r \le 5-m$, $r \le k+1$, $m \ge 1$. Assume $s_1,\ldots ,s_{m} \ge 3$, let $D_{m+1}$ be an effective divisor of degree $s_{m+1}>0$ such that $h^0(M(-B-D_1-\cdots -D_{m+1})) = h^0(M(-B-D_1-\cdots -D_{m})) -1 = n-m$. Then $s_{m+1}\le 5+2(m+1)-(r+b+s_1+\cdots +s_{m}) -2 h^0(F \otimes \alpha ^{-1}) \le 7-m-(r+b)-2 h^0(F \otimes \alpha ^{-1}) .$

Proof

We have $deg(F(B+D_1+\cdots +D_{m+1}) \otimes \alpha ^{-1})= k+b +s_1+\cdots +s_{m+1} -r$,

$$\begin{aligned} h^1(F(B+D_1+\cdots +D_{m+1}) \otimes \alpha ^{-1})= & {} h^0(M(-B-D_1-\cdots -D_{m+1})) = n-m,\\= & {} h^0(F \otimes \alpha ^{-1}) + g-1 +r-k -(m+1) \ge 2, \end{aligned}$$

since $g+r \ge k+ 4+m$,

$$\begin{aligned}&h^0(F(B+D_1+\cdots D_{m+1}) \otimes \alpha ^{-1}) \\&\quad =h^0(F \otimes \alpha ^{-1}) + s_1+\cdots +s_{m+1} +b -(m+1) \ge 2m \ge 2, \end{aligned}$$

since $s_i \ge 3,$ $i=1,\ldots ,m$, $s_{m+1} \ge 1$, $m \ge 1$. So $F(B+D_1+\cdots +D_{m+1}) \otimes \alpha ^{-1}$ contributes to the Clifford index, hence we have

$$\begin{aligned}&cliff(F(B+D_1+\cdots +D_{m+1}) \otimes \alpha ^{-1}) \\&\quad =k-(r+b+s_1+\cdots +s_{m+1}) + 2(m+2) -2 h^0(F \otimes \alpha ^{-1}) \ge k-3, \end{aligned}$$

if and only if

$$\begin{aligned} s_{m+1}\le & {} 5+2(m+1)-(r+b+s_1+\cdots +s_{m}) -2 h^0(F \otimes \alpha ^{-1}) \\\le & {} 7-m-(r+b) -2 h^0(F \otimes \alpha ^{-1}) . \end{aligned}$$

$\square$

Corollary 3.10

If $g+r \ge k+4+m$, $b +r = 5-m$, then the morphism induced by $|M(-B)|$ is either birational or it has degree 2 on its image.

Proof

By Proposition 3.9 we know that $s_{m+1} \le 2$, hence the morphism induced by $|M(-B-D_1-\cdots -D_{m})|$ is either birational, or it has degree 2 on its image, hence the same holds for the morphism induced by $|M(-B)|$. $\square$

4 Quadrics and estimate on the second fundamental form

In this section we explain how to construct certain quadrics contained in $I_2(K_C \otimes \alpha )$ on which we are able to compute the second fundamental form of the Prym map $\rho _P$. Here we adapt the techniques introduced in [6, 9] for the Torelli map to the case of the Prym maps.

4.1 The quadrics and the second fundamental form

With the above notation, assume that the morphism f induced by $|M(-B)|$ is birational on its image. Set $d = deg(M(-B))= 2g-2+r-k-b$, $n+1 = h^0(M(-B)) = h^0(M)= h^0(F \otimes \alpha ^{-1}) +g-1+r-k$, and fix now and for all two independent sections $x,y \in H^0(F).$ Take a section $t \in H^0(M(-B))$ and consider the associated divisor $D(t)\in |M(-B)|.$ Assume that t is such that:

• (1) $D(t)\cap (Z\cup B)=\emptyset ,$ where Z is the ramification divisor of the map $\phi _{\langle x, y \rangle }: C {\rightarrow }{{\mathbb {P}}}^1$.

• (2) $D(t)=p_1+\cdots +p_d$, $p_i\ne p_j$ if $i\ne j.$

• (3) The points $p_i$ are in general linear position: for any group of or n distinct points $p_{i_1},\ldots , p_{i_n},$ we have $H^0(M(-B-(p_{i_1}+\cdots +p_{i_n})))=\langle t \rangle$.

The last condition follows for instance from the uniform lemma of Castelnuovo (see e.g. [1, Ch.3]) since $f:C\rightarrow {{\mathbb {P}}}^n$ is birational onto its image.

Consider the exact sequence induced by t

$$\begin{aligned} 0\rightarrow {{\mathcal {O}}}_C{\mathop {\rightarrow }\limits ^{t}}M(-B)\rightarrow M(-B)_D\rightarrow 0. \end{aligned}$$

We get

$$\begin{aligned} M(-B)_D\cong \sum {{\mathbb {C}}}_{p_i}, \end{aligned}$$

where the last isomorphism follows from the choice of local trivializations of $M(-B).$ Let $W\subset H^0(M(-B))$ be complementary to t :  $H^0(M(-B))=\langle t \rangle \oplus W$, so that $\dim W=n$. Consider the induced injection $j: W\rightarrow H^0(\oplus {{\mathbb {C}}}_{p_i})={{\mathbb {C}}}^d.$ We can rewrite the linear uniform condition. For any $s\in W$, $s\ne 0$, then the vector $j(s)=(a_1,\ldots , a_d)$ has at most $n-1$ coordinates that are zero.

Let $\tau \in H^0({{{\mathcal {O}}}}_C(B))$, such that $D(\tau ) = B$, then $\forall s \in W$, ${\tilde{s}} = \tau s$ and ${\tilde{t}} = \tau t$, are sections in $H^0(M)$. Let $I_2(K_C \otimes \alpha )\subset S^2 H^0(K_C\otimes \alpha ))$ be the kernel of the multiplication map

$$\begin{aligned} m: S^2 H^0(K_C\otimes \alpha ) {\rightarrow }H^0(K_C^{\otimes 2}(R)). \end{aligned}$$

Consider the quadric

$$\begin{aligned} Q_{s} = x{\tilde{t}} \odot y{\tilde{s}} - x{\tilde{s}} \odot y{\tilde{t}} \end{aligned}$$

Clearly $Q_s \in I_2(K_C \otimes \alpha ),$ $\forall s \in W$. Denote by $\pi : {\tilde{C}} {\rightarrow }C$ the 2 : 1 cover given by $\alpha$, and by $\psi : C {\rightarrow }{{\mathbb {P}}}^1$ the morphism induced by $\langle s,t\rangle$. Then clearly $\{p_1,\ldots ,p_d\} = Z(t) = \psi ^{-1}(1:0)$. Set $\{T_i, \sigma (T_i)\} = \pi ^{-1}(p_i)$, where $\sigma$ denotes the involution of the double cover $\pi$. For all $i = 1,\ldots ,d$, set $v_i:= \xi _{T_i} + \xi _{\sigma (T_i)} \in H^1(T_{{{\tilde{C}}}})^+ \cong H^1(T_C(-R))$.

Denote by $V = \langle v_1,\ldots ,v_d \rangle \subset H^1(T_{{{\tilde{C}}}})^+$, and set

$$\begin{aligned} \beta : W\rightarrow S^2V^*, \ \beta (s)(v \odot w) = \rho _P(Q_s)( v \odot w) ={\tilde{\rho }}(\pi ^*(Q_s))(v \odot w), \ \forall v,w \in V. \end{aligned}$$

(8)

where the last equality follows from (5).

The following result has been proved in [5, (3.2),(3.3)], with the variant that all the quadrics $Q_s$, $s \in W$, are taken into account.

Theorem 4.1

Let $x_1,\ldots x_d$ be the basis of $V^*$ dual to the basis $\{v_1,\ldots ,v_d\}$ of V. Then

$$\begin{aligned} \beta (s)= \lambda \sum _{i=1}^d a_ix_i^2 \end{aligned}$$

where $\lambda \ne 0$ is a constant independent of s, $j(s)=(a_1,\ldots ,a_d)$ where $j:W\rightarrow {{\mathbb {C}}}^d$ is the evaluation map. The quadrics $\beta (s)$ are simultaneously diagonalized and for any $s\ne 0,$ $\mathrm{{rank}} (\beta (s))\ge d-n+1.$

Proof

By [5, (3.2)], for $i \ne j$ we have:

$$\begin{aligned} \pi ^*(Q_s)(T_i, T_j) = Q_s(p_i, p_j) =0. \end{aligned}$$

Hence,

$$\begin{aligned} \rho _P(Q) (v_i \odot v_j)= & {} {\tilde{\rho }}(\pi ^*(Q_s))(v_i \odot v_j) = 0, \ \forall i \ne j, \\ \rho _P(Q) (v_i \odot v_i)= & {} {\tilde{\rho }}(\pi ^*(Q_s))(v_i \odot v_i) = c \mu _2(Q_s)(p_i), \end{aligned}$$

where $\mu _2: I_2(K_C \otimes \alpha ) {\rightarrow }H^0(4K_C(R))$ is the second Gaussian map of the bundle $K_C \otimes \alpha$ (see [5, section 2]). A local computation gives $\mu _2(Q_s) = \mu _{1,F}(x \wedge y) \mu _{1, M}({\tilde{s}} \wedge {\tilde{t}})$, where, for a line bundle L, $\mu _{1,L}: \Lambda ^2 H^0(L) {\rightarrow }H^0(K_C \otimes L^2)$ denotes the first Gaussian map of L (see e.g. [3, Lemma 2.2]). In local coordinates one computes

$$\begin{aligned} \mu _1({\tilde{s}} \wedge {\tilde{t}}) = (\tau s)' (\tau t) - (\tau s) (\tau t)' = (\tau ' s + \tau s') (\tau t) - (\tau s)( \tau ' t + \tau t') = \tau ^2(s't-st'). \end{aligned}$$

Hence $\mu _1({\tilde{s}} \wedge {\tilde{t}}) (p_i) = \tau ^2(p_i) (-s(p_i)t'(p_i)) =k s(p_i) = ka_i,$ where k is a non zero constant, that does not depend on s, by the assumptions on the section t. So $\mu _2(Q_s)(p_i) = (\mu _1(x \wedge y) \mu _1({\tilde{s}} \wedge {\tilde{t}}))(p_i) = \lambda a_i$, for a non zero constant $\lambda$, which is independent of s. $\square$

4.2 The zero locus of the quadrics and the estimate

For an element $z \in V$, write $z = \sum _{i=1}^d z_i v_i$ and denote by $[z]:=[z_1,\ldots ,z_d] \in {{\mathbb {P}}}^{d-1} \cong {{\mathbb {P}}}(V)$. Consider the locus

$$\begin{aligned} Z=\left\{ [z] \in {{\mathbb {P}}}^{d-1}: \beta (s)(z \odot z)={\tilde{\rho }}(\pi ^*(Q_s))(z \odot z)=0, \ \forall s\in W\right\} . \end{aligned}$$

First we have the following

Lemma 4.2

Set $H=\{[z] \in {{\mathbb {P}}}^{d-1}: z_i=0,\ i>n\},$ then $Z \cap H=\emptyset ,$ therefore $\dim Z= d-n-1.$

Proof

Notice that by the uniform position (see e.g. [1, Ch.3]), we know that for all $i \in \{1,\ldots ,n\}$, there exists exactly a section $s_i\in W$ such that $s_i(p_j) =0$, $\forall j \in \{1,\ldots ,n\}$, $j \ne i$, $s_i(p_i) \ne 0$, $s_i(p_k) \ne 0$, $\forall k >n$, hence by Theorem 4.1 we get

$$\begin{aligned} \beta (s_i)= a_{i,i}x^2_i+\sum _{j>n} a_{i,j}x^2_j, \end{aligned}$$

(9)

with $a_{i,i} \ne 0$, $a_{i,j} \ne 0$, $\forall j >n$. Take $[z] = [z_1,\ldots ,z_n,0,\ldots ,0]\in H$ such that $\beta (s)(z \odot z) =0$, $\forall s \in W$. Set $Q_i:= Q_{s_i},$ then ${\tilde{\rho }}(\pi ^*(Q_i))(z \odot z)= a_{i,i} z_i^2 =0$ $\forall i$ if and only if $z_i=0$, $\forall i =1,\ldots ,n$, which is impossible, since $[z] \in H$. Therefore we have $H\cap Z=\emptyset$ and then $\dim Z \le d-1-n.$ Notice that $Z=\{[v]\in {{\mathbb {P}}}^{d-1}: {\tilde{\rho }}(\pi ^*(Q_i))(v \odot v)=0, \ i=1,\ldots ,n\}$, so $\dim Z = d-1-n$. $\square$

We need to estimate the dimension of a linear space $\Pi \subset Z.$ Denote by T the linear subspace of V corresponding to $\Pi$.

Consider the map $h:V\rightarrow V$, $\pi (x_1,\ldots , x_d)= (0,\dots ,0,x_{n},\ldots , x_d)$ The restriction of h to T is injective since $\Pi \subset Z$. By formula (9) we can see $\beta (s_n)$ as a quadric in h(V).

We have the inclusion

$$\begin{aligned} h(T)\subset \{v \in \pi (V) \ | \ \beta (s_n)(v \odot v) = {\tilde{\rho }}(\pi ^*(Q_{n}))(v \odot v)=0\}. \end{aligned}$$

Since $\beta (s_{n})$ has rank $d-n+1$, $\dim (h(V)) = d-n+1$ and $h: T \longrightarrow V$ is injective, we get:

Proposition 4.3

Assume that the morphism induced by $|M(-B)|$ is birational on its image. Set $d = deg(M(-B))= 2g-2+r-k-b$, $n+1 = h^0(M(-B)) = h^0(M)= h^0(F \otimes \alpha ^{-1}) +g-1+r-k$. With the above notation, let $\Pi$ be a linear subspace contained in Z,  and let T be the corresponding subspace of V. Then

$$\begin{aligned} \dim T \le \frac{d-n+1}{2} = \frac{g+1-b}{2} - \frac{h^0(F \otimes \alpha ^{-1})}{2}. \end{aligned}$$

5 The case $b+r \ge 5$

In this section we apply Proposition 4.3 in the case $b+r \ge 5$. We have the following

Theorem 5.1

Let $[C, \alpha , R] \in {{\mathcal {R}}}^0_{g,r}$, where C is a curve of genus $g>0$. Denote by k its gonality and assume that C has no involutions and that $g+r \ge k+3$. Denote by Y a totally geodesic subvariety contained in $P_{g,r}({{\mathcal {R}}}^0_{g,r})$ and passing through $P_{g,r}([C, \alpha , R])$.

• (1) If $r >k+1$, then

  $$\begin{aligned} \dim (Y) \le \frac{3}{2}g - \frac{1}{2} + r +k . \end{aligned}$$

• (2) If $g +r \ge k+4$, $r \le k+1$, $b+r \ge 5$, then

  $$\begin{aligned} \dim (Y) \le \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2} \le \frac{3}{2}g +2+ r +k. \end{aligned}$$

Proof

Under the above assumptions in both cases, by Corollaries 3.2, 3.6, the linear system $|M(-B)|$ induces a birational map on its image (in the first case $B = \emptyset$). Let S be the tangent space of Y at $P_{g,r}([C, \alpha , R])$. Let V be as above, then $T=S\cap V$ is a linear subspace where all the quadrics vanish, so by Proposition 4.3 we get: $\dim (S\cap V)\le \frac{g+1-b}{2}- \frac{h^0(F \otimes \alpha ^{-1})}{2}$.

Then $\dim S+ \dim V \le 3g-3 + 2r +\dim (S\cap V)$, hence

$$\begin{aligned} \dim S\le & {} 3g-3 + 2r -(2g-k-2+r-b)+\frac{g+1-b}{2}- \frac{h^0(F \otimes \alpha ^{-1})}{2}\\= & {} \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2}. \end{aligned}$$

In case (1) we have $b = 0 = h^0(F \otimes \alpha ^{-1})$, by Proposition 3.1. So we get $\dim Y = \dim S \le \frac{3}{2}g - \frac{1}{2} + r +k$. In case (2), we get $\dim Y = \dim S \le \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2}$. Then we conclude, since $b \le 5$, by Corollary 3.4. $\square$

Remark 5.2

In the second case of Theorem 5.1, since $b+r \ge 5$, we can apply Lemma 3.3 and we have the following cases:

• (1) If $b \ge 2$, then $b+r =5$ and $h^0(F \otimes \alpha ^{-1}) =0$, then $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2} = \frac{3}{2}g +2+ \frac{r}{2} +k$.

• (2) If $b=1$, then $h^0(F \otimes \alpha ^{-1}) =0$ , then $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + \frac{b}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2} = \frac{3}{2}g + r +k.$

• (3) If $b=0$, then $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + r +k - \frac{h^0(F \otimes \alpha ^{-1})}{2} \le \frac{3}{2}g - \frac{1}{2} + r +k .$

Corollary 5.3

Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,r})$, with $g \ge 3$. Assume that there exists a point $[C, \alpha , R] \in Y$ such that C has no involutions.

• (1) If $g<2r-5$, then $\dim Y\le 2g+r$ if g even, $\dim Y\le 2g+r+1$ if g is odd.

• (2) If $g \ge 2r-5$, $r \ge 5$, then $\dim Y\le 2g+r+3$.

Proof

In case (1), by assumption we have $r > \frac{g+5}{2} = \frac{g+3}{2} +1 \ge [(g+3)/2]+1 \ge k+1$. Since $k \ge 3$, $r \ge 5$, hence the result follows by Theorem 5.1, using that $k\le [(g+3)/2].$

In case (2), the condition $r \ge 5$ implies $g + r \ge \frac{g+3}{2} + 4 \ge k+4$. So if $r \le k+1$, we use the estimate (2) in Theorem 5.1, and the inequality $k\le [(g+3)/2]$ to conclude that $\dim Y\le 2g+r+3$. If $r >k+1$, we use the estimate (1) in Theorem 5.1 and we get $\dim Y\le 2g+r$ if g is even, $\dim Y\le 2g+r+1$ if g is odd. $\square$

6 The case $b+r \le 4$

Let us now consider the case $b+r \le 4$. We have the following.

Theorem 6.1

Let $[C, \alpha , R] \in {{\mathcal {R}}}^0_{g,r}$, where C is a curve of genus $g>0$, denote by k its gonality. Assume that C has no involutions. Denote by Y a totally geodesic subvariety contained in $P_{g,r}({{\mathcal {R}}}^0_{g,r})$ and passing through $P_{g,r}([C, \alpha , R])$. If $g+r \ge k+4+m$, $r \le k+1$, $b+r=5-m$, $m \ge 1$, then

$$\begin{aligned} \dim (Y) \le \frac{3}{2}g + k + 2 - \frac{m}{2} + \frac{r}{2}. \end{aligned}$$

Proof

By Corollaries 3.8, 3.10, the linear system $|M(-B)|$ is birational on its image. Let S be the tangent space of Y at $P_{g,r}([C, \alpha , R])$. Let V be as above, then $T=S\cap V$ is a linear subspace where all the quadrics vanish, so by Proposition 4.3 we get:

$$\begin{aligned} \dim S\le & {} 3g-3 + 2r -(2g-k-2+r-b)+\frac{g+1-b}{2}- \frac{h^0(F \otimes \alpha ^{-1})}{2}\\= & {} \frac{3}{2}g - \frac{1}{2} + \frac{b+r}{2} + \frac{r}{2} +k - \frac{h^0(F \otimes \alpha ^{-1})}{2}\\= & {} \frac{3}{2}g + 2 -\frac{m}{2} + \frac{r}{2} +k - \frac{h^0(F \otimes \alpha ^{-1})}{2}. \end{aligned}$$

$\square$

Corollary 6.2

Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,0})$, with $g \ge 4m$, $r=5-m$, $m \ge 1$. Assume that there exists a point $[C, \alpha , R] \in Y$ such that C has no involutions.Then $\dim Y \le 2g -m + 5$ if g is even, $\dim Y \le 2g -m + 6$ if g is odd.

Proof

By assumption we have $g+r = g+5-m \ge [\frac{g+3}{2}] + 4+m \ge k+4+m$, $r \le 4 \le k+1$, hence Theorem 6.1 gives the estimate: $\dim (Y) \le \frac{3}{2}g + k - \frac{m}{2}+ 2+ \frac{r}{2} = \frac{3}{2}g + k - m+ \frac{9}{2}$, and we conclude using $k \le [\frac{g+3}{2}].$ $\square$

In conclusion we have the following estimates:

Corollary 6.3

Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,0})$, with $r \le 4$. Assume that there exists a point $[C, \alpha , R] \in Y$ such that C has no involutions.

• (1) If $r=4$, if $g \ge 4$, then $\dim Y \le 2g+4$ if g is even, $\dim Y \le 2g+5$ if g is odd.

• (2) If $r=3$, if $g \ge 8$, then $\dim Y \le 2g+3$ if g is even, $\dim Y \le 2g+4$ if g is odd.

• (3) If $r=2$, if $g \ge 12$, then $\dim Y \le 2g+2$ if g is even, $\dim Y \le 2g+3$ if g is odd.

• (4) If $r=1$, if $g \ge 16$, then $\dim Y \le 2g+1$ if g is even, $\dim Y \le 2g+2$ if g is odd.

• (5) If $r=0$, if $g \ge 20$, then $\dim Y \le 2g$ if g is even, $\dim Y \le 2g+1$ if g is odd.

7 Estimate with one quadric

In the cases in which the assumptions of Corollaries 5.3, 6.2 do not hold, we still have the estimates obtained in [4, 5] using only one quadric, which can be improved by Lemma 3.3 and Corollary 3.4 in the case $r \le k+1$.

In fact, assume $g+r \ge k+3$, so that $h^0(M(-B)) \ge 2$. With the above notation, choose $s \in W$, $t \in H^0(M(-B))$ satisfying conditions (1) and (2) in Sect. 4.1 and such that $s(p_i) \ne 0$, $\forall i = 1,\ldots ,d$. By Theorem 4.1 we know that

$$\begin{aligned} \beta (s)= {\tilde{\rho }}(\pi ^*(Q_s))= \lambda \sum _{i=1}^d a_ix_i^2, \end{aligned}$$

where $a_i = s(p_i) \ne 0$, $\forall i$, hence $\mathrm{{rank}} (\beta (s)) =\mathrm{{rank}}({\tilde{\rho }}(\pi ^*(Q_s)))\ge d = 2g-2+r-k-b.$

Then

Theorem 7.1

Assume that $[(C,\alpha ,R)] \in {{\mathcal {R}}}^0_{g,r}$ where C is a k-gonal curve of genus $g>0$ with $g+r\ge k+3$. Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,r})$ and passes through $P(C,\alpha ,R)$. Then

• (1) If $r > k+1$, then $\dim Y\le 2g-2+\frac{3}{2}r + \frac{k}{2}$.

• (2) If $r \le k+1$, then:

  • If $r \ge 4$, then $\dim Y\le 2g-\frac{3}{2} + \frac{k}{2} + \frac{3}{2} r$.

  • If $r=3,$ then $\dim Y\le 2g+\frac{7}{2} + \frac{k}{2}$.

  • If $r \le 2$, then $\dim Y\le 2g+\frac{1}{2} + \frac{k}{2} + \frac{3}{2}r.$

Proof

Since $\mathrm{{rank}} (\beta (s)) =\mathrm{{rank}}({\tilde{\rho }}(\pi ^*(Q_s)))\ge d = 2g-2+r-k-b,$ we have

$$\begin{aligned} \dim Y\le 3g-3 +2r -\frac{d}{2} = 2g-2 + \frac{3}{2}r + \frac{k}{2} + \frac{b}{2}. \end{aligned}$$

In the first case $b=0$, so we get the same estimate as in [5]. If $r \le k+1$, we use Corollary 3.4 in the case $r\ge 4$, and in the case $r=3$, since we have respectively $b \le 1$, $b \le 2$. If $r \le 2$, by Lemma 3.3 we know that $b \le 5$.

$\square$

So, in the cases in which the assumptions of Corollaries 5.3, 6.2 do not hold, we have the following

Theorem 7.2

Let Y be a germ of a totally geodesic submanifold of ${{\mathcal {A}}}^{\delta }_{g-1+r}$ which is contained in $P_{g,r}( {{\mathcal {R}}}^0_{g,r})$.

• (1) If $g=1$, $r =4$, then $\dim Y\le 7$.

• (2) If $g=2$, $r =4$, then $\dim Y\le 9$.

• (3) If $g=3$, $r =4$, then $\dim Y\le 12$.

• (4) If $2 \le g\le 7$, $r=3$, $\dim Y\le \frac{9}{4} g +4$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{17}{4}$, if g is odd.

• (5) If $4 \le g \le 11$, $r =2$, then $\dim Y\le \frac{9}{4}g+4$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{17}{4}$, if g is odd.

• (6) If $6 \le g \le 15$, $r =1$, then $\dim Y\le \frac{9}{4}g+\frac{5}{2}$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{11}{4}$, if g is odd.

• (7) If $8 \le g \le 19$, $r =0$, then $\dim Y\le \frac{9}{4}g+1$, if g is even, $\dim Y\le \frac{9}{4} g +\frac{5}{4}$, if g is odd.

Proof

By assumption $g + r \ge [\frac{g+3}{2}] + 3 \ge k +3$. Moreover if $r \le 3$, then $r \le k+1$, so we use the estimate (2) in Theorem 7.1 (and the inequality $k\le [\frac{g+3}{2}]$). For $r=4$, if $g \le 2$, $r =4 >k+1 =3$, so we use the estimate (1) in Theorem 7.1 (and the inequality $k\le [\frac{g+3}{2}]$). If $r =4$ and $g =3$, if $k=2$ we use the estimate (1) in Theorem 7.1 and we get $\dim Y \le 11$. If $k =3$, we use the estimate (2) in Theorem 7.1 and we obtain $\dim Y\le 12$. $\square$

Assume now that $r \ge 4$, hence ${{\mathcal {R}}}^0_{g,r} = {{\mathcal {R}}}_{g,r}$, since a global Prym–Torelli theorem holds for $r \ge 3$ [12, 19]. Let $[C, \alpha , R] \in {{\mathcal {R}}}_{g,r}$ be a hyperelliptic curve, so $k=2$. We have the following

Proposition 7.3

Let $[C, \alpha , R] \in {{\mathcal {R}}}_{g,r}$, where C is a hyperelliptic curve of genus g, $r \ge 4$. Denote by Y a totally geodesic subvariety contained in $P_{g,r}({{\mathcal {R}}}_{g,r})$ and passing through $P_{g,r}([C, \alpha , R])$.

• (1) If $r >4$, then $\dim (Y) \le \frac{3}{2}g + \frac{3}{2} + r.$

• (2) If $r=4$ and either $\alpha \otimes F^{-1}$ is not effective, or $\alpha \otimes F^{-1} \cong {{\mathcal {O}}}_C(D)$ with D effective and ${{\mathcal {O}}}_C(D) \not \cong F$, then $\dim (Y) \le \frac{3}{2}g + \frac{11}{2}.$

Proof

Since $r\ge 4 > k+1 =3$, by Proposition 3.1, we have $b=0$ and $h^0(F \otimes \alpha ^{-1}) = 0$. Moreover, if $r > 4= k+2$, |M| is very ample, hence the proof of Theorem 5.1, applies, so we have the same estimate as in Theorem 5.1, (1): $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + r +2 = \frac{3}{2}g + \frac{3}{2} + r.$

If $r=4 =k+2$, by Proposition 3.1, (3), we know that if $\alpha \otimes F^{-1}$ is not effective, then |M| is very ample, hence we have again the estimate in Theorem 5.1, (1), so $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + r +2 = \frac{3}{2}g + \frac{11}{2}.$

Assume $r=4$ and $\alpha \otimes F^{-1} \cong {{\mathcal {O}}}_C(D)$ with D effective. Then if ${{\mathcal {O}}}_C(D) \not \cong F$, by Proposition 3.1, (3), we know that |M| gives a birational map that contracts D, so once again we have the estimate in Theorem 5.1, (1): $\dim (Y) \le \frac{3}{2}g - \frac{1}{2} + 4+2 = \frac{3}{2}g + \frac{11}{2}.$ $\square$