A fixed point theorem for smooth extension maps

Abstract

Let X be a compact smooth n-manifold, with or without boundary, and let A be an $(n-1)$-dimensional smooth submanifold of the interior of X. Let $\varphi :A\to A$ be a smooth map and $f:(X,A)\to (X,A)$ be a smooth map whose restriction to A is ϕ. If $p\in A$ is an isolated fixed point of f that is a transversal fixed point of ϕ, that is, the linear transformation $d{\varphi }_p-I_A:T_{p}A\to T_{p}A$ is nonsingular, then the fixed point index of f at p satisfies the inequality $|i(X,f,p)|\le 1$. It follows that if ϕ has k fixed points, all transverse, and the Lefschetz number $L(f)>k$, then there is at least one fixed point of f in $X\setminus A$. Examples demonstrate that these results do not hold if the maps are not smooth.

MSC:55M20, 54C20.

1 Introduction

It has been known at least since the work of Shub and Sullivan in 1974 [1] that the values of the fixed point index of smooth maps are more restricted than they are for continuous functions in general. In [2] it is proved that, given integers r and s, there is a map $f:X\to X$ of a manifold with boundary ∂X that restricts to $f{|}_{\partial X}=\varphi :\partial X\to \partial X$ and an isolated fixed point p of f such that the fixed point indices are $i(\partial X,\varphi ,p)=r$ and $i(X,f,p)=s$. On the other hand, it is proved in that paper that if $f:(X,\partial X)\to (X,\partial X)$ is smooth and p is a transverse fixed point of ϕ, then either $i(X,f,p)=0$ or $i(X,f,p)=i(\partial X,\varphi ,p)$. A consequence of this result is that, under appropriate hypotheses on a smooth map f, it must have fixed points on $X\setminus \partial X$, the interior of the manifold X. Those same hypotheses are shown to be insufficient to imply the existence of such interior fixed points if the map f is not smooth. (See also [3, 4].)

We will consider a somewhat different setting, as follows. Let X be a compact smooth n-manifold, with or without boundary, and let A be a smooth $(n-1)$-dimensional submanifold of the interior of X. As in [2], we shall consider $f:(X,A)\to (X,A)$ to be an extension of its restriction $f{|}_A=\varphi :A\to A$. Suppose that p is a transverse fixed point of ϕ, then a simple example will show that the relationship between $i(X,f,p)$ and $i(A,\varphi ,p)$ cannot be as close as it is when $A=\partial X$. However, we will prove that there is still a very strong restriction on the value of $i(X,f,p)$, namely, that $|i(f,X,p)|\le 1$. As a consequence, we obtain a condition on the Lefschetz number $L(f)$ of f that implies the existence of fixed points of f in $X\setminus A$. We demonstrate by an example that the same Lefschetz number condition is not sufficient to imply the existence of fixed points in $X\setminus A$ for maps $f:(X,A)\to (X,A)$ in general.

2 The index of fixed points of smooth extension maps

The index theorem of [2] is the following.

Theorem 1 Let X be a compact smooth n-manifold with boundary ∂X. Given a smooth map $\varphi :\partial X\to \partial X$ and a smooth map $f:(X,\partial X)\to (X,\partial X)$ extending ϕ, suppose that $p\in \partial X$ is an isolated fixed point of f and that $d{\varphi }_p-I_{\partial X}:T_p(\partial X)\to T_p(\partial X)$ is a nonsingular linear transformation. Then either $i(X,f,p)=0$ or $i(X,f,p)=i(\partial X,\varphi ,p)$.

We will modify the proof of Theorem 1 that was given in [2] in order to obtain a result of this type in the setting of smooth extension maps on a pair consisting of an n-dimensional compact smooth manifold X and an $(n-1)$-dimensional smooth submanifold A of the interior of X. Although our index result is similar to Theorem 1, the following example demonstrates that we cannot expect that there will be as close a relationship between the indices of ϕ and of f as there is in Theorem 1. Let $f:(S^2,S^1)\to (S^2,S^1)$ where $S^2$ is viewed as the complex plane ℂ compactified at infinity, $S^1$ is the unit circle, $f(z)=z^2$ for $z\in \mathbb{C}$ and $f(\mathrm{\infty })=\mathrm{\infty }$. Let $\varphi :S^1\to S^1$ be the restriction of f. Then $i(S^1,\varphi ,1)=-1$, whereas $i(S^2,f,1)=1$.

Theorem 2 Let X be a smooth n-manifold, with or without boundary, and let A be an $(n-1)$-dimensional smooth submanifold of the interior of X. Let $\varphi :A\to A$ be a smooth map with smooth extension $f:(X,A)\to (X,A)$. Suppose that $p\in A$ is an isolated fixed point of f and that $d{\varphi }_p-I_A:T_{p}A\to T_{p}A$ is a nonsingular linear transformation. Then $|i(X,f,p)|\le 1$.

Proof If the linear transformation $d{f}_p-I:T_{p}X\to T_{p}X$ is nonsingular, then $i(X,f,p)=\pm 1$, see [5]. Therefore, we assume that the linear transformation $d{f}_p-I:T_{p}X\to T_{p}X$ is singular. Note that determining the index of a map f at a fixed point p is a local problem, so we may choose a local coordinate system about p in which the smooth manifold X is identified with ${\mathbb{R}}^n$ such that p is the origin in ${\mathbb{R}}^n$ and the smooth submanifold A is identified with the subspace ${\mathbb{R}}^{n-1}$.

Let us define $G:{\mathbb{R}}^n\to {\mathbb{R}}^n$ by $G(\mathbf{x})=f(\mathbf{x})-\mathbf{x}$. To calculate the index of f at p, we need to determine the degree of −G restricted to a sphere around 0. More specifically, for $ϵ>0$ sufficiently small, we may consider the map $E_{ϵ}:S^{n-1}\to {\mathbb{R}}^n-\{\mathbf{0}\}$ defined by

$$E_{ϵ}(\mathbf{x})=-G(ϵ\mathbf{x}).$$

Since G is a $C^1$ function with value 0 at the point p, we know from the definition of the derivative that

$$|G(ϵ\mathbf{x})-d{G}_p(ϵ\mathbf{x})|\le o(ϵ)$$

uniformly over $\{\mathbf{x}\in A|:|\mathbf{x}|=1\}$. Our goal is to define maps $P_{ϵ}^{\sigma }$ related to $E_{ϵ}$ such that $P_{ϵ}^{\sigma }$ takes each of the upper and lower half-plane into either the upper or the lower half-plane and that will allow us to calculate the index $i(X,f,p)$. However, we need to make use of a change of coordinate. Since $d{f}_p-I=d{G}_p$ is singular but $d{G}_p{|}_A$ is not, we know that $d{G}_p(0,\dots ,0,1)\in span\{x_1,\dots ,x_{n-1}\}$. Now rotate the $x_1,\dots ,x_{n-1}$ coordinate system so that $x_{n-1}$ points in the direction of $d{G}_p(0,\dots ,0,1)$. Choose new coordinates $y_1,\dots ,y_n$ so that the hyperplane $y_n=0$ is the same as $x_n=0$ and, for each $j=1,\dots ,n-1$, the transformation $d{G}_p$ takes the $x_j$ unit vector to the $y_i$ unit vector. Now we have

$$\begin{array}{c}\{(x_1,\dots ,x_{n-1},0)\in {\mathbb{R}}^n\}=\{(y_1,\dots ,y_{n-1},0)\in {\mathbb{R}}^n\},\hfill \\ \{(x_1,\dots ,x_{n-1},x_n)\in {\mathbb{R}}^n\}=\{(y_1,\dots ,y_{n-1},y_n)\in {\mathbb{R}}^n\}\hfill \end{array}$$

and the positive $y_n$ axis is in the same half-space as the positive $x_n$ axis. With this change of coordinates, $d{G}_p$, as a map from x coordinates into y coordinates, has the following form:

$$d{G}_p({\alpha }_1,\dots ,{\alpha }_{n-1},{\alpha }_n)=({\alpha }_1,\dots ,{\alpha }_{n-1},0)+{\alpha }_n(0,\dots ,0,B,0)$$

for some constant B. Define

$$S_{+}=\{(x_1,\dots ,x_n)\in {\mathbb{R}}^n:x_n\ge 0\}\cap S^{n-1},S_{-}=\{(x_1,\dots ,x_n)\in {\mathbb{R}}^n:x_n\le 0\}\cap S^{n-1}.$$

It is easy to check the following:

• (i) The point

  $${\mathbf{x}}^{+}=(0,\dots ,0,-\frac{B}{{(1+B^2)}^{\frac{1}{2}}},\frac{1}{{(1+B^2)}^{\frac{1}{2}}})$$

  is the unique point of $S_{+}$ with the property that $d{G}_p({\mathbf{x}}^{+})=\mathbf{0}$. From the inverse function theorem, there is a small neighborhood $U^{+}$ of the point ${\mathbf{x}}^{+}$ such that $d{G}_p{|}_{S_{+}}$ is a nonsingular diffeomorphism of $U^{+}$ onto a neighborhood of 0 in the hyperplane $y_n=0$ and ${(d{G}_p{|}_{S_{+}})}^{-1}(d{G}_p(U^{+}))=U^{+}$.

• (ii) The point

  $${\mathbf{x}}^{-}=(0,\dots ,0,-\frac{B}{{(1+B^2)}^{\frac{1}{2}}},-\frac{1}{{(1+B^2)}^{\frac{1}{2}}})$$

is the unique point of $S_{-}$ with the property that $d{G}_p({\mathbf{x}}^{-})=\mathbf{0}$. Similarly, from the inverse function theorem, there is a small neighborhood $U^{-}$ of the point ${\mathbf{x}}^{-}$ such that $d{G}_p{|}_{S_{-}}$ is a nonsingular diffeomorphism of $U^{-}$ onto a neighborhood of 0 in the hyperplane $y_n=0$ and ${(d{G}_p{|}_{S_{-}})}^{-1}(d{G}_p(U^{-}))=U^{-}$.

Now consider the maps $P_{ϵ}$ defined by

$$P_{ϵ}(\mathbf{x})={ϵ}^{-1}{E}_{ϵ}(\mathbf{x}).$$

The map $P_{ϵ}{|}_{S^{n-1}}$ converges in the $C^1$ topology to $-d{G}_p{|}_{S^{n-1}}$ because

$$\begin{array}{rcl}d(P_{ϵ}(\mathbf{x}))& =& d({ϵ}^{-1}{E}_{ϵ}(\mathbf{x}))\\ =& d\left({ϵ}^{-1}(-G(ϵ\mathbf{x}))\right)\\ =& -{ϵ}^{-1}d(G(ϵ\mathbf{x}))\\ =& -{ϵ}^{-1}d{G}_p(ϵ\mathbf{x})ϵ\\ =& -d{G}_p(ϵ\mathbf{x}).\end{array}$$

Consequently, we have the following analogues of (i)-(ii) above.

• (1) There is a unique point ${\mathbf{x}}_{ϵ}^{+}\in S_{+}$ such that the $y_1,\dots ,y_{n-1}$ coordinates of $P_{ϵ}({\mathbf{x}}_{ϵ}^{+})$ are all 0. Also, since p is an isolated fixed point of $f,P_{ϵ}({\mathbf{x}}_{ϵ}^{+})\ne \mathbf{0}$. In particular, the $y_n$ coordinate of $P_{ϵ}({\mathbf{x}}_{ϵ}^{+})$ is nonzero. Hence we have

  $$P_{ϵ}\left({\mathbf{x}}_{ϵ}^{+}\right)=(0,\dots ,0,y_n^{{ϵ}^{+}}),$$

  where $y_n^{{ϵ}^{+}}\ne 0$.

• (2) There is a unique point ${\mathbf{x}}_{ϵ}^{-}\in S_{-}$ such that the $y_1,\dots ,y_{n-1}$ coordinates of $P_{ϵ}({\mathbf{x}}_{ϵ}^{-})$ are all 0. Also, since p is an isolated fixed point of $f,P_{ϵ}({\mathbf{x}}_{ϵ}^{-})\ne \mathbf{0}$. In particular, the $y_n$ coordinate of $P_{ϵ}({\mathbf{x}}_{ϵ}^{-})$ is nonzero. Hence we have

  $$P_{ϵ}\left({\mathbf{x}}_{ϵ}^{-}\right)=(0,\dots ,0,y_n^{{ϵ}^{-}}),$$

where $y_n^{{ϵ}^{-}}\ne 0$.

If $P_{ϵ}(\mathbf{x})=(y_1,y_2,\dots ,y_n)$, define a map $P_{ϵ}^{\sigma }:S^{n-1}\to {\mathbb{R}}^n-\{\mathbf{0}\}$ by

$$P_{ϵ}^{\sigma }=\{\begin{array}{ll}(y_1,\dots ,y_{n-1},{\sigma }^{+}|y_n|)& \text{if }\mathbf{x}\in S_{+},\\ (y_1,\dots ,y_{n-1},{\sigma }^{-}|y_n|)& \text{if }\mathbf{x}\in S_{-}\mathrm{\setminus }{\mathbb{R}}^{n-1},\end{array}$$

where ${\sigma }^{+}=y_n^{{ϵ}^{+}}/|y_n^{{ϵ}^{+}}|=\pm 1$ and ${\sigma }^{-}=y_n^{{ϵ}^{-}}/|y_n^{{ϵ}^{-}}|=\pm 1$.

Notice that the values of $P_{ϵ}^{\sigma }{|}_{S_{+}}$ lie entirely in the half-space where $P_{ϵ}({\mathbf{x}}_{ϵ}^{+})$ is located with respect to the $y_1,\dots ,y_n$ coordinates and the values of $P_{ϵ}^{\sigma }{|}_{S_{-}}$ lie entirely in the half-space where $P_{ϵ}({\mathbf{x}}_{ϵ}^{-})$ is located with respect to the $y_1,\dots ,y_n$ coordinates. Furthermore, $P_{ϵ}$ and $P_{ϵ}^{\sigma }$ are homotopic as maps into ${\mathbb{R}}^n-\{\mathbf{0}\}$ by the following homotopy:

$$H(\mathbf{x},t)=t{P}_{ϵ}(\mathbf{x})+(1-t)P_{ϵ}^{\sigma }(\mathbf{x}).$$

For $\mathbf{x}\in S_{+}$, either $(y_1,\dots ,y_{n-1})\ne \mathbf{0}$ or $\mathbf{x}={\mathbf{x}}_{ϵ}^{+}$. If $(y_1,\dots ,y_{n-1})\ne \mathbf{0}$, then $H(\mathbf{x},t)\ne \mathbf{0}$ by definition. If $\mathbf{x}={\mathbf{x}}_{ϵ}^{+}$, then $H(\mathbf{x},t)=P_{ϵ}(\mathbf{x})$ which we know is never 0. A similar argument shows that $H(\mathbf{x},t)\ne \mathbf{0}$ for $\mathbf{x}\in S_{-}\mathrm{\setminus }{\mathbb{R}}^{n-1}$.

Since the map $P_{ϵ}^{\sigma }$ takes $S_{+}$ into either the upper or lower half-planes with respect to the y coordinates and takes $S_{-}$ into either the upper or lower half-planes with respect to the $y_1,\dots ,y_n$ coordinates, the map $P_{ϵ}^{\sigma }$ is homotopic either to a constant map, the suspension of $P_{ϵ}^{\sigma }{|}_{S^{n-2}}=P_{ϵ}{|}_{S^{n-2}}$ or the suspension of $P_{ϵ}^{\sigma }{|}_{S^{n-2}}=P_{ϵ}{|}_{S^{n-2}}$ followed by a reflection about the hyperplane $y_n=0$. This homotopy tells us that either $P_{ϵ}$ has degree 0 or $deg(P_{ϵ}{|}_{S^{n-2}})$ or $-deg(P_{ϵ}{|}_{S^{n-2}})$.

Although $P_{ϵ}$ is a map from x to y coordinates, we know that the x and y coordinates are related by a linear map, call it L, satisfying $L(x_i)=y_i$, for $i=1,\dots ,n$. Thus, the map $L^{-1}\circ P_{ϵ}$ takes $x_1,\dots ,x_n$ coordinates to $x_1,\dots ,x_n$ coordinates and

$$deg(L^{-1}\circ P_{ϵ})=deg\left(L^{-1}\right)\cdot deg(P_{ϵ})=deg(P_{ϵ}).$$

Thus the index $i(X,f,p)=deg(P_{ϵ})$ which is either 0 or ±1. □

3 Fixed point theorem for smooth extension maps

We can now use Theorem 2 to establish the existence of fixed points in $X\setminus A$.

Theorem 3 Suppose that A is an $(n-1)$-dimensional smooth submanifold of the interior of a compact smooth n-manifold X. Given a smooth map $\varphi :A\to A$ and a smooth map $f:(X,A)\to (X,A)$ extending ϕ, suppose that the fixed points of ϕ are $\{x_1,x_2,\dots ,x_k\}$, all of which are transversal, that is, the linear map $d{\varphi }_{x_j}-I_A$ is nonsingular for each $x_j$. If the Lefschetz number $L(f)>k$, then there must be at least one fixed point in $X\setminus A$.

Proof Suppose that f only has fixed points in A. This means the fixed point set of f is $\{x_1,\dots ,x_k\}$, the set of fixed points of ϕ. Then, by the Lefschetz-Hopf theorem [6], the Lefschetz number of f is

$$L(f)=\sum _{j=0}^{k}i(X,f,x_j)\le k$$

because $i(X,f,x_j)\le 1$ by Theorem 2. This is contrary to the assumption that $L(f)>k$, so f has fixed points in $X\setminus A$. □

A consequence of this theorem is the following.

Corollary 4 Let $S^2$ be the complex plane ℂ compactified at infinity and $S^1$ be the unit circle. Suppose that $\varphi :S^1\to S^1$ is a smooth map defined by $\varphi (\zeta )={\zeta }^k$ for some $k\ge 2$ and that $f:(S^2,S^1)\to (S^2,S^1)$ is a smooth extension of ϕ. If f is homotopic to the suspension of ϕ, then there is at least one fixed point in $S^2\mathrm{\setminus }{S}^1$.

Proof Since f is homotopic to the suspension of ϕ, then f is of degree k and thus $L(f)=1+k$. However, ϕ has only $k-1$ fixed points. Note here that the $k-1$ fixed points of ϕ on $S^1$ are all transversal so that the hypotheses of Theorem 3 hold. □

The following example illustrates the fact that the corollary, and therefore Theorem 3, require the hypothesis that the map f is smooth, by exhibiting a non-smooth map $f:(S^2,S^1)\to (S^2,S^1)$ homotopic to the suspension of ϕ that has no fixed points on $S^2\mathrm{\setminus }{S}^1$.

Example 1 Let $S^2=\mathbb{C}\cup \{\mathrm{\infty }\}$ be the complex plane ℂ compactified at infinity and $S^1$ be the unit circle. Let $\varphi :S^1\to S^1$ be defined by $\varphi (\zeta )={\zeta }^2$. Let $B_{+}^2=\{z\in \mathbb{C}:|z|\le 1\}$ and $B_{-}^2=\{z\in \mathbb{C}:|z|\ge 1\}\cup \{\mathrm{\infty }\}$, so $S^2=B_{+}^2\cup B_{-}^2$ and $B_{+}^2\cap B_{-}^2=S^1$. For $z\in B_{-}^2\setminus \{1\}$, there exist $\zeta \in S^1$ and $t\in [0,1)$ such that $z=t1+(1-t)\zeta$. As in Lemma 6.1 of [7], define $f:B_{+}^2\to B_{+}^2$ by setting $f(z)=t1+(1-t)\varphi (\zeta )$ for $z\ne 1$ and $f(1)=1$, then f has no fixed points in $B_{+}^2\setminus S^1$. In order to extend f to a selfmap of $B_{-}^2$ so that there are no fixed points on $B_{-}^2\setminus S^1$, we define $\rho :S^2\to S^2$ by $\rho (z)=\rho (r{e}^{i\theta })=\frac{1}{r}{e}^{i\theta }$, $\rho (0)=\mathrm{\infty }$ and $\rho (\mathrm{\infty })=0$, so $\rho (B_{-}^2)=B_{+}^2$ and $\rho (B_{+}^2)=B_{-}^2$. Then, for $z\in B_{-}^2$, let $f(z)=\rho f\rho (z)$. The only fixed point of f is $z=1$ and thus there are no fixed points in $S^2\setminus S^1$.