Diagonalizations of two classes of unbounded Hankel operators

Abstract

We show that every Hankel operator $H$ is unitarily equivalent to a pseudo-differential operator $A$ of a special structure acting in the space $L^2(\mathbb{R})$. As an example, we consider integral operators $H$ in the space $L^2({\mathbb{R}}_{+})$ with kernels $P(ln(t+s)){(t+s)}^{-1}$ where $P(x)$ is an arbitrary real polynomial of degree $K$. In this case, $A$ is a differential operator of the same order $K$. This allows us to study spectral properties of Hankel operators $H$ with such kernels. In particular, we show that the essential spectrum of $H$ coincides with the whole axis for $K$ odd, and it coincides with the positive half-axis for $K$ even. In the latter case we additionally find necessary and sufficient conditions for the positivity of $H$. We also consider Hankel operators whose kernels have a strong singularity at some positive point. We show that spectra of such operators consist of the zero eigenvalue of infinite multiplicity and eigenvalues accumulating to $+\infty$ and $-\infty$. We find the asymptotics of these eigenvalues.

1 Introduction

Hankel operators can be defined as integral operators

$$\begin{array}{c}\hfill (Hf)(t)={\int }_0^{\infty }h(t+s)f(s)ds\end{array}$$

(1.1)

in the space $L^2({\mathbb{R}}_{+})$ with kernels $h$ that depend on the sum of variables only. We refer to the books [4, 5] for basic information on Hankel operators. Of course $H$ is symmetric if $h(t)=\overline{h(t)}$. There are very few cases when Hankel operators can be explicitly diagonalized. The most simple and important case $h(t)=t^{-1}$ was considered by T. Carleman in [2].

Here we study a class of Hankel operators generalizing the Carleman operator. The corresponding kernels are given by the formula

$$\begin{array}{c}\hfill h(t)=P(lnt)t^{-1}\end{array}$$

(1.2)

where $P(x)$ is an arbitrary polynomial. Hankel operators $H$ with such kernels are not bounded unless $P(x)=\mathrm{const}$, but, for real $P(x)$, they can be uniquely defined as self-adjoint operators. We show that the Hankel operator with kernel (1.2) is unitarily equivalent to the differential operator

$$\begin{array}{c}\hfill A=vQ(D)v,D=id/d\mathit{\xi },\end{array}$$

(1.3)

in the space $L^2(\mathbb{R})$. Here $v$ is the operator of multiplication by the universal function

$$\begin{array}{c}\hfill v(\mathit{\xi })=\frac{\sqrt{\mathit{\pi }}}{\sqrt{cosh(\mathit{\pi }\mathit{\xi })}}\end{array}$$

(1.4)

and the polynomial $Q(x)$ is determined by $P(x)$. The polynomials $P(x)$ and $Q(x)$ have the same degree, and their coefficients are linked by an explicit formula (see subs. 3.2). In particular, $Q(x)=1$ if $P(x)=1$ which yields the familiar diagonalization of the Carleman operator.

Thus the spectral analysis of Hankel operators with kernels (1.2) reduces to the spectral analysis of differential operators which in principle is very well developed. However operators (1.3) are somewhat unusual because the function $v(\mathit{\xi })$ tends to zero exponentially as $|\mathit{\xi }|\to \infty$ so that there is a strong degeneracy at infinity. Nevertheless we describe completely the essential spectrum of differential operators (1.3) under rather general assumptions on the function $v(\mathit{\xi })$. We show that ${\mathrm{spec}}_{\mathrm{ess}}(A)=\mathbb{R}$ if $K:=degP$ is odd, and ${\mathrm{spec}}_{\mathrm{ess}}(A)=[0,\infty )$ if $K$ is even. Moreover, it turns out that zero is never an eigenvalue of $A$. In the case of even $K$ we also find necessary and sufficient conditions for the positivity of $A$ and for the infinitude of its negative spectrum. Of course the same spectral results are true for Hankel operators $H$ with kernels (1.2). For real polynomials $P(x)$ of first order, our approach yields the explicit diagonalization of Hankel operators $H$. In particular, we show that in this case the spectrum of $H$ is absolutely continuous, has multiplicity $1$ and covers the whole real line.

Actually, the unitary equivalence of the operators $H$ and $A$ is quite explicit. Let $M:L^2({\mathbb{R}}_{+})\to L^2(\mathbb{R})$ be the Mellin transform; it is a unitary mapping. Set

$$\begin{array}{c}\hfill (Ff)(\mathit{\xi })=\frac{\mathrm{\Gamma }(1/2+i\mathit{\xi })}{|\mathrm{\Gamma }(1/2+i\mathit{\xi })|}(Mf)(\mathit{\xi })\end{array}$$

(1.5)

where $\mathrm{\Gamma }(·)$ is the gamma function. We show that

$$\begin{array}{c}\hfill H=F^{\ast }AF.\end{array}$$

(1.6)

Our proof of this identity follows the approach of [6] where general Hankel operators $H$ were considered. For an arbitrary $H$, the function $Q(x)$ in formula (1.3) is a distribution which may be (for example, for finite rank $H$) very singular. However it is a polynomial for Hankel operators with kernels (1.2) so that $A$ is the explicit differential operator in this case.

1.2. Kernels (1.2) are singular at the points $t=0$ and $t=\infty$. We also consider another class of kernels which are singular at some point $t_0>0$. We assume that

$$\begin{array}{c}\hfill h(t)=\sum _{k=0}^K{h}_k{\mathit{\delta }}^{(k)}(t-t_0),h_k={\overline{h}}_k,h_K\ne 0,\end{array}$$

(1.7)

where $\mathit{\delta }(·)$ is the Dirac delta function. It turns out that Hankel operators with such kernels reduce to “differential” operators with the reflection and a shift of the argument.

Spectral properties of Hankel operators with kernels (1.2) and (1.7) are completely different. As discussed in [6], Hankel operators can be sign-definite only for $h\in C^{\infty }({\mathbb{R}}_{+})$ which is of course not true for kernels (1.7). If $K=0$, then, as shown in [6], the spectrum of $H$ consists of three eigenvalues $0$, $h_0$ and $-h_0$ of infinite multiplicity each. We shall prove here that for $K\ge 1$ the spectrum of the operator $H$ with kernel (1.7) consists of the zero eigenvalue of infinite multiplicity and an infinite number of eigenvalues of finite multiplicities accumulating both at $+\infty$ and $-\infty$. Moreover, we shall find the leading term of asymptotics of these eigenvalues.

Recall that a symbol of a Hankel operator with kernel $h(t)$ is defined as a function $\mathit{\omega }(\mathit{\lambda })$, $\mathit{\lambda }\in \mathbb{R}$, such that ${(2\mathit{\pi })}^{-1/2}(\mathrm{\Phi }\mathit{\omega })(t)=h(t)$ where $\mathrm{\Phi }$ is the Fourier transform. Since by the Nehari theorem, Hankel operators with bounded symbols are bounded, the symbols of operators with kernels (1.1) and (1.7) are necessarily unbounded functions. For kernels (1.1) symbols can be constructed by the formula

$$\begin{array}{c}\hfill \mathit{\omega }(\mathit{\lambda })=2i{\int }_0^{\infty }sin(\mathit{\lambda }t)h(t)dt=2i{\int }_0^{\infty }sintP(lnt-ln\mathit{\lambda })t^{-1}dt\end{array}$$

for $\mathit{\lambda }>0$ and $\mathit{\omega }(-\mathit{\lambda })=-\mathit{\omega }(\mathit{\lambda })$. Thus $\mathit{\omega }(\mathit{\lambda })$ is a $C^{\infty }$ function for $\mathit{\lambda }\ne 0$ with logarithmic singularities at $\mathit{\lambda }=0$ and $\mathit{\lambda }=\infty$. For kernels (1.7), the symbol equals

$$\begin{array}{c}\hfill \mathit{\omega }(\mathit{\lambda })=\sum _{k=0}^K{h}_k{(-i\mathit{\lambda })}^k{e}^{i\mathit{\lambda }{t}_0},\end{array}$$

so that it is a $C^{\infty }$ function with a power growth and an oscillation as $|\mathit{\lambda }|\to \infty$.

1.3. Let us introduce some standard notation. We denote by $\mathrm{\Phi }$,

$$\begin{array}{c}\hfill (\mathrm{\Phi }u)(\mathit{\xi })={(2\mathit{\pi })}^{-1/2}{\int }_{-\infty }^{\infty }u(x)e^{-ix\mathit{\xi }}dx,\end{array}$$

the Fourier transform. The space $\mathcal{Z}=\mathcal{Z}(\mathbb{R})$ of test functions is defined as the subset of the Schwartz space $\mathcal{S}=\mathcal{S}(\mathbb{R})$ which consists of functions $u$ admitting the analytic continuation to entire functions in the complex plane $\mathbb{C}$ and satisfying bounds

$$\begin{array}{c}\hfill |u(z)|\le C_n{(1+|z|)}^{-n}{e}^{r|\mathrm{Im}z|},z\in \mathbb{C},\end{array}$$

for some $r=r(u)>0$ and all $n$. We recall (see, e.g., the book [3]) that the Fourier transform $\mathrm{\Phi }:\mathcal{Z}\to C_0^{\infty }(\mathbb{R})$ and ${\mathrm{\Phi }}^{\ast }:C_0^{\infty }(\mathbb{R})\to \mathcal{Z}$. The dual classes of distributions (continuous antilinear functionals on $\mathcal{S}$, $C_0^{\infty }(\mathbb{R})$ and $\mathcal{Z}$) are denoted ${\mathcal{S}}^{\prime }$, $C_0^{\infty }{(\mathbb{R})}^{\prime }$ and ${\mathcal{Z}}^{\prime }$, respectively. We use the notation $\mathbf{〈}·,·\mathbf{〉}$ and $〈·,·〉$ for the duality symbols in $L^2({\mathbb{R}}_{+})$ and $L^2(\mathbb{R})$, respectively. They are linear in the first argument and antilinear in the second argument.

We denote by ${\mathsf{H}}^K(\mathcal{J})$ the Sobolev space of functions defined on an interval $\mathcal{J}\subset \mathbb{R}$; $C_0^K(\mathcal{J})$ is the class of $k$-times continuously differentiable functions with compact supports in $\mathcal{J}$. We often use the same notation for a function and the operator of multiplication by this function. The letters $c$ and $C$ (sometimes with indices) denote various positive constants whose precise values are inessential.

Let us briefly describe the structure of the paper. We collect necessary results of [6] in Sect. 2. In Sect. 3 we establish the unitary equivalence of the Hankel operator $H$ with kernel (1.2) and differential operator (1.3). Spectral properties of the operators $A$ and hence of $H$ are studied in Sect. (4). We emphasize that our results on the operator $A$ do not require specific assumption (1.4). Finally, Hankel operators $H$ with kernels (1.7) are studied in Sect. 5. Our presentation in this section is independent of general results of Sect. 2. On the other hand, it is rather similar to that in Section 6 of [6] where Hankel operators with discontinuous kernels (but not as singular as kernels (1.7) were considered.

2 Hankel and pseudo-differential operators

In this section we show that an arbitrary Hankel operator $H$ is unitarily equivalent to a pseudo-differential operator $A$ defined by formula (1.3) with a distribution $Q(x)$. Our presentation is close to [6], but we here insist upon the unitary equivalence of the operators $H$ and $A$.

2.1. Let us consider a Hankel operator $H$ defined by equality (1.1) in the space $L^2({\mathbb{R}}_{+})$. Actually, it is more convenient to work with sesquilinear forms instead of operators. Let us introduce the Laplace convolution

$$\begin{array}{c}\hfill ({\overline{f}}_1\star f_2)(t)={\int }_0^t\overline{f_1(s)}{f}_2(t-s)ds\end{array}$$

of functions ${\overline{f}}_1$ and $f_2$. Then

$$\begin{array}{c}\hfill (H{f}_1,f_2)=\mathbf{〈}h,{\overline{f}}_1\star f_2\mathbf{〉}\end{array}$$

(2.1)

where we write $\mathbf{〈}·,·\mathbf{〉}$ instead of $(·,·)$ because $h$ may be a distribution.

We consider form (2.1) on elements $f_1,f_2\in \mathcal{D}$ where the set $\mathcal{D}$ is defined as follows. Put

$$\begin{array}{c}\hfill (Uf)(x)=e^{x/2}f(e^x).\end{array}$$

Then $U:L^2({\mathbb{R}}_{+})\to L^2(\mathbb{R})$ is the unitary operator, and $f\in \mathcal{D}$ if and only if $Uf\in \mathcal{Z}$. Since $f(t)=t^{-1/2}(Uf)(lnt)$ and $\mathcal{Z}\subset \mathcal{S}$, we see that functions $f\in \mathcal{D}$ and their derivatives satisfy the estimates

$$\begin{array}{c}\hfill |f^{(m)}(t)|=C_{n,m}{t}^{-1/2-m}{(1+|lnt|)}^{-n}\end{array}$$

(2.2)

for all $n$ and $m$. Of course, the set $\mathcal{D}$ is dense in the space $L^2({\mathbb{R}}_{+})$. It is shown in [6] that if $f_1,f_2\in \mathcal{D}$, then the function

$$\begin{array}{c}\hfill \mathrm{\Omega }(x)=({\overline{f}}_1\star f_2)(e^x)\end{array}$$

belongs to the space $\mathcal{Z}$.

With respect to $h$, we assume that the distribution

$$\begin{array}{c}\hfill \mathit{\theta }(x)=e^{x}h(e^x)\end{array}$$

(2.3)

is an element of the space ${\mathcal{Z}}^{\prime }$. The set of all such $h$ will be denoted ${\mathcal{Z}}_{+}^{\prime }$, that is,

$$\begin{array}{c}\hfill h\in {\mathcal{Z}}_{+}^{\prime }⟺\mathit{\theta }\in {\mathcal{Z}}^{\prime }.\end{array}$$

It is shown in [6] that this condition is satisfied for all bounded Hankel operators$H$. Since $\mathrm{\Omega }\in \mathcal{Z}$, the form

$$\begin{array}{c}\hfill \mathbf{〈}h,{\overline{f}}_1\star f_2\mathbf{〉}={\int }_0^{\infty }h(t)(f_1\star {\overline{f}}_2)(t)dt={\int }_{-\infty }^{\infty }\mathit{\theta }(x)\overline{\mathrm{\Omega }(x)}dx=:〈\mathit{\theta },\mathrm{\Omega }〉\end{array}$$

is correctly defined for all $f_1,f_2\in \mathcal{D}$.

Note that $h\in {\mathcal{Z}}_{+}^{\prime }$ if $h\in L_{\mathrm{loc}}^1({\mathbb{R}}_{+})$ and the integral

$$\begin{array}{c}\hfill {\int }_0^{\infty }|h(t)|(1+{|lnt|)}^{-\mathit{\kappa }}dt<\infty \end{array}$$

(2.4)

converges for some $\mathit{\kappa }$. In this case the corresponding function (2.3) satisfies the condition

$$\begin{array}{c}\hfill {\int }_{-\infty }^{\infty }|\mathit{\theta }(x)|(1+{|x|)}^{-\mathit{\kappa }}dx<\infty ,\end{array}$$

and hence $\mathit{\theta }\in {\mathcal{S}}^{\prime }\subset {\mathcal{Z}}^{\prime }$.

2.2. Let us now give the definitions of the $b$- and $s$-functions of a Hankel operator $H$. We formally define

$$\begin{array}{c}\hfill b(\mathit{\xi })=\frac{1}{2\mathit{\pi }}\frac{{\int }_0^{\infty }h(t)t^{-i\mathit{\xi }}dt}{{\int }_0^{\infty }{e}^{-t}{t}^{-i\mathit{\xi }}dt}.\end{array}$$

(2.5)

Of course $b(-\mathit{\xi })=\overline{b(\mathit{\xi })}$ if $h(t)=\overline{h(t)}$. We call $b(\mathit{\xi })$ the $b$-function of a Hankel operator $H$. Formula (2.5) can be rewritten as

$$\begin{array}{c}\hfill b(\mathit{\xi })={(2\mathit{\pi })}^{-1/2}a(\mathit{\xi })\mathrm{\Gamma }{(1-i\mathit{\xi })}^{-1}\end{array}$$

(2.6)

where

$$\begin{array}{c}\hfill a(\mathit{\xi })=(\mathrm{\Phi }\mathit{\theta })(\mathit{\xi })={(2\mathit{\pi })}^{-1/2}{\int }_0^{\infty }h(t)t^{-i\mathit{\xi }}dt\end{array}$$

(2.7)

is the Fourier transform of function (2.3).

Recall that the gamma function $\mathrm{\Gamma }(z)$ is a holomorphic function in the right half-plane and $\mathrm{\Gamma }(z)\ne 0$ for all $z\in \mathbb{C}$. According to the Stirling formula the gamma function $\mathrm{\Gamma }(z)$ tends to zero exponentially as $|z|\to \infty$ parallel with the imaginary axis. To be more precise, we have

$$\begin{array}{c}\hfill \mathrm{\Gamma }(\mathit{\alpha }+i\mathit{\xi })=e^{\mathit{\pi }i(2\mathit{\alpha }-1)/4}{(2\mathit{\pi }/e)}^{1/2}{\mathit{\xi }}^{\mathit{\alpha }-1/2}{e}^{i\mathit{\xi }(ln\mathit{\xi }-1)}{e}^{-\mathit{\pi }\mathit{\xi }/2}(1+O({\mathit{\xi }}^{-1}))\end{array}$$

for a fixed $\mathit{\alpha }>0$ and $\mathit{\xi }\to +\infty$. We also note that $\mathrm{\Gamma }(\mathit{\alpha }-i\mathit{\xi })=\overline{\mathrm{\Gamma }(\mathit{\alpha }+i\mathit{\xi })}$ and

$$\begin{array}{c}\hfill {|\mathrm{\Gamma }(1/2+i\mathit{\xi })|}^2=\frac{\mathit{\pi }}{cosh(\mathit{\pi }\mathit{\xi })}.\end{array}$$

Since the denominator in (2.6) tends to zero exponentially as $|\mathit{\xi }|\to \infty$, $b(\mathit{\xi })$ is a “nice” function only under very stringent assumptions on $a(\mathit{\xi })$ and hence on $h(t)$. Therefore we are obliged to work with distributions which turn out to be very convenient. The Schwartz class is too restrictive for our purposes because of the exponential decay of $\mathrm{\Gamma }(1-i\mathit{\xi })$. Therefore we assume that $a\in C_0^{\infty }{(\mathbb{R})}^{\prime }$; in this case $b(\mathit{\xi })$ belongs to the same class. Our assumption on $a$ means that $\mathit{\theta }\in {\mathcal{Z}}^{\prime }$ or equivalently $h\in {\mathcal{Z}}_{+}^{\prime }$.

Thus we are led to the following

Definition 2.1

Let $h\in {\mathcal{Z}}_{+}^{\prime }$. The distribution $b\in C_0^{\infty }{(\mathbb{R})}^{\prime }$ defined by formulas (2.3), (2.6) and (2.7) is called the $b$-function of the Hankel operator $H$ (or of its kernel $h(t)$). Its Fourier transform $s=\sqrt{2\mathit{\pi }}{\mathrm{\Phi }}^{\ast }b\in {\mathcal{Z}}^{\prime }$ is called the $s$-function or the sign-function of $H$.

Let the unitary mapping $F:L^2({\mathbb{R}}_{+})\to L^2(\mathbb{R})$ be defined by formula (1.5) where $M=\mathrm{\Phi }U$ is the Mellin transform. If $f\in \mathcal{D}$, then $Uf\in \mathcal{Z}$ and hence the function $Ff\in C_0^{\infty }(\mathbb{R})$. We recall that the function $v(\mathit{\xi })$ was defined by formula (1.4) and set $(\mathcal{J}g)(\mathit{\xi })=g(-\mathit{\xi })$.

The following result was obtained in [6].

Theorem 2.2

Suppose that $h\in {\mathcal{Z}}_{+}^{\prime }$, and let $b\in C_0^{\infty }{(\mathbb{R})}^{\prime }$ be the corresponding $b$-function. Let $f_j\in \mathcal{D}$, $j=1,2$. Then $g_j=F{f}_j\in C_0^{\infty }(\mathbb{R})$ and the representation

$$\begin{array}{c}\hfill \mathbf{〈}h,{\overline{f}}_1\star f_2\mathbf{〉}=〈b,(v\mathcal{J}{\overline{g}}_1)\ast (v{g}_2)〉\end{array}$$

(2.8)

holds.

Passing in the right-hand side of (2.8) to the Fourier transforms and using that

$$\begin{array}{c}\hfill {\mathrm{\Phi }}^{\ast }((v\mathcal{J}{\overline{g}}_1)\ast (v{g}_2))={(2\mathit{\pi })}^{1/2}\overline{{\mathrm{\Phi }}^{\ast }(v{g}_1)}{\mathrm{\Phi }}^{\ast }(v{g}_2),\end{array}$$

we obtain

Corollary 2.3

Let $s\in {\mathcal{Z}}^{\prime }$ be the sign-function of $h$, and let $u_j={\mathrm{\Phi }}^{\ast }(vF{f}_j)\in \mathcal{Z}$. Then

$$\begin{array}{c}\hfill \mathbf{〈}h,{\overline{f}}_1\star f_2\mathbf{〉}=〈s,{\overline{u}}_1{u}_2〉.\end{array}$$

We note that, formally, the identity (2.8) can be rewritten as relation (1.6) where $A$ is the “integral operator” with kernel $v(\mathit{\xi })b(\mathit{\xi }-\mathit{\eta })v(\mathit{\eta })$. To put it differently,

$$\begin{array}{c}\hfill A=v\mathrm{\Phi }s{\mathrm{\Phi }}^{\ast }v,\end{array}$$

(2.9)

that is, $A$ is the pseudo-differential operator defined by the amplitude $v(\mathit{\xi })s(x)v(\mathit{\eta })$. We emphasize that in general $s(x)$ is a distribution so that formula (2.9) has only a formal meaning. According to relation (1.6) a study of the operator $H$ reduces to that of the operator $A$.

2.3. For an arbitrary distribution $h\in {\mathcal{Z}}_{+}^{\prime }$, we have constructed in the previous subsection its sign-function $s\in {\mathcal{Z}}^{\prime }$. It turns out that, conversely, the kernel $h(t)$ can be recovered from its sign-function $s(x)$. It is convenient to introduce the distribution

$$\begin{array}{c}\hfill h^{♮}(\mathit{\lambda })={\mathit{\lambda }}^{-1}s(-ln\mathit{\lambda }).\end{array}$$

(2.10)

Note that the inclusions $s\in {\mathcal{Z}}^{\prime }$ and $h^{♮}\in {\mathcal{Z}}_{+}^{\prime }$ are equivalent. The proof of the following result can be found in [6].

Theorem 2.4

Let $h\in {\mathcal{Z}}_{+}^{\prime }$, and let $s\in {\mathcal{Z}}^{\prime }$ be the corresponding sign-function $($see Definition (2.1)$)$. Then $h$ can be recovered from function (2.10) by the formula

$$\begin{array}{c}\hfill h(t)={\int }_0^{\infty }{e}^{-t\mathit{\lambda }}{h}^{♮}(\mathit{\lambda })\mathit{\lambda }d\mathit{\lambda }.\end{array}$$

(2.11)

Formula (2.11) is understood of course in the sense of distributions. We emphasize the mappings $h\mapsto h^{♮}$ as well as its inverse $h^{♮}\mapsto h$ are one-to-one continuous mappings of the space ${\mathcal{Z}}_{+}^{\prime }$ onto itself.

3 Quasi-Carleman and differential operators

3.1. Now we are in a position to consider Hankel operators with kernels defined by formula (1.2) where

$$\begin{array}{c}\hfill P(x)=\sum _{k=0}^K{p}_k{x}^k,p_K\ne 0,\end{array}$$

(3.1)

is a polynomial with coefficients $p_k$, $k=0,1,\dots ,K$. It is easy to see that such operators (they will be denoted by $H_0$) are well defined on the set ${\mathcal{D}}_0$ of functions $f(t)$ satisfying estimate (2.2) for $m=0$ and some $n>K+1$. Indeed, by the Schwarz inequality for an arbitrary $\mathit{\epsilon }>0$, we have the estimate

$$\begin{array}{cc}& {\displaystyle {\int }_0^{\infty }dt|{\int }_0^{\infty }ds\frac{{|ln(t+s)|}^k}{t+s}{s}^{-1/2}{(1+|lns|)}^{-n}{|}^2}\hfill \\ & {\displaystyle \le C{\int }_0^{\infty }dt{\int }_0^{\infty }ds\frac{{|ln(t+s)|}^{2k}}{{(t+s)}^2}{(1+|lns|)}^{-2n+1+\mathit{\epsilon }}.}\hfill \end{array}$$

(3.2)

Let us make the change of variables $(t,s)\mapsto (\mathit{\tau },s)=(t+s,s)$ in the right-hand side and integrate first over $\mathit{\tau }\ge s$. Then using inequality

$$\begin{array}{c}\hfill {\int }_s^{\infty }{|ln\mathit{\tau }|}^{2k}{\mathit{\tau }}^{-2}d\mathit{\tau }\le C_1{(1+|lns|)}^{2k}{s}^{-1},\end{array}$$

we see that expression (3.2) is bounded by the integral

$$\begin{array}{c}\hfill C_2{\int }_0^{\infty }{(1+|lns|)}^{-2n+2k+1+\mathit{\epsilon }}{s}^{-1}ds\end{array}$$

which converges if $n>k+1+\mathit{\epsilon }/2$. It follows that $H_{0}f\in L^2(\mathbb{R})$ for $f\in {\mathcal{D}}_0$. Moreover, using the Fubini theorem, we see that $(H_0{f}_1,f_2)=(f_1,H_0{f}_2)$ for $f_1,f_2\in {\mathcal{D}}_0$ if all coefficients $p_k$ are real.

Let us formulate the results obtained.

Lemma 3.1

Let the kernel $h(t)$ of a Hankel operator $H_0$ be given by formulas (1.2) and (3.1). Then $H_0$ is well defined on the set ${\mathcal{D}}_0$, and it is symmetric on ${\mathcal{D}}_0$ if all coefficients $p_k$ are real.

As we shall see below, the operator $H_0$ is essentially self-adjoint (see, e.g., the book [1], for background information on the theory of self-adjoint extensions of symmetric operators). The proof of this result as well as our study of spectral properties of the closure ${\overline{H}}_0=:H$ of $H_0$ rely on the identity (1.6). We emphasize however that the proof of (1.6) does not require the assumption $p_k={\overline{p}}_k$, $k=0,1,\dots ,K$. The symmetricity of $H_0$ on the domain $\mathcal{D}$ is also a consequence of (1.6) so that the direct proof of Lemma (3.1) could be avoided.

3.2. Since kernels (1.2) satisfy condition (2.4) with any $\mathit{\kappa }>K+1$, Theorem 2.2 can be directly applied in this case. We only have to calculate the corresponding $b$- and $s$-functions. If $h_k(t)=t^{-1}{ln}^{k}t$, then the function (2.3) equals ${\mathit{\theta }}_k(x)=x^k$ and its Fourier transform equals

$$\begin{array}{c}\hfill a_k(\mathit{\xi })=(\mathrm{\Phi }{\mathit{\theta }}_k)(\mathit{\xi })={(2\mathit{\pi })}^{1/2}{i}^k{\mathit{\delta }}^{(k)}(\mathit{\xi }).\end{array}$$

To simplify notation, we set$\mathit{\omega }(z)=\mathrm{\Gamma }{(1-z)}^{-1}.$ Then function (2.6) equals

$$\begin{array}{c}\hfill b_k(\mathit{\xi })=i^k\mathit{\omega }(i\mathit{\xi }){\mathit{\delta }}^{(k)}(\mathit{\xi })=\sum _{\ell =0}^k{i}^{\ell }{C}_k^{\ell }{\mathit{\omega }}^{(k-\ell )}(0){\mathit{\delta }}^{(\ell )}(\mathit{\xi })\end{array}$$

where $C_k^{\ell }$ are the binomial coefficients.

It follows that the $b$-function of kernel (1.2), (3.1) is given by the formula

$$\begin{array}{c}\hfill b(\mathit{\xi })=\sum _{k=0}^K{q}_k{i}^k{\mathit{\delta }}^{(k)}(\mathit{\xi })\end{array}$$

where

$$\begin{array}{c}\hfill q_k=\sum _{\ell =k}^K{C}_{\ell }^k{\mathit{\omega }}^{(\ell -k)}(0)p_{\ell },k=0,\dots ,K,\mathit{\omega }(z)=\mathrm{\Gamma }{(1-z)}^{-1}.\end{array}$$

(3.3)

It means that the operator $A$ acts by formula (1.3) where

$$\begin{array}{c}\hfill Q(x)=\sum _{k=0}^K{q}_k{x}^k.\end{array}$$

(3.4)

Thus for kernels (1.2) the sign-function $s(x)=Q(x)$ is the polynomial. Note that according to general formula (2.11), $P(x)$ can be recovered from $Q(x)$ by the equality

$$\begin{array}{c}\hfill P(lnt)=t{\int }_0^{\infty }Q(-ln\mathit{\lambda })e^{-t\mathit{\lambda }}d\mathit{\lambda }.\end{array}$$

(3.5)

Observe that $q_K=p_K$ for all $K$. Recall that ${\mathrm{\Gamma }}^{\prime }(1)=-\mathit{\gamma }$ (the Euler constant) and ${\mathrm{\Gamma }}^{\prime \prime }(1)={\mathit{\gamma }}^2+{\mathit{\pi }}^2/6.$ Therefore we have

$$\begin{array}{c}\hfill q_0=p_0-\mathit{\gamma }{p}_1,\mathrm{if}K=1,\end{array}$$

(3.6)

and

$$\begin{array}{c}\hfill q_0=p_0-\mathit{\gamma }{p}_1+({\mathit{\gamma }}^2-{\mathit{\pi }}^2/6)p_2,q_1=p_1-2\mathit{\gamma }{p}_2,\mathrm{if}K=2.\end{array}$$

(3.7)

The following assertion is a particular case of Theorem 2.2.

Theorem 3.2

Let a kernel $h(t)$ be defined by formulas (1.2) and (3.1). Let $Q(x)$ be polynomial (3.4) with coefficients (3.3), and let $A$ be differential operator (1.3). Then for all $f_j\in \mathcal{D}$, $j=1,2$, the identity

$$\begin{array}{c}\hfill (H{f}_1,f_2)=(AF{f}_1,F{f}_2)\end{array}$$

(3.8)

holds.

Note that, for $h(t)=t^{-1}$, the identity (3.8) yields the familiar diagonalization of the Carleman operator. Indeed, in this case we have

$$\begin{array}{c}\hfill \mathit{\theta }(x)=1,a(\mathit{\xi })={(2\mathit{\pi })}^{1/2}\mathit{\delta }(\mathit{\xi }),b(\mathit{\xi })=\mathit{\delta }(\mathit{\xi }),s(x)=1.\end{array}$$

Therefore the identity (3.8) reads as

$$\begin{array}{c}\hfill (H{f}_1,f_2)={\int }_{-\infty }^{\infty }\frac{\mathit{\pi }}{cosh(\mathit{\pi }\mathit{\xi })}{\tilde{f}}_1(\mathit{\xi })\overline{{\tilde{f}}_2(\mathit{\xi })}d\mathit{\xi }\end{array}$$

where ${\tilde{f}}_j=M{f}_j=\mathrm{\Phi }U{f}_j$, $j=1,2$, is the Mellin transform of $f_j$.

We emphasize that Theorem 3.2 does not require that the coefficients of $P(x)$ be real.

3.3. In view of Theorem 3.2 spectral properties of the Hankel operator $H$ are the same as those of the differential operator $A$. Therefore we forget for a while Hankel operators and study differential operators $A$ defined by formula (1.3), but we not assume that the function $v(\mathit{\xi })$ has special form (1.4). We suppose that$v=\overline{v}\in C^K(\mathbb{R})$and that the coefficients of the polynomial$Q(x)$of degree$K$are real and$q_K\ne 0$. Then the operator $A_0$ defined by formula (1.3) on the domain $C_0^K(\mathbb{R})$ is symmetric in $L^2(\mathbb{R})$. We emphasize that operators (1.3) require a special study because the function $v(\mathit{\xi })$ may tend to zero as $|\mathit{\xi }|\to \infty$.

Let us start with the case $K=1$ when $Q(x)=q_0+q_{1}x$ and $A_0$ can be standardly reduced by a change of variables and a gauge transformation to the differential operator $q_{1}D$. We suppose that $v(\mathit{\xi })>0$ and

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}_{+}}v{(\mathit{\xi })}^{-2}d\mathit{\xi }={\int }_{{\mathbb{R}}_{-}}v{(\mathit{\xi })}^{-2}d\mathit{\xi }=\infty .\end{array}$$

(3.9)

Under this assumption the operator $T:L^2(\mathbb{R})\to L^2(\mathbb{R})$ defined by the relation

$$\begin{array}{c}\hfill (Tg)(\mathit{\xi })=v{(\mathit{\xi })}^{-1}{e}^{i{q}_0{q}_1^{-1}\mathit{\xi }}g\left({\int }_0^{\mathit{\xi }}v{(\mathit{\eta })}^{-2}d\mathit{\eta }\right)\end{array}$$

(3.10)

is unitary and

$$\begin{array}{c}\hfill A_{0}Tg=i{q}_{1}T{g}^{\prime },g\in C_0^1(\mathbb{R}).\end{array}$$

Recall that $D=id/d\mathit{\xi }$. Let the set ${\mathcal{D}}_{\ast }\subset L^2(\mathbb{R})$ consist of functions $g\in {\mathsf{H}}_{\mathrm{loc}}^1(\mathbb{R})$ such that $vQ(D)(vg)\in L^2(\mathbb{R})$. It is easy to see that ${\mathcal{D}}_{\ast }=T{\mathsf{H}}^1(\mathbb{R})$. Thus we are led to the following assertion.

Lemma 3.3

Suppose that $v\in C^1(\mathbb{R})$, $v(\mathit{\xi })>0$ and condition (3.9) is satisfied. Then the operator $A_0=v(q_0+q_{1}D)v$ is essentially self-adjoint on $C_0^1(\mathbb{R})$, and its closure ${\overline{A}}_0=:A$ is self-adjoint on the domain ${\mathcal{D}}_{\ast }=:\mathcal{D}(A)$. The spectrum of the operator $A$ is simple, absolutely continuous, and it coincides with $\mathbb{R}$.

Remark 3.4

If both integrals (3.9) are finite, then $A_0$ reduces to the operator $q_{1}D$ on a finite interval. Its deficiency indices equal $(1,1)$. If only one of integrals (3.9) is finite, then $A_0$ reduces to the operator $q_{1}D$ on a half-axis. Its deficiency indices equal $(0,1)$ or $(1,0)$.

As a by-product of our considerations, we obtain the following result. It is simple but perhaps was never explicitly mentioned.

Proposition 3.5

Suppose that $v\in C^1(\mathbb{R})$ and $v(\mathit{\xi })>0$. Let the space $\mathcal{K}={\mathcal{K}}_v$ consist of functions $g\in {\mathsf{H}}_{\mathrm{loc}}^1(\mathbb{R})$ with the norm

$$\begin{array}{c}\hfill {\Vert g\Vert }_v^2={\int }_{-\infty }^{\infty }(v^2(\mathit{\xi })|g^{\prime }{(\mathit{\xi })|}^2+v^{-2}(\mathit{\xi }){|g(\mathit{\xi })|}^2)d\mathit{\xi }.\end{array}$$

Then the set $C_0^1(\mathbb{R})$ is dense in $\mathcal{K}$ if and only if condition (3.9) is satisfied.

Proof

Let us make the change of variables

$$\begin{array}{c}\hfill \tilde{g}(\tilde{\mathit{\xi }})=g(\mathit{\xi })\mathrm{where}\tilde{\mathit{\xi }}={\int }_0^{\mathit{\xi }}v{(\mathit{\eta })}^{-2}d\mathit{\eta }\end{array}$$

and put

$$\begin{array}{c}\hfill v_{\pm }=\pm {\int }_{{\mathbb{R}}_{\pm }}v{(\mathit{\xi })}^{-2}d\mathit{\xi }.\end{array}$$

Then

$$\begin{array}{c}\hfill {\Vert g\Vert }_v^2={\int }_{v_{-}}^{v_{+}}(|{\tilde{g}}^{\prime }(\tilde{\mathit{\xi }}){|}^2+{|\tilde{g}(\tilde{\mathit{\xi }})|}^2)d\tilde{\mathit{\xi }}.\end{array}$$

Since $g\in C_0^1(\mathbb{R})$ if and only if $\tilde{g}\in C_0^1(v_{-},v_{+})$, it remains to use that the set $C_0^1(v_{-},v_{+})$ is dense in the Sobole space ${\mathsf{H}}^1(v_{-},v_{+})$ if and only if $v_{-}=-\infty$ and $v_{+}=+\infty$. $\square$

Returning to the Hankel operator $H$ with kernel (1.2) and using Theorem 3.2 and equality (3.6), we obtain the following result.

Theorem 3.6

Suppose that $P(x)=p_0+p_{1}x$ where $p_0={\overline{p}}_0$ and $p_1={\overline{p}}_1\ne 0$. Then

$$\begin{array}{c}\hfill H=q_1{F}^{\ast }TD{T}^{\ast }F\end{array}$$

where $T$ is defined by formula (3.10) with $q_0=p_0-\mathit{\gamma }{p}_1$, $q_1=p_1$ and $v(\mathit{\xi })$ is function (1.4). The operator $H$ is essentially self-adjoint on the set ${\mathcal{D}}_0$, and it is self-adjoint on the set $F^{\ast }{\mathcal{D}}_{\ast }$. The spectrum of the operator $H$ is simple, absolutely continuous, and it coincides with $\mathbb{R}$.

3.4. Let us pass to the case $K\ge 2$. We recall that the operator $A_0=vQ(D)v$ is symmetric in $L^2(\mathbb{R})$ on $C_0^K(\mathbb{R})$. Let us use the notation ${\mathcal{A}}_0$ for the same operator considered as a mapping ${\mathcal{A}}_0:C_0^K(\mathbb{R})\to L^2(\mathbb{R})$. The operator ${\mathcal{A}}_0^{\ast }:L^2(\mathbb{R})\to C_0^K{(\mathbb{R})}^{\prime }$ is defined by the relation

$$\begin{array}{c}\hfill ({\mathcal{A}}_{0}g,y)=〈g,{\mathcal{A}}_0^{\ast }y〉,g\in C_0^K(\mathbb{R}),y\in L^2(\mathbb{R}),\end{array}$$

(3.11)

and is given by the same differential expression (1.3) where derivatives are understood in the sense of distributions.

It is also quite easy to construct the operator $A_0^{\ast }$ adjoint to $A_0$ in the space $L^2(\mathbb{R})$. Let the domain ${\mathcal{D}}_{\ast }\subset L^2(\mathbb{R})$ consist of $y$ such that ${\mathcal{A}}_0^{\ast }y\in L^2(\mathbb{R})$. The following assertion is rather standard.

Lemma 3.7

The operator $A_0$ is symmetric on $C_0^{\infty }(\mathbb{R})$ and its adjoint $A_0^{\ast }$ is defined on the domain $\mathcal{D}(A_0^{\ast })={\mathcal{D}}_{\ast }$. For $y\in \mathcal{D}(A_0^{\ast })$, we have $A_0^{\ast }y={\mathcal{A}}_0^{\ast }y$.

Proof

By definition, $\mathcal{D}(A_0^{\ast })$ consists of $g\in L^2(\mathbb{R})$ such that

$$\begin{array}{c}\hfill (A_{0}g,y)=(g,y_{\ast })\end{array}$$

(3.12)

for all $g\in C_0^K(\mathbb{R})$ and some $y_{\ast }\in L^2(\mathbb{R})$; in this case $y_{\ast }=A_0^{\ast }y$. Observe that the left-hand sides of (3.11) and (3.12) coincide. If ${\mathcal{A}}_0^{\ast }y\in L^2(\mathbb{R})$, then (3.12) is satisfied with $y_{\ast }={\mathcal{A}}_0^{\ast }y$. Conversely, if (3.12) is satisfied, then

$$\begin{array}{c}\hfill 〈g,{\mathcal{A}}_0^{\ast }y〉=(g,y_{\ast }),\forall g\in C_0^K(\mathbb{R}),\end{array}$$

and hence $y_{\ast }={\mathcal{A}}_0^{\ast }y$ so that ${\mathcal{A}}_0^{\ast }y\in L^2(\mathbb{R})$. $\square$

Under additional assumptions on $v(\mathit{\xi })$ the operator $A_0^{\ast }$ is symmetric. The proof of this result requires the following auxiliary assertion. Recall that $D=id/d\mathit{\xi }$.

Lemma 3.8

Suppose that $v\in C^1(\mathbb{R})$, $v\in L^{\infty }(\mathbb{R})$, $v(\mathit{\xi })>0$ and

$$\begin{array}{c}\hfill |v^{\prime }(\mathit{\xi })|\le Cv(\mathit{\xi }).\end{array}$$

(3.13)

Let $z\in \mathbb{C}$ and let $|\text{Im}z|$ be sufficiently large. Then the operator $v{(D-z)}^{-1}{v}^{-1}$ in $L^2(\mathbb{R})$ defined on functions with compact supports extends to a bounded operator.

Proof

Let $z=a+ib$. Since

$$\begin{array}{c}\hfill ({(D-z)}^{-1}{y}_a)(\mathit{\xi })=e^{-ia\mathit{\xi }}({(D-ib)}^{-1}y)(\mathit{\xi }),\mathrm{where}{y}_a(\mathit{\xi })=e^{-ia\mathit{\xi }}y(\mathit{\xi }),\end{array}$$

we can suppose that $z=ib$ where $b\in \mathbb{R}$. We have to check the inequality

$$\begin{array}{c}\hfill {\Vert v(D-ib)}^{-1}{v}^{-1}w\Vert \le C\Vert w\Vert \end{array}$$

(3.14)

on a dense in $L^2(\mathbb{R})$ set of elements $w$ with compact supports. Consider $w=v(D-ib)u$ where $u\in C_0^{\infty }(\mathbb{R})$ is arbitrary; then $w\in C_0^1(\mathbb{R})$. The set of such elements $w$ is dense in $L^2(\mathbb{R})$. Indeed, suppose that

$$\begin{array}{c}\hfill (v(D-ib)u,g_0)=0\end{array}$$

for some $g_0\in L^2(\mathbb{R})$ and all $u\in C_0^{\infty }(\mathbb{R})$. Then $(D+ib)(v{g}_0)=0$ and hence

$$\begin{array}{c}\hfill v(\mathit{\xi })g_0(\mathit{\xi })=c{e}^{-b\mathit{\xi }}.\end{array}$$

Since $v\in L^{\infty }(\mathbb{R})$, it implies that $c=0$, and the equality $v(\mathit{\xi })g_0(\mathit{\xi })=0$ implies that $g_0(\mathit{\xi })=0$ because $v(\mathit{\xi })\ne 0$.

For $w=v(D-ib)u$, (3.14) is equivalent to the inequality

$$\begin{array}{cc}& {\int }_{-\infty }^{\infty }{v}^2{(\mathit{\xi })|u(\mathit{\xi })|}^{2}d\mathit{\xi }\le C{\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi }){|u^{\prime }(\mathit{\xi })-bu(\mathit{\xi })|}^{2}d\mathit{\xi }\hfill \\ & =C\left({\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi }){|u^{\prime }(\mathit{\xi })|}^{2}d\mathit{\xi }-2b\text{Re}{\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi })u^{\prime }(\mathit{\xi })\overline{u}(\mathit{\xi })d\mathit{\xi }\right.\hfill \\ & \left.+b^2{\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi }){|u(\mathit{\xi })|}^{2}d\mathit{\xi }\right).\hfill \end{array}$$

(3.15)

Integrating in the second term in the right-hand side by parts and using condition (3.13), we see that

$$\begin{array}{c}\hfill \text{Re}{\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi })u^{\prime }(\mathit{\xi })\overline{u}(\mathit{\xi })d\mathit{\xi }=-{\int }_{-\infty }^{\infty }{v}^{\prime }(\mathit{\xi })v(\mathit{\xi }){|u(\mathit{\xi })|}^{2}d\mathit{\xi }\end{array}$$

is bounded by

$$\begin{array}{c}\hfill C{\int }_{-\infty }^{\infty }{v}^2(\mathit{\xi }){|u(\mathit{\xi })|}^{2}d\mathit{\xi }.\end{array}$$

This proves inequality (3.15) if $b$ is large enough. $\square$

Corollary 3.9

Let $b\in \mathbb{R}$ be sufficiently large. Then for all $k=0,1,\dots ,K$, the operator $v{D}^k{(Q(D)-ib)}^{-1}{v}^{-1}$ in $L^2(\mathbb{R})$ defined on functions with compact supports extends to a bounded operator.

Proof

The equation $Q(x)=ib$ has $K$ solutions $x_1(b),\dots ,x_K(b)$ which for large $b$ are close to the solutions of the equation $q_K{x}^K=ib$. Therefore the roots $x_{\ell }(b)$ are simple and $|\text{Im}{x}_{\ell }(b)|\to \infty$ as $b\to \infty$ for all $\ell =1,\dots K$. Let us expand the function $x^k{(Q(x)-ib)}^{-1}$ in a linear combination of the functions ${(x-x_{\ell })}^{-1}$ and of the constant term $1$ (for $k=K$). We can apply Lemma 3.8 to every term $v{(D-x_{\ell }(b))}^{-1}{v}^{-1}$. The contribution of $1$ gives the identity operator. $\square$

Recall that $\mathcal{D}(A_0^{\ast })={\mathcal{D}}_{\ast }$ according to Lemma 3.7. Below we need additional information on this set. Let us accept the following

Assumption 3.10

The function $v\in C^K(\mathbb{R})$, $v\in L^{\infty }(\mathbb{R})$, $v(\mathit{\xi })>0$ and estimate (3.13) holds.

Lemma 3.11

Let Assumption 3.10 be satisfied. If $g\in \mathcal{D}(A_0^{\ast })$, then $v{D}^k(vg)\in L^2(\mathbb{R})$ for all $k=1,\dots ,K$ and, in particular, $g\in {\mathsf{H}}_{\mathrm{loc}}^K(\mathbb{R})$. Moreover, the coercitive estimates hold:

$$\begin{array}{c}\hfill \Vert v{D}^k(vg)\Vert \le C(\Vert vQ(D)(vg)\Vert +\Vert g\Vert ),k=1,\dots ,K.\end{array}$$

Proof

By definition of ${\mathcal{D}}_{\ast }$, we have $vQ(D)(vg)\in L^2(\mathbb{R})$ and hence $w:=v(Q(D)-ib)(vg)\in L^2(\mathbb{R})$ for all $b$. Observe that $v{D}^k(vg)=(v{D}^k{(Q(D)-ib)}^{-1}{v}^{-1})w$. Thus it remains to use Corollary 3.9. $\square$

This lemma shows that the set ${\mathcal{D}}_{\ast }\subset L^2(\mathbb{R})$ consists of functions $g\in {\mathsf{H}}_{\mathrm{loc}}^K(\mathbb{R})$ such that $v{D}^k(vg)\in L^2(\mathbb{R})$ for all $k=1,\dots ,K$. Now it is easy to check the following assertion.

Lemma 3.12

Under Assumption 3.10 the set $C_0^K(\mathbb{R})$ is dense in $\mathcal{D}(A_0^{\ast })$ in the graph-norm $\Vert g\Vert +\Vert A_0^{\ast }g\Vert$.

Proof

Let $\mathit{\phi }\in C_0^{\infty }(\mathbb{R})$ and $\mathit{\phi }(\mathit{\xi })=1$ for $|\mathit{\xi }|\le 1$. Set ${\mathit{\phi }}_n(\mathit{\xi })=\mathit{\phi }(\mathit{\xi }/n)$. For an arbitrary $g\in \mathcal{D}(A_0^{\ast })$, we put $g_n=g{\mathit{\phi }}_n$. Of course $\Vert g-g_n\Vert \to 0$ as $n\to \infty$. Set $u=vg$, $u_n=v{g}_n$. We have to show that $\Vert vQ(D)(u-u_n)\Vert \to 0$ as $n\to \infty$ or that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\Vert v{D}^k(u-u_n)\Vert =0,k=0,1,\dots ,K.\end{array}$$

(3.16)

Recall that $v{u}^{(k)}\in L^2(\mathbb{R})$ by Lemma 3.11. Therefore

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\Vert v{u}^{(k)}(1-{\mathit{\phi }}_n)\Vert =0\mathrm{and}\underset{n\to \infty }{lim}\Vert v{u}^{(k)}{\mathit{\phi }}_n^{(l)}\Vert =0\end{array}$$

for all $k=0,1,\dots ,K$ and $l\ge 1$. These relations imply (3.16). $\square$

Lemma 3.12 shows that the operator $A_0^{\ast }$ coincides with the closure ${\overline{A}}_0$ of the operator $A_0$. This yields the following assertion.

Theorem 3.13

Let Assumption 3.10 be satisfied. Then the operator $A_0$ defined by formula (1.3) on $C_0^K(\mathbb{R})$ is essentially self-adjoint. Its closure ${\overline{A}}_0=:A$ is self-adjoint on the set ${\mathcal{D}}_{\ast }=:\mathcal{D}(A)$ and $Ag=vQ(D)(vg)$ for $g\in {\mathcal{D}}_{\ast }$.

For $K$ even, it is also possible to define $A$ in terms of the quadratic form

$$\begin{array}{c}\hfill (Ag,g)={\int }_{-\infty }^{\infty }(Q(D)(vg))v\overline{g}d\mathit{\xi }.\end{array}$$

(3.17)

We suppose that $q_K>0$; then the form $(Ag,g)+{c\Vert g\Vert }^2$ is positive-definite for a sufficiently large $c>0$. Similarly to Theorem 3.13, it can be verified that this form defined on $C_0^K(\mathbb{R})$ admits the closure, and it is closed on the set ${\tilde{\mathcal{D}}}_{\ast }$ of functions $g\in L^2(\mathbb{R})$ such that $D^k(vg)\in L^2(\mathbb{R})$ for all $k=1,\dots ,K/2$. Then the operator $A+cI$ can be defined as a self-adjoint operator corresponding to this closed form. Note that ${\tilde{\mathcal{D}}}_{\ast }=\mathcal{D}(\sqrt{A+cI})$.

3.5. Let us return to Hankel operators. We recall that according to Theorem 3.2 the Hankel operator $H$ with kernel (1.2) is unitarily equivalent to differential operator (1.3) where $v$ is defined by formula (1.4) and $Q(x)$ is polynomial (3.4) with the coefficients defined by formula (3.3). To be more precise, the operators $H$ and $A$ are linked by relation (1.6) where $F$ is operator (1.5). In particular, we have

$$\begin{array}{c}\hfill \mathcal{D}(H)=F^{\ast }\mathcal{D}(A)\mathrm{and}\mathcal{D}(\sqrt{H+cI})=F^{\ast }\mathcal{D}(\sqrt{A+cI})\mathrm{for}K\mathrm{even}.\end{array}$$

Therefore the following result is a direct consequence of Theorem 3.13. Recall that the set ${\mathcal{D}}_0$ consists of functions $f(t)$ satisfying estimate (2.2) for $m=0$ and some $n>K+1$.

Theorem 3.14

Let kernel $h(t)$ be defined by formulas (1.2) and (3.1) where $p_k={\overline{p}}_k$ for $k=0,1,\dots ,K$. The Hankel operator $H$ with kernel $h(t)$ is essentially self-adjoint on the domain ${\mathcal{D}}_0$, and its closure is self-adjoint on the domain $F^{\ast }{\mathcal{D}}_{\ast }$.

4 Spectral results

Here we study spectral properties of the operators $A$ and $H$.

4.1. We recall that the precise definition of the operator $A$ was given in Theorem 3.13. The following result relies on a construction of trial functions.

Theorem 4.1

Let Assumption 3.10 be satisfied. Suppose additionally that

$$\begin{array}{c}\hfill v^{(k)}{v}^{-1}\in L^{\infty }(\mathbb{R}),k=1,\dots ,K-1,\end{array}$$

(4.1)

and that, for some $\mathit{\delta }>0$,

$$\begin{array}{c}\hfill \left({\int }_{-n(1+\mathit{\delta })}^{n(1+\mathit{\delta })}v{(\mathit{\xi })}^{-2+4/K}d\mathit{\xi }\right){\left({\int }_{-n}^{n}v{(\mathit{\xi })}^{-2}d\mathit{\xi }\right)}^{-1}\to 0\end{array}$$

(4.2)

as $n\to \infty$. If $K$ is odd, then $\mathrm{spec}(A)=\mathbb{R}$. If $K$ is even and $q_K>0$, then $[0,\infty )\subset \mathrm{spec}(A)$.

Proof

We shall construct Weyl sequences for all $\mathit{\lambda }\in \mathbb{R}$ in the case of odd $K$ and for all $\mathit{\lambda }\in [0,\infty )$ in the case of even $K$. Let $\mathit{\phi }=\overline{\mathit{\phi }}\in C_0^{\infty }(\mathbb{R})$, $\mathit{\phi }(\mathit{\xi })=1$ for $|\mathit{\xi }|\le 1$ and $\mathit{\phi }(\mathit{\xi })=0$ for $|\mathit{\xi }|\ge 1+\mathit{\delta }$. We set

$$\begin{array}{c}\hfill G(\mathit{\xi };\mathit{\lambda })={\mathit{\lambda }}^{1/K}{\int }_0^{\mathit{\xi }}v{(\mathit{\eta })}^{-2/K}d\mathit{\eta }\end{array}$$

(4.3)

and

$$\begin{array}{c}\hfill g_n(\mathit{\xi })=v{(\mathit{\xi })}^{-1}{e}^{iG(\mathit{\xi };\mathit{\lambda })}{\mathit{\phi }}_n(\mathit{\xi })\mathrm{where}{\mathit{\phi }}_n(\mathit{\xi })=\mathit{\phi }(\mathit{\xi }/n).\end{array}$$

Obviously, we have

$$\begin{array}{c}\hfill \Vert g_n{\Vert }^2\ge {\int }_{-n}^{n}v{(\mathit{\xi })}^{-2}d\mathit{\xi }.\end{array}$$

(4.4)

Let us calculate

$$\begin{array}{c}\hfill A{g}_n-{(-1)}^K{q}_K\mathit{\lambda }{g}_n=vQ(D)(e^{iG}{\mathit{\phi }}_n)-{(-1)}^K{q}_K\mathit{\lambda }{v}^{-1}{e}^{iG}{\mathit{\phi }}_n.\end{array}$$

(4.5)

Differentiating exponentials and using definition (4.3) and conditions (4.1), we see that

$$\begin{array}{c}\hfill D^k(e^{iG(\mathit{\xi };\mathit{\lambda })})=O(v{(\mathit{\xi })}^{-2k/K}),k=0,1,\dots ,K-1,\end{array}$$

and

$$\begin{array}{c}\hfill D^K(e^{iG(\mathit{\xi };\mathit{\lambda })})={(-1)}^K\mathit{\lambda }v{(\mathit{\xi })}^{-2}{e}^{iG(\mathit{\xi };\mathit{\lambda })}+O(v{(\mathit{\xi })}^{-2(K-1)/K})\end{array}$$

as $|\mathit{\xi }|\to \infty$. Estimating the functions ${\mathit{\phi }}_n(\mathit{\xi })$ and their derivatives by constants, we find that

$$\begin{array}{cc}& v(\mathit{\xi })Q(D)(e^{iG(\mathit{\xi };\mathit{\lambda })}{\mathit{\phi }}_n(\mathit{\xi }))={(-1)}^K{q}_K\mathit{\lambda }v{(\mathit{\xi })}^{-1}{e}^{iG(\mathit{\xi };\mathit{\lambda })}{\mathit{\phi }}_n(\mathit{\xi })\hfill \\ & +O(v{(\mathit{\xi })}^{1-2(K-1)/K}).\hfill \end{array}$$

(4.6)

Substituting this expression into (4.5), we see that the first term in the right-hand side of (4.6) is cancelled with the second term in the right-hand side of (4.5). This yields the estimate

$$\begin{array}{c}\hfill \Vert A{g}_n-{(-1)}^K{q}_K\mathit{\lambda }{g}_n{\Vert }^2\le C{\int }_{-n(1+\mathit{\delta })}^{n(1+\mathit{\delta })}v{(\mathit{\xi })}^{2-4(K-1)/K}d\mathit{\xi }.\end{array}$$

(4.7)

By virtue of condition (4.2), it follows from (4.4) and (4.7) that

$$\begin{array}{c}\hfill \Vert A{g}_n-{(-1)}^K{q}_K\mathit{\lambda }{g}_n\Vert \Vert g_n{\Vert }^{-1}\to 0\end{array}$$

as $n\to \infty$ so that ${(-1)}^K{q}_K\mathit{\lambda }\in \mathrm{spec}(A)$. $\square$

Let us discuss condition (4.2). If $K=2$, it means that

$$\begin{array}{c}\hfill n^{-1}{\int }_{-n}^{n}v{(\mathit{\xi })}^{-2}d\mathit{\xi }\to \infty \end{array}$$

as $n\to \infty$. Since the integral here can be estimated from below by $n{min}_{|\mathit{\xi }|\ge n/2}v{(\mathit{\xi })}^{-2}$, this condition is automatically satisfied provided $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$.

Let $K>2$. If

$$\begin{array}{c}\hfill {c|\mathit{\xi }|}^{-\mathit{\rho }}\le v(\mathit{\xi })\le C{|\mathit{\xi }|}^{-\mathit{\rho }},0<c<C<\infty ,\mathit{\rho }>0,\end{array}$$

then expression (4.2) is estimated by $C(\mathit{\delta })n^{-4\mathit{\rho }/K}$. Hence condition (4.2) is satisfied in this case (for all $\mathit{\delta }$). If

$$\begin{array}{c}\hfill c{e}^{-\mathit{\rho }|\mathit{\xi }|}\le v(\mathit{\xi })\le C{e}^{-\mathit{\rho }|\mathit{\xi }|},0<c<C<\infty ,\mathit{\rho }>0,\end{array}$$

then expression (4.2) is estimated by

$$\begin{array}{c}\hfill Cexp(2\mathit{\rho }n((1-2K^{-1})(1+\mathit{\delta })-1)).\end{array}$$

This expression tends to zero if $\mathit{\delta }<2{(K-2)}^{-1}$ so that condition (4.2) is again satisfied for such $\mathit{\delta }$. On the other hand, condition (4.2) can be violated for $K>2$ if $v(\mathit{\xi })$ tends to zero very rapidly (as $e^{-e^{|\mathit{\xi }|}}$, for example).

For the next result, assumptions (4.1) and (4.2) are not necessary.

Proposition 4.2

Let Assumption 3.10 be satisfied. Then $0$ is not an eigenvalue of the operator $A$.

Proof

Let $Ag=0$ for some $g\in \mathcal{D}(A)$. Put $u=vg$. Then $u\in L^2(\mathbb{R})$ because $v\in L^{\infty }(\mathbb{R})$. Since $v(\mathit{\xi })>0$, we have $Q(D)u=0$. Denote by $x_1,\dots ,x_{K_0}\in \mathbb{C}$ different roots of the equation $Q(x)=0$. Then

$$\begin{array}{c}\hfill u(\mathit{\xi })=\sum _{k=1}^{K_0}{P}_k(\mathit{\xi })e^{-i{x}_k\mathit{\xi }}\end{array}$$

(4.8)

for some polynomials $P_k(\mathit{\xi })$. Observe that all exponentials do not decay at least at one of the infinities. Therefore function (4.8) does not belong to $L^2(\mathbb{R})$ unless all polynomials $P_k(\mathit{\xi })$, $k=1,\dots ,K_0$, are zeros. It follows that $u=0$ whence $g=0$. $\square$

4.2. Let $K$ be even and $q_K>0$; then $A$ is semi-bounded from below and according to (3.17) we have

$$\begin{array}{c}\hfill (Ag,g)={\int }_{-\infty }^{\infty }Q(x){|({\mathrm{\Phi }}^{\ast }(vg))(x)|}^{2}dx,\forall g\in \mathcal{D}(A).\end{array}$$

(4.9)

Clearly, $A\ge 0$ if $Q(x)\ge 0$. On the other hand, if $Q(x_0)<0$ for some $x_0\in \mathbb{R}$, then $Q(x)<0$ for some interval $\mathrm{\Delta }$ centered at the point $x_0$. For every $N$, we choose functions ${\mathit{\psi }}_n\in C_0^{\infty }(\mathbb{R})$ with $\mathrm{supp}{\mathit{\psi }}_n\subset \mathrm{\Delta }$ for all $n=1,\dots ,N$ such that $\mathrm{supp}{\mathit{\psi }}_n\cap \mathrm{supp}{\mathit{\psi }}_m=\varnothing$ if $n\ne m$. The functions $g_n=v^{-1}\mathrm{\Phi }{\mathit{\psi }}_n\in \mathcal{D}(A)$ and according to (4.9) the form

$$\begin{array}{c}\hfill (Ag,g)=\sum _{n=1}^N|{\mathit{\alpha }}_n{|}^2{\int }_{-\infty }^{\infty }Q(x){|{\mathit{\psi }}_n(x)|}^{2}dx<0\end{array}$$

on all non-trivial linear combinations $g={\sum }_{n=1}^N{\mathit{\alpha }}_n{g}_n$ of the functions $g_1,\dots ,g_N$. This leads to the following result.

Theorem 4.3

Let Assumption 3.10 be satisfied. Suppose that $K$ is even and $q_K>0$. Then the operator $A$ is positive if and only if $Q(x)\ge 0$ for all $x\in \mathbb{R}$. Moreover, if $Q(x_0)<0$ for some $x_0\in \mathbb{R}$, then the negative spectrum of $A$ is infinite.

Let $K\ge 2$ be even and $q_K>0$. According to Theorem 4.1, $[0,\infty )\subset {\mathrm{spec}}_{\mathrm{ess}}(A)$. Let us show that actually we have the equality here. It follows from Theorem 4.3 that for sufficiently large $\mathit{\nu }$ the operator $A_{\mathit{\nu }}=v(Q(D)+\mathit{\nu })v\ge 0$. Thus we have to check that adding the operator $\mathit{\nu }{v}^2$ does not change the essential spectrum of $A$.

Lemma 4.4

In addition to the assumptions of Theorem 4.3 suppose that $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$. Then the operator $v^2{(A+i)}^{-1}$ is compact.

Proof

Let a set of functions $g_n$ be bounded in the graph-norm $\Vert Ag\Vert +\Vert g\Vert$. We have to check that it is compact in the norm $\Vert v^{2}g\Vert$. Put $u_n=v{g}_n$. Lemma 3.11 implies that

$$\begin{array}{c}\hfill \Vert v{u}_n^{\prime }\Vert +\Vert v^{-1}{u}_n\Vert \le C<\infty .\end{array}$$

(4.10)

We have to show that the set $u_n$ is compact in the norm $\Vert v{u}_n\Vert$ or in $L^2(\mathbb{R})$ because the function $v(\mathit{\xi })$ is bounded. Since $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$, the boundedness of the second term in (4.10) shows that the norms of $u_n$ in $L^2(-\infty ,-R)$ and $L^2(R,\infty )$ can be made arbitrary small uniformly in $n$ if $R$ is sufficiently large. Observe that $v(\mathit{\xi })\ge c>0$ on every compact interval, and hence the boundedness of the first term in (4.10) shows that the set $u_n$ is bounded in the Sobolev space ${\mathsf{H}}^1(-R,R)$. It follows that this set is compact in $L^2(-R,R)$ for all $R<\infty$. $\square$

Corollary 4.5

For an arbitrary $\mathit{\nu }$, we have

$$\begin{array}{c}\hfill {\mathrm{spec}}_{\mathrm{ess}}(A+\mathit{\nu }{v}^2)={\mathrm{spec}}_{\mathrm{ess}}(A).\end{array}$$

Putting together this result with Theorem 4.1, we obtain the following assertion.

Theorem 4.6

In addition to the assumptions of Theorem 4.1 suppose that $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$. If $K$ is even and $q_K>0$, then

$$\begin{array}{c}\hfill {\mathrm{spec}}_{\mathrm{ess}}(A)=[0,\infty ).\end{array}$$

(4.11)

We emphasize that equality (4.11) is due to the condition $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$. If $v(\mathit{\xi })=1$, then of course $\mathrm{spec}(A)={\mathrm{spec}}_{\mathrm{ess}}(A)=[P_{\min },\infty )$ where $P_{\min }=minP(x)$ for $x\in \mathbb{R}$.

4.3. Theorem 3.2 allows us to reformulate the results of the previous subsections in terms of Hankel operators. We recall that the precise definition of the operator $H$ was given in Theorem 3.14. Since function (1.4) satisfies Assumption 3.10 and conditions (4.1), (4.2), the following result is a consequence of Theorems 4.1 and 4.6.

Theorem 4.7

Let kernel $h(t)$ be defined by formulas (1.2) and (3.1) where $p_k={\overline{p}}_k$ for $k=0,1,\dots ,K$. Then:

$1^0$ The point $0$ is not an eigenvalue of $H$.

$2^0$ If $K$ is odd, then $\mathrm{spec}(H)=\mathbb{R}$.

$3^0$ If $K$ is even and $p_K>0$, then ${\mathrm{spec}}_{\mathrm{ess}}(H)=[0,\infty )$.

We emphasize that, for $K=1$, Theorem 3.6 yields a much stronger result.

Apparently the theory of Weyl-Titchmarsh-Kodaira does not apply to operators (1.3) because $v(\mathit{\xi })\to 0$ as $|\mathit{\xi }|\to \infty$. Nevertheless we conjecture that the spectrum of $A$ is absolutely continuous up to perhaps a discrete set of eigenvalues. Moreover, we expect that the spectrum of $A$ is simple for odd $K$ and it has multiplicity $2$ for even $K$.

Let $K$ be even and $p_K>0$; then $H$ is semi-bounded from below. Let us find conditions of the positivity of $H$. Since the operators $H$ and $A$ are unitarily equivalent, the following result is a direct consequence of Theorem 4.3.

Theorem 4.8

Let kernel $h(t)$ be defined by formulas (1.2) and (3.1) where $p_k={\overline{p}}_k$ for $k=0,1,\dots ,K$. Suppose that $K$ is even and $p_K>0$. Let $Q(x)$ be polynomial (3.4) with coefficients (3.3). Then the Hankel operator $H$ is positive if and only if $Q(x)\ge 0$ for all $x\in \mathbb{R}$. Moreover, if $Q(x_0)<0$ for some $x_0\in \mathbb{R}$, then the negative spectrum of $H$ is infinite.

Theorem 4.8 shows that the positivity of the Hankel operator with kernel (1.2) defined by a polynomial $P(x)$ is determined by another polynomial $Q(x)$ defined by formula (3.4). Of course the condition $Q(x)\ge 0$ is stronger than $P(x)\ge 0$. This follows, for example, from representation (3.5).

In the case $K=2$, the condition $Q(x)\ge 0$ reads as $q_1^2\le 4q_0{q}_2$. By virtue of (3.7) it can be rewritten as

$$\begin{array}{c}\hfill p_1^2+2{\mathit{\pi }}^2{p}_2^2/3\le 4p_0{p}_2.\end{array}$$

(4.12)

Obviously, this condition is stronger than the condition $p_1^2\le 4p_0{p}_2$ guaranteeing that $h(t)\ge 0$.

The following assertion is a particular case of Theorem 4.8.

Proposition 4.9

The Hankel operator $H$ with kernel

$$\begin{array}{c}\hfill h(t)=(p_0+p_{1}lnt+p_2{ln}^{2}t)t^{-1},p_0={\overline{p}}_0,p_1={\overline{p}}_1,p_2>0,\end{array}$$

is positive if and only if condition (4.12) is satisfied. Moreover, if $p_1^2+2{\mathit{\pi }}^2{p}_2^2/3>4p_0{p}_2$, then the negative spectrum of $H$ is infinite.

5 Hankel operators with discontinuous kernels

5.1. We here consider Hankel operators with singular kernels defined by formula (1.7). Hankel operators with such kernels are formally symmetric, and we shall see later that they are essentially self-adjoint on $C_0^{\infty }({\mathbb{R}}_{+})$. According to (1.1) we have

$$\begin{array}{c}\hfill (Hf)(t)=\sum _{k=0}^K{(-1)}^k{h}_k{f}^{(k)}(t_0-t),t\in (0,t_0),\end{array}$$

(5.1)

and $(Hf)(t)=0$ for $t>t_0$. Formula (5.1) gives us the precise definition of the Hankel operator with distributional kernel (1.7).

Since $L^2(t_0,\infty )\subset \mathrm{Ker}H$, it suffices to study the restriction of $H$ on the subspace $L^2(0,t_0)$. It is again given by differential expression (5.1) on $C^{\infty }$ functions vanishing in a neighborhood of the point $t=0$. Let us denote by $H_0$ the operator (5.1) in $L^2(0,t_0)$ with such domain $\mathcal{D}(H_0)$. Recall that ${\mathsf{H}}^K(0,t_0)$ is the Sobolev class. Let the set ${\mathsf{D}}_{\ast }\subset {\mathsf{H}}^K(0,t_0)$ consist of functions satisfying the boundary conditions

$$\begin{array}{c}\hfill f(0)=f^{\prime }(0)=\cdots =f^{(K-1)}(0)=0.\end{array}$$

(5.2)

The following assertion defines $H$ as a self-adjoint operator.

Lemma 5.1

The operator $H_0$ is symmetric and essentially self-adjoint. Its closure ${\overline{H}}_0=:H$ is self-adjoint in $L^2(0,t_0)$ on the domain $\mathcal{D}(H)={\mathsf{D}}_{\ast }$, and it acts by formula (5.1).

Proof

Let us denote by $H_{\ast }$ differential operator (5.1) considered as a mapping $H_{\ast }:L^2(0,t_0)\to C_0^{\infty }{(0,t_0)}^{\prime }$. Notice that $H_{\ast }:{\mathsf{H}}^K(0,t_0)\to L^2(0,t_0)$. Integrating by parts and using that $h_k={\overline{h}}_k$, we see that

$$\begin{array}{c}\hfill (H_{\ast }f,z)=-\sum _{k=1}^K{(-1)}^k{h}_k\sum _{l=0}^{k-1}{f}^{(k-1-l)}(t_0-t)\overline{z^{(l)}(t)}{|}_{t=0}^{t=t_0}+(f,H_{\ast }z)\end{array}$$

(5.3)

for all $f,z\in {\mathsf{H}}^K(0,t_0)$. Observe that the non-integral terms here vanish if both functions $f$ and $z$ satisfy boundary conditions (5.2). Since $H_{0}f=H_{\ast }f$ for $f\in \mathcal{D}(H_0)$, it follows that $(H_{0}f,z)=(f,H_{0}z)$ if $f,z\in \mathcal{D}(H_0)$.

Let us construct the adjoint operator $H_0^{\ast }$. Suppose that $(H_{0}f,z)=(f,z_{\ast })$ for all $f\in \mathcal{D}(H_0)$ and some $z,z_{\ast }\in L^2(0,t_0)$. Choosing first $f\in C_0^{\infty }(0,t_0)$ and using again (5.3), we see that $(H_{0}f,z)=(f,H_{\ast }z)$ and hence $z_{\ast }=H_{\ast }z$. Since $z_{\ast }\in L^2(0,t_0)$, we find that $z\in {\mathsf{H}}^K(0,t_0)$.

For an arbitrary $f\in \mathcal{D}(H_0)$, only the nonintegral terms in (5.3) corresponding to $t=t_0$ are equal to zero. Therefore it follows from (5.3) that the sum of terms corresponding to $t=0$ is also zero, that is,

$$\begin{array}{cc}& \sum _{k=1}^K{(-1)}^k{h}_k\sum _{l=0}^{k-1}{f}^{(k-1-l)}(t_0)\overline{z^{(l)}(0)}\hfill \\ & =-\sum _{p=0}^{K-1}{(-1)}^p{f}^{(p)}(t_0)\sum _{l=0}^{K-p-1}{(-1)}^l{h}_{p+l+1}\overline{z^{(l)}(0)}=0.\hfill \end{array}$$

Since the numbers $f^{(p)}(t_0)$ are arbitrary, we obtain a system of $K$ equations

$$\begin{array}{c}\hfill \sum _{l=0}^{K-p-1}{(-1)}^l{h}_{p+l+1}\overline{z^{(l)}(0)}=0,p=0,\dots ,K-1,\end{array}$$

(5.4)

for $K$ numbers $\overline{z(0)},\overline{z^{\prime }(0)},\dots ,\overline{z^{(K-1)}(0)}$. The matrix corresponding to this system consists of elements $a_{p,l}$ parametrized by indices $p,l=0,\dots ,K-1$. We have $a_{p,l}={(-1)}^l{h}_{p+l+1}$ for $p+l\le K-1$ and $a_{p,l}=0$ for $p+l>K-1$. The determinant of this matrix is the product of skew diagonal elements $a_{p,l}$ where $p+l=K-1$ times the factor ${(-1)}^{(K-1)K/2}$. Thus it equals $h_K^K$ which is not zero. Therefore it follows from (5.4) that necessarily $z(0)=z^{\prime }(0)=\cdots =z^{(K-1)}(0)=0$. It means that $\mathcal{D}(H_0^{\ast })\subset {\mathsf{D}}_{\ast }$ and $H_0^{\ast }z=H_{\ast }z$ for $z\in \mathcal{D}(H_0^{\ast })$.

Conversely, using again (5.3), we see that $(H_{\ast }f,z)=(f,H_{\ast }z)$ for all $f,z\in {\mathsf{D}}_{\ast }$. It follows that $\mathcal{D}(H_0^{\ast })={\mathsf{D}}_{\ast }$ and that the operator $H_0^{\ast }$ is symmetric. Hence the operator $H_0^{\ast \ast }={\overline{H}}_0$ is self-adjoint. $\square$

We note that zero is not an eigenvalue of the operator $H$. Indeed, after the change of variables $t\mapsto t_0-t$ the equation $(Hf)(t)=0$ reduces to the differential equation of order $K$. Therefore the unique solution of the equation $(Hf)(t)=0$ satisfying conditions (5.2) is zero.

5.2. Clearly, $H^2$ is the differential operator of order $2K$ defined by the formula

$$\begin{array}{c}\hfill (H^{2}f)(t)=\sum _{k,l=0}^K{(-1)}^k{h}_k{h}_l{f}^{(k+l)}(t)\end{array}$$

(5.5)

on functions in ${\mathsf{H}}^{2K}(0,t_0)$ satisfying boundary conditions (5.2) and

$$\begin{array}{c}\hfill \sum _{l=0}^K{(-1)}^l{h}_l{f}^{(k+l)}(t_0)=0,k=0,1,\dots ,K-1.\end{array}$$

Of course the spectrum of the operator $H^2$ consists of positive eigenvalues of multiplicity not exceeding $K$ (because the differential equation $H^{2}f=\mathit{\lambda }f$ together with conditions (5.2) has $K$ linearly independent solutions). These eigenvalues accumulate to $+\infty$ and their asymptotics is given by the Weyl formula. However, to find the asymptotics of eigenvalues of the operator $H$, we have to distinguish its positive and negative eigenvalues. For this reason, it is convenient to introduce an auxiliary operator $\tilde{H}$ with symmetric (with respect to the point $0$) spectrum having the same asymptotics of eigenvalues as $H$.

We define the operator $\tilde{H}$ by the same formula (5.1) as $H$ but consider it on functions in ${\mathsf{H}}^K(0,t_0/2)\oplus {\mathsf{H}}^K(t_0/2,t_0)$ satisfying the boundary conditions

$$\begin{array}{cc}\hfill f^{(k)}(0)& =f^{(k)}(t_0/2-0),f^{(k)}(t_0/2+0)=f^{(k)}(t_0),\hfill \\ & k=0,\dots ,K-1,\hfill \end{array}$$

(5.6)

for $K$ odd or

$$\begin{array}{cc}\hfill f^{(k)}(0)& =f^{(k)}(t_0/2-0)=0,f^{(k)}(t_0/2+0)=f^{(k)}(t_0)=0,\hfill \\ & k=0,\dots ,K/2-1,\hfill \end{array}$$

(5.7)

for $K$ even. The operator $\tilde{H}$ is self-adjoint in the space $L^2(0,t_0/2)\oplus L^2(t_0/2,t_0)$, and it is determined by the matrix

$$\begin{array}{c}\hfill \tilde{H}=\left(\begin{array}{cc}0& H_{1,2}\\ H_{2,1}& 0\end{array}\right),H_{1,2}=H_{2,1}^{\ast },\end{array}$$

(5.8)

where $H_{2,1}:L^2(0,t_0/2)\to L^2(t_0/2,t_0)$. The operator $H_{2,1}$ is again given by formula (5.1 on functions in ${\mathsf{H}}^K(0,t_0/2)$ satisfying conditions (5.6) for $K$ odd or (5.7) for $K$ even at the points $0$ and $t_0/2-0$. It follows from representation (5.8) that the spectrum of the operator $\tilde{H}$ is symmetric with respect to the point $0$ and consists of eigenvalues $\pm \sqrt{{\mathit{\mu }}_n}$ where ${\mathit{\mu }}_n$ are eigenvalues of the operator $H_{2,1}^{\ast }{H}_{2,1}=:\mathbf{H}$.

Obviously, the operator $H_{2,1}^{\ast }$ is again given by formula (5.1) on functions in ${\mathsf{H}}^K(t_0/2,t_0)$ satisfying conditions (5.6) for $K$ odd or (5.7) for $K$ even at the points $t_0/2+0$ and $t_0$. The operator $\mathbf{H}$ acts in the space $L^2(0,t_0/2)$ according to equality (5.5) and its domain $\mathcal{D}(\mathbf{H})$ consists of functions $f\in \mathcal{D}(H_{2,1})$ such that $H_{2,1}f\in \mathcal{D}(H_{2,1}^{\ast })$; in particular, $\mathcal{D}(\mathbf{H})\subset {\mathsf{H}}^{2K}(0,t_0/2)$. If $K$ is odd, we have the boundary conditions $f^{(k)}(0)=f^{(k)}(t_0/2)$ for $k=0,\dots ,2K-1$. If $K$ is even, then equalities (5.7) should be complemented by the boundary conditions

$$\begin{array}{c}\hfill \sum _{l=K/2-k}^K{(-1)}^l{h}_l{f}^{(l+k)}(0)=\sum _{l=K/2-k}^K{(-1)}^l{h}_l{f}^{(l+k)}(t_0/2)=0\end{array}$$

(5.9)

for $k=0,\dots ,K/2-1$. Note that conditions (5.7) and (5.9) at the point $0$ as well as at the point $t_0/2$ are linearly independent because $h_K\ne 0$.

Let ${\mathit{\mu }}_n$ be eigenvalues of the operator $\mathbf{H}$ enumerated in increasing order with multiplicities taken into account. According to the Weyl formula we have

$$\begin{array}{c}\hfill {\mathit{\mu }}_n=h_K^2{(2\mathit{\pi }{t}_0^{-1}n)}^{2K}(1+O(n^{-1})),n\to \infty .\end{array}$$

This yields the asymptotics of eigenvalues $\pm \sqrt{{\mathit{\mu }}_n}$ of the operator $\tilde{H}$.

Let us now observe that the operators $H$ and $\tilde{H}$ are self-adjoint extensions of the same symmetric operator $H_{00}$ with finite deficiency indices $(2K,2K)$. The operator $H_{00}$ can be defined by formula (5.1) on $C^{\infty }$ functions vanishing in some neighbourhoods of the points $0$, $t_0/2$ and $t_0$. Therefore the operators $H$ and $\tilde{H}$ have the same asymptotics of spectra. Thus we have obtained the following result.

Theorem 5.2

Let $H$ be the self-adjoint Hankel operator with kernel (1.7). Then $\mathrm{Ker}H=L^2(t_0,\infty )$. The non-zero spectrum of $H$ consists of infinite number of eigenvalues ${\mathit{\lambda }}_n^{(\pm )}$ of multiplicities not exceeding $K$ such that $0<{\mathit{\lambda }}_1^{(+)}\le {\mathit{\lambda }}_2^{(+)}\le \cdots \le {\mathit{\lambda }}_n^{(+)}\le \cdots$ and $0>{\mathit{\lambda }}_1^{(-)}\ge {\mathit{\lambda }}_2^{(-)}\ge \cdots \ge {\mathit{\lambda }}_n^{(-)}\ge \cdots$. Eigenvalues ${\mathit{\lambda }}_n^{(\pm )}$ accumulate to $\pm \infty$ as $n\to \infty$ and have the asymptotics

$$\begin{array}{c}\hfill {\mathit{\lambda }}_n^{(\pm )}=\pm |h_K|{(2\mathit{\pi }{t}_0^{-1}n)}^K(1+O(n^{-1}))\end{array}$$

as $n\to \infty$. The corresponding eigenfunctions $f_n^{(\pm )}(t)$ satisfy the equation

$$\begin{array}{c}\hfill \sum _{k=0}^K{(-1)}^k{h}_k\frac{d^k{f}_n^{(\pm )}(t)}{d{t}^k}={\mathit{\lambda }}_n^{(\pm )}{f}_n^{(\pm )}(t_0-t),t\in (0,t_0),\end{array}$$

and boundary conditions (5.2).

Remark 5.3

In the case $h(t)={\mathit{\delta }}^{\prime }(t-t_0)$ we have the explicit formulas

$$\begin{array}{c}\hfill {\mathit{\lambda }}_n^{(+)}=2\mathit{\pi }{t}_0^{-1}(n-1/4),{\mathit{\lambda }}_n^{(-)}=-2\mathit{\pi }{t}_0^{-1}(n-3/4).\end{array}$$