On square integrable solutions and principal and antiprincipal solutions for linear Hamiltonian systems

Abstract

New results in the Weyl–Titchmarsh theory for linear Hamiltonian differential systems are derived by using principal and antiprincipal solutions at infinity. In particular, a non-limit circle case criterion is established and a close connection between the Weyl solution and the minimal principal solution at infinity is shown in the limit point case. In addition, the square integrability of the columns of the minimal principal solution at infinity is investigated. All results are obtained without any controllability assumption. Several illustrative examples are also provided.

1 Introduction

In this paper we study the linear Hamiltonian differential system

where $t\in [a,\infty )$, $\lambda \in \mathbb {C}$ is a spectral parameter, and $\mathcal {H}(t)$ and $\mathcal {W}(t)$ are piecewise continuous even order matrix-valued functions such that the matrix $\mathcal {H}(t)$ is Hamiltonian, i.e., $\mathcal {J}\mathcal {H}(t)+\mathcal {H}^*(t)\mathcal {J}=0$, and $\mathcal {W}(t)=\mathcal {W}^*(t)\ge 0$ for all $t\in [a,\infty )$. For some results we will also assume the Legendre condition, which means that the right upper block of the matrix $\mathcal {H}(t,\lambda )$ is positive semidefinite, i.e.,

Precise assumptions about the coefficients are summarized in Notation 2.1 below. By using recent theory of principal and antiprincipal solutions at infinity, we obtain new results in the Weyl–Titchmarsh theory concerning the square integrable solutions of ($\text {H}_\lambda$). We do not assume any controllability assumption and generalize the results in [7] dealing with the second-order Sturm–Liouville differential equations to system ($\text {H}_\lambda$).

One of the fundamental contribution to the initial development of the spectral theory for system ($\text {H}_\lambda$) goes back to the monograph [1] by Atkinson. This theory has been intensively studied in the last four decades, see e.g., [3, 5, 17, 18, 20, 22, 26]. In this paper we are interested in the Weyl–Titchmarsh theory, which is devoted to square integrable solutions of system ($\text {H}_\lambda$) for $\lambda \in \mathbb {C}$, i.e., to solutions $z(\cdot ,\lambda )$ with

$$\begin{aligned} \int _a^\infty z^*(t,\lambda )\,\mathcal {W}(t)\,z(t,\lambda )\,\mathrm {d}t<\infty . \end{aligned}$$

In this theory the so-called Weyl solution plays a crucial role, because it provides a lower bound for the number of linearly independent square integrable solutions of ($\text {H}_\lambda$) for $\lambda \in \mathbb {C}\backslash \mathbb {R}$, see [1, Section 9] and, e.g., [44, Formula (5.4)]. The minimal and maximal numbers of such linearly independent square integrable solutions of ($\text {H}_\lambda$) then lead to the limit point and limit circle classification of system ($\text {H}_\lambda$).

For the second-order Sturm–Liouville differential equation

$$\begin{aligned} -[P_1(t)\,y'(t,\lambda )]'+P_0(t)\,y(t,\lambda )=\lambda W(t)\,y(t,\lambda ), \end{aligned}$$

(1.1)

being nonoscillatory and in the limit point case, it is known that for all $\lambda \in \mathbb {R}$ sufficiently small the Weyl solution of (1.1) coincides up to a nonsingular constant multiple with the principal solution of (1.1) at infinity, see [7, Theorems 2.13 and 3.11]. This relationship allows to transfer the knowledge from the Weyl–Titchmarsh theory to the oscillation theory of (1.1) and vice versa.

The latter result is a motivation for the present paper, in which we establish a similar connection between the Weyl solution and the principal solution at infinity for system ($\text {H}_\lambda$). We show that this theory can be developed without any controllability assumption on system ($\text {H}_\lambda$), as opposed to the controllable equation (1.1). In this respect the unique minimal principal solution of ($\text {H}_\lambda$) at infinity represents the (analytic) extension of the unique Weyl solution to real values of $\lambda$, see Theorem 3.3. For the existence of the minimal principal solution of ($\text {H}_\lambda$) at infinity we require the validity of the Legendre condition ($\text {LC}_\lambda$), see [35]. On the other hand, the Weyl solution may exist for $\lambda \in \mathbb {R}$ even when ($\text {LC}_\lambda$) is not satisfied, as we show in Example 3.7. This means that the property of “being the Weyl solution for $\lambda \in \mathbb {R}$” is more general that the property of “being the minimal principal solution at infinity.” This problem is also closely related to the square integrability of the columns of the minimal principal solution at infinity, which we discuss in Theorem 3.8.

In the second main result of this paper (Theorem 4.1) we extend a well-known limit point criterion for the second-order Sturm–Liouville differential equation (1.1) from [30, Theorem 4.1]. In the context of system ($\text {H}_\lambda$) with the block diagonal weight matrix $\mathcal {W}(t)$ it is formulated as a non-limit circle criterion, i.e., we prove the existence of a non-square integrable solution of system ($\text {H}_\lambda$) for any $\lambda \in \mathbb {C}$. This result is based on an asymptotic characterization of (maximal) antiprincipal solutions of ($\text {H}_\lambda$) at infinity from [36].

The paper is organized as follows. In Sect. 2 we recall some relevant results about system ($\text {H}_\lambda$), in particular about the Weyl solution, the minimal principal solution at infinity, and the antiprincipal solutions at infinity. In Sect. 3 we study a connection between the minimal principal solution at infinity and the Weyl solution, as well as the square integrability of the columns of the minimal principal solution at infinity. In Sect. 4 we establish a new non-limit circle criterion for system ($\text {H}_\lambda$). Throughout the paper we provide several examples, which illustrate our results.

2 Preliminaries

2.1 Notation and basic facts

Throughout the paper all vectors, vector-valued functions, matrices, and matrix-valued functions are considered over $\mathbb {C}$ if not specified otherwise, with vectors being written by lowercase letters and matrices by capital letters. The block diagonal matrix $\left( \begin{array}{ll}M &{} 0\\ 0 &{} N\end{array}\right)$ is abbreviated as ${{\mathrm{diag}}}\{M,N\}$. The transpose, conjugate transpose, inverse, Moore–Penrose pseudoinverse, positive definiteness, positive semidefiniteness, determinant, rank, trace, kernel, Hermitian components (or real and imaginary parts), and the largest eigenvalue for a given matrix M are indicated, respectively, by $M^\top$, $M^*$, $M^{-1}$, $M^\dagger$, $M>0$, $M\ge 0$, $\det M$, ${{\mathrm{rank}}}M$, ${{\mathrm{tr}}}M$, ${{\mathrm{Ker}}}M$, ${{\mathrm{re}}}(M):=(M+M^*)/2$, ${{\mathrm{im}}}(M):=(M-M^*)/(2i)$, and $\Lambda _{\mathrm {max}}(M)$. Moreover, the notation $M\le N$ for Hermitian matrices M and N means that $N-M\ge 0$. For matrix-valued functions we also write $M^*(\cdot ):=[M(\cdot )]^{*}$, $M^{-1}(\cdot ):=[M(\cdot )]^{-1}$, and $M^{*-1}(\cdot ):=[M^*(\cdot )]^{-1}$. A $2n\times 2n$ matrix M is said to be (conjugate) symplectic if $M^*\mathcal {J}M=\mathcal {J}$, where $\mathcal {J}$ is the $2n\times 2n$ canonical skew-symmetric matrix defined in ($\text {H}_\lambda$) in Sect. 1.

By $\Vert {M}\Vert _\sigma := \sqrt{\Lambda _{\mathrm {max}}(M^*M)}$ we denote the spectral norm of a matrix M. It is well known that the norm $\Vert {\cdot }\Vert _\sigma$ is submultiplicative and self-adjoint, i.e., $\Vert {MN}\Vert _\sigma \le \Vert {M}\Vert _\sigma \times \Vert {N}\Vert _\sigma$ and $\Vert {M^*}\Vert _\sigma =\Vert {M}\Vert _\sigma$, and for any Hermitian matrices M and N with $0\le M\le N$ it satisfies, see [4, Section 9],

$$\begin{aligned} \text {(i)}\quad \Vert {M}\Vert _\sigma =\Lambda _{\mathrm {max}}(M), \qquad \text {(ii)}\quad \Vert {M}\Vert ^{1/2}_\sigma =\Vert {M^{1/2}}\Vert _\sigma , \qquad \text {(iii)}\quad \Vert {M}\Vert _\sigma \le \Vert {N}\Vert _\sigma . \end{aligned}$$

(2.1)

For convenience of the reader we summarize the basic notation concerning system ($\text {H}_\lambda$), which is used throughout the paper.

Notation 2.1

The numbers $a\in \mathbb {R}$ and $n\in \mathbb {N}$ are fixed and

where $\mathcal {W}(t)\ge 0$ on $[a,\infty )$, and the complex $n\times n$ matrix-valued functions A(t), B(t), C(t), $W_1(t)$, $W_2(t)$, $W_4(t)$ are piecewise continuous on $[a,\infty )$ with $B(t)=B^*(t)$, $C^*(t)=C(t)$, $W_1^*(t)=W_1(t)$, and $W_4^*(t)=W_4(t)$.

With this notation system ($\text {H}_\lambda$) can be equivalently written as

$$\begin{aligned} -\mathcal {J}z'(t,\lambda )=[-\mathcal {J}\mathcal {H}(t)+\lambda \mathcal {W}(t)]\,z(t,\lambda ) \end{aligned}$$

with $-\mathcal {J}\mathcal {H}(t)$ being a Hermitian matrix. This form is traditionally used in the spectral theory of linear Hamiltonian systems, see e.g., [1, 3, 22, 26, 46]. Moreover, the matrices $\mathcal {H}(t)$ and $\mathcal {J}\mathcal {W}(t)$ are Hamiltonian, and the blocks of $\mathcal {H}(t,\lambda )$ satisfy

$$\begin{aligned} \mathcal {B}^*(t,\lambda )=\mathcal {B}(t,\bar{\lambda }),\quad \mathcal {C}^*(t,\lambda )=\mathcal {C}(t,\bar{\lambda }), \quad \mathcal {A}^*(t,\lambda )=-\mathcal {D}(t,\bar{\lambda }), \quad t\in [a,\infty ),\ \ \lambda \in \mathbb {C}. \end{aligned}$$

(2.2)

It is easy to see that the matrix $\mathcal {H}(t,\lambda )$ is Hamiltonian on $[a,\infty )$ when $\lambda \in \mathbb {R}$.

By a (vector) solution of ($\text {H}_\lambda$) we mean a 2n vector-valued function $z(\cdot ,\lambda )\in \mathrm {C}_p^1$ (i.e., piecewise continuously differentiable) satisfying system ($\text {H}_\lambda$) for all $t\in [a,\infty )$. Matrix-valued solutions are defined accordingly. In addition, by $x(\cdot ,\cdot )$, $u(\cdot ,\cdot )$ and $X(\cdot ,\cdot )$, $U(\cdot ,\cdot )$ we denote the “halves” of a vector solution $z(\cdot ,\cdot )$ and a matrix solution $Z(\cdot ,\cdot )$, respectively, i.e., $z(t,\lambda )=\big (x^\top (t,\lambda ),u^\top (t,\lambda )\big )^{\top }$ and $Z(t,\lambda )=\big (X^\top (t,\lambda ),U^\top (t,\lambda )\big )^{\top }$ for all $t\in [a,\infty )$ and $\lambda \in \mathbb {C}$.

The following Lagrange identity represents one of the crucial tools in the Weyl–Titchmarsh theory. If $p,q\in \mathbb {N}$ and $\lambda ,\nu \in \mathbb {C}$ are arbitrary, then for any $2n\times p$ and $2n\times q$ solutions $Z_1(\cdot ,\lambda )$ and $Z_2(\cdot ,\nu )$ of systems ($\text {H}_\lambda$) and ($\text {H}_{\nu }$) and $s\in [a,\infty )$ we have

$$\begin{aligned} W[Z_1(t,\lambda ),Z_2(t,\nu )]=W[Z_1(s,\lambda ),Z_2(s,\nu )]+(\bar{\lambda }-\nu )\int _s^t Z_1^*(\tau ,\lambda )\,\mathcal {W}(\tau )\,Z_2(\tau ,\nu )\,\mathrm {d}\tau , \end{aligned}$$

(2.3)

see e.g., [1, Formula (9.1.11)]. Here $W[Z_1(t,\lambda ),Z_2(t,\nu )]$ denotes the natural extension of the Wronskian to the complex $\lambda$-dependent systems, i.e.,

$$\begin{aligned} W[Z_1(t,\lambda ),Z_2(t,\nu )]:= Z_1^*(t,\lambda )\,\mathcal {J}Z_2(t,\nu )=X_1^*(t,\lambda )\,U_2(t,\nu )-U_1^*(t,\lambda )\,X_2(t,\nu ). \end{aligned}$$

If $\nu =\bar{\lambda }$ (in particular if $\nu =\lambda \in \mathbb {R}$), identity (2.3) reduces to the well-known fact about the constancy of the Wronskian, i.e., $W[Z_1(t,\lambda ),Z_2(t,\bar{\lambda })]\equiv W[Z_1(a,\lambda ),Z_2(a,\bar{\lambda })]$ on $[a,\infty )$. We will pay a special attention to solutions $Z(\cdot ,\lambda )$ satisfying ${{\mathrm{rank}}}Z(t,\lambda )=n$ for some (and hence for any) $t\in [a,\infty )$ and

$$\begin{aligned} W[Z(t,\lambda ),Z(t,\bar{\lambda })]\equiv 0,\quad \text {i.e.,}\quad X^*(t,\lambda )\,U(t,\bar{\lambda })=U^*(t,\lambda )\,X(t,\bar{\lambda }). \end{aligned}$$

(2.4)

When $\lambda \in \mathbb {R}$ such solutions are called conjoined bases of ($\text {H}_\lambda$). An alternative terminology is an isotropic or prepared or self-conjugate solution of ($\text {H}_\lambda$), see [8, 16, 23, 32, 34].

In this paper we will utilize two $2n\times n$ solutions

$$\begin{aligned} \widehat{Z}_\alpha (\cdot ,\lambda )=\big (\widehat{X}^\top _\alpha (\cdot ,\lambda ),\widehat{U}^\top _\alpha (\cdot ,\lambda )\big )^\top \quad \text {and}\quad \widetilde{Z}_\alpha (\cdot ,\lambda )=\big (\widetilde{X}^\top _\alpha (\cdot ,\lambda ),\widetilde{U}^\top _\alpha (\cdot ,\lambda )\big )^\top \end{aligned}$$

(2.5)

of system ($\text {H}_\lambda$) determined by the initial conditions $\widehat{Z}_\alpha (a,\lambda )=\alpha ^*$ and $\widetilde{Z}_\alpha (a,\lambda )=-\mathcal {J}\alpha ^*$, where $\alpha \in \Gamma :=\{\alpha \in \mathbb {C}^{n\times 2n}\mid \alpha \alpha ^*=I,\ \alpha \,\mathcal {J}\alpha ^*=0\}$. Since these initial conditions do not depend on $\lambda$ and the $2n\times 2n$ matrix $(\alpha ^*,-\mathcal {J}\alpha ^*)$ is symplectic, it follows that $\Phi _\alpha (t,\lambda ):=\big (\widehat{Z}_\alpha (t,\lambda ),\widetilde{Z}_\alpha (t,\lambda )\big )$ is a fundamental matrix of ($\text {H}_\lambda$), which satisfies the symplectic-type identity

$$\begin{aligned} \Phi _\alpha ^*(t,\lambda )\,\mathcal {J}\Phi _\alpha (t,\bar{\lambda })= & {} \left( \begin{array}{ll}W[\widehat{Z}_\alpha (t,\lambda ),\widehat{Z}_\alpha (t,\bar{\lambda })] &{} W[\widehat{Z}_\alpha (t,\lambda ),\widetilde{Z}_\alpha (t,\bar{\lambda })]\\ W[\widetilde{Z}_\alpha (t,\lambda ),\widehat{Z}_\alpha (t,\bar{\lambda })] &{} W[\widetilde{Z}_\alpha (t,\lambda ),\widetilde{Z}_\alpha (t,\bar{\lambda })]\end{array}\right) \\\equiv & {} \Phi _\alpha ^*(a,\lambda )\,\mathcal {J}\Phi _\alpha (a,\bar{\lambda })=\mathcal {J}. \end{aligned}$$

For simplicity, we write $\Phi _I(t,\lambda )=\big (\widehat{Z}_I(t,\lambda ),\widetilde{Z}_I(t,\lambda )\big )$ if $\alpha =(I,0)$, that is, $\Phi _I(a,\lambda )=I$.

Remark 2.2

If the matrices $\widetilde{X}_\alpha (t,\lambda )$ and $\widetilde{X}_\alpha (t,\bar{\lambda })$ are invertible for some $t\in [a,\infty )$ and $\lambda \in \mathbb {C}$, then the equalities $W[\widetilde{Z}_\alpha (t,\lambda ),\widetilde{Z}_\alpha (t,\bar{\lambda })]=0$ and $W[\widetilde{Z}_\alpha (t,\lambda ),\widehat{Z}_\alpha (t,\bar{\lambda })]=-I$ imply

$$\begin{aligned} \widehat{U}_\alpha (t,\lambda )-\widetilde{U}_\alpha (t,\lambda )\,\widetilde{X}^{-1}_\alpha (t,\lambda )\,\widehat{X}_\alpha (t,\lambda ) =-\widetilde{X}_\alpha ^{*-1}(t,\bar{\lambda }). \end{aligned}$$

(2.6)

Similarly, when the matrices $\widehat{X}_\alpha (t,\lambda )$ and $\widehat{X}_\alpha (t,\bar{\lambda })$ are invertible, then

$$\begin{aligned} \widetilde{U}_\alpha (t,\lambda )-\widehat{U}_\alpha (t,\lambda )\,\widehat{X}^{-1}_\alpha (t,\lambda )\,\widetilde{X}_\alpha (t,\lambda ) =\widehat{X}_\alpha ^{*-1}(t,\bar{\lambda }). \end{aligned}$$

(2.7)

2.2 Elements of Weyl–Titchmarsh theory

In this subsection we recall some basic results from the Weyl–Titchmarsh theory for system ($\text {H}_\lambda$). We denote by $\mathcal {L}^2_{\mathcal {W}}$ the usual space of Lebesgue measurable functions $z:[a,\infty )\rightarrow \mathbb {C}^{2n}$ such that $\int _a^\infty z^*(t)\,\mathcal {W}(t)\,z(t)\,\mathrm {d}t<\infty$. Moreover, we use the notation

$$\begin{aligned} \mathcal {N}(\lambda ):= \Big \{z(\cdot ,\lambda )\in \mathrm {C}_p^1\cap \mathcal {L}^2_{\mathcal {W}}\bigm \vert z(\cdot ,\lambda ) \text {solves system}~(\text {H}_\lambda )\Big \} \end{aligned}$$

for the linear space of all square integrable solutions of ($\text {H}_\lambda$). Let us note that $\mathcal {L}^2_{\mathcal {W}}$ is not a Hilbert space in general, because of the semidefiniteness of $\mathcal {W}(\cdot )$. The number of linearly independent square integrable solutions of ($\text {H}_\lambda$) is then equal to $\dim \mathcal {N}(\lambda )$. If $\dim \mathcal {N}(\lambda )=n$, we say that system ($\text {H}_\lambda$) is in the limit point case, while in the maximal case $\dim \mathcal {N}(\lambda )=2n$ we say that system ($\text {H}_\lambda$) is in the limit circle case. For a basic estimate of the value of $\dim \mathcal {N}(\lambda )$ we will need the following weak Atkinson condition, see [44, Hypothesis 4.5].

Hypothesis 2.3

(Weak Atkinson condition) There exists $\lambda \in \mathbb {C}$ and $b\in (a,\infty )$ such that

$$\begin{aligned} \int _a^b z^*(t,\lambda )\,\mathcal {W}(t)\,z(t,\lambda )\,\mathrm {d}t>0 \end{aligned}$$

for any nontrivial linear combination $z(\cdot ,\lambda )$ of the columns of the solution $\widetilde{Z}_\alpha (\cdot ,\lambda )$.

In other words, Hypothesis 2.3 means that any solution $z(\cdot ,\lambda )$ such that $z(t,\lambda )=\widetilde{Z}_\alpha (t,\lambda )\,\xi$ on $[a,\infty )$ with $\xi \in \mathbb {C}^n$ and $\int _a^\infty z^*(t,\lambda )\,\mathcal {W}(t)\,z(t,\lambda )\,\mathrm {d}t=0$ is necessarily a trivial solution. Moreover, it can be shown similarly as in [3, Lemma 2.10] that Hypothesis 2.3 holds for one $\lambda \in \mathbb {C}$ if and only if the same condition is satisfied for any $\lambda \in \mathbb {C}$.

If $\lambda \in \mathbb {C}\backslash \mathbb {R}$ is fixed, then system ($\text {H}_\lambda$) possesses at least n linearly independent square integrable solutions under Hypothesis 2.3, see e.g., [1, Theorem 9.11.1] and [44, Theorem 5.1]. This fact follows from the square integrability of the columns of the Weyl solution

$$\begin{aligned} \mathcal {X}(t,\lambda ):= \Phi _\alpha (t,\lambda )\,\left( \begin{array}{ll}I\\ M\end{array}\right) =\widehat{Z}_\alpha (t,\lambda )+\widetilde{Z}_\alpha (t,\lambda )\,M, \end{aligned}$$

(2.8)

where $M\in \mathbb {C}^{n\times n}$ belongs to the limiting Weyl disk

$$\begin{aligned} D_+(\lambda ):=\{P_+(\lambda )+R_+(\lambda )\mathcal {V}R_+(\bar{\lambda })\mid \mathcal {V}\in \mathbb {V}\}. \end{aligned}$$

(2.9)

Here $\mathbb {V}:=\{\mathcal {V}\in \mathbb {C}^{n\times n}\mid \mathcal {V}^*\mathcal {V}\le I\}$ is the set of all $n\times n$ contractive matrices, and the $n\times n$ matrices $P_+(\lambda )$ and $R_+(\lambda )$ are defined as the limits

$$\begin{aligned} P_+(\lambda )&:= \lim _{t\rightarrow \infty }P(t,\lambda ), \quad P(t,\lambda ) := H^{-1}(t,\lambda )\,G(t,\lambda ), \\ R_+(\lambda )&:= \lim _{t\rightarrow \infty }R(t,\lambda ), \quad R(t,\lambda ) := H^{-1/2}(t,\lambda ), \end{aligned}$$

where

$$\begin{aligned} G(t,\lambda ):= i\delta (\lambda )\,\widetilde{Z}_\alpha ^*(t,\lambda )\,\mathcal {J}\widehat{Z}_\alpha (t,\lambda ),\quad H(t,\lambda ):= i\delta (\lambda )\,\widetilde{Z}_\alpha ^*(t,\lambda )\,\mathcal {J}\widetilde{Z}_\alpha (t,\lambda ),\quad \delta (\lambda ):= {{\mathrm{sgn}}}({{\mathrm{im}}}\lambda ), \end{aligned}$$

see [1, Section 9.8], [40, Section 3], and [44, Section 4]. Note that Hypothesis 2.3 implies the invertibility of $H(t,\lambda )$ for all $t\ge b$ by (2.3), see also [44, Identity (4.15)]. The Weyl disks $D(t,\lambda )$, $t\ge b$, can be expressed similarly as $D_+(\lambda )$ in (2.9) with the matrices $P_+(\lambda )$, $R_+(\lambda )$, $R_+(\bar{\lambda })$ replaced by $P(t,\lambda )$, $R(t,\lambda )$, $R(t,\bar{\lambda })$.

For $\beta \in \Gamma$, $\lambda \in \mathbb {C}\backslash \mathbb {R}$, and $t\ge b$ we define the Weyl–Titchmarsh function

$$\begin{aligned} M(t,\lambda ):=-[\beta \,\widetilde{Z}_\alpha (t,\lambda )]^{-1}\,\beta \,\widehat{Z}_\alpha (t,\lambda ). \end{aligned}$$

(2.10)

For simplicity, we suppress the dependence on $\alpha \in \Gamma$ in the notation of the above matrices, which are defined through $\widehat{Z}_\alpha (t,\lambda )$ and $\widetilde{Z}_\alpha (t,\lambda )$. Note that the matrix $M(t,\lambda )$ in (2.10) depends also on $\beta \in \Gamma$. However, note that the resulting number of linearly independent square integrable solutions, i.e., the dimension of $\mathcal {N}(\lambda )$, does not depend on $\alpha$, see [44, Theorem 5.7]. In particular, $\dim \mathcal {N}(\lambda )=n+{{\mathrm{rank}}}R_+(\lambda )$, where ${{\mathrm{rank}}}R_+(\lambda )$ does not depend on $\alpha$.

In [44, Section 4] it is shown that $M(t,\lambda )$ is closely related to the matrices $M\in \mathbb {C}^{n\times n}$ lying on the boundary of the Weyl disk $D(t,\lambda )$, i.e., on the Weyl circle

$$\begin{aligned} C(t,\lambda ):=\{P(t,\lambda )+R(t,\lambda )\,\mathcal {U}R(t,\bar{\lambda })\mid \mathcal {U}\in \mathbb {U}\}, \quad \mathbb {U}:=\{\mathcal {U}\in \mathbb {C}^{n\times n}\mid \mathcal {U}^*\mathcal {U}=I\}. \end{aligned}$$

A subsequent limit of $M(t,\lambda )$ with $\beta =\beta (t)\in \Gamma$ as $t\rightarrow \infty$ is called the half-line Weyl–Titchmarsh function and denoted by $M_+(\lambda )$. Especially, if system ($\text {H}_\lambda$) is in the limit point case for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$, then we can take $\beta =\beta (t)\equiv (I,0)$ or $\beta =\beta (t)\equiv (0,I)$, which yields that

$$\begin{aligned} M_+(\lambda )=-\lim _{t\rightarrow \infty } \widetilde{X}_\alpha ^{-1}(t,\lambda )\,\widehat{X}_\alpha (t,\lambda )=-\lim _{t\rightarrow \infty } \widetilde{U}_\alpha ^{-1}(t,\lambda )\,\widehat{U}_\alpha (t,\lambda ), \end{aligned}$$

(2.11)

see [17, Theorem 3.1]. Note that in this case we have $R_+(\lambda )=0$ and consequently $D_+(\lambda )=\{P_+(\lambda )\}$ is a singleton. Therefore, in the limit point case the square integrable Weyl solution is uniquely determined by the matrix $M=M_+(\lambda )=P_+(\lambda )$. The invertibility of $\widetilde{X}_\alpha (t,\lambda )$ and $\widetilde{U}_\alpha (t,\lambda )$ for large t used in (2.11) is guaranteed by Hypothesis 2.3, as we prove in the following lemma. The analogous result for $\widehat{X}_\alpha (\cdot ,\lambda )$ and $\widehat{U}_\alpha (\cdot ,\lambda )$ can be shown by the same arguments, e.g., under Hypothesis 2.5 below.

Lemma 2.4

Let Hypothesis 2.3 be satisfied. Then for any $\lambda \in \mathbb {C}\backslash \mathbb {R}$ the $n\times n$ matrices $\widetilde{X}_\alpha (t,\lambda )$ and $\widetilde{U}_\alpha (t,\lambda )$ are invertible for all $t\ge b$, where $b\in (a,\infty )$ is given in Hypothesis 2.3.

Proof

Let $\lambda \in \mathbb {C}\backslash \mathbb {R}$ and $t \ge b$ be fixed. Then identity (2.3) yields

$$\begin{aligned} W[\widetilde{Z}_\alpha (t,\lambda ),\widetilde{Z}_\alpha (t,\lambda )]=-2i{{\mathrm{im}}}(\lambda )\int _a^t \widetilde{Z}_\alpha ^*(\tau ,\lambda )\,\mathcal {W}(\tau )\,\widetilde{Z}_\alpha (\tau ,\lambda )\,\mathrm {d}\tau . \end{aligned}$$

(2.12)

Assume that $\widetilde{X}_\alpha (t,\lambda )$ is singular, i.e., $\widetilde{X}_\alpha (t,\lambda )\,\xi =0$ for some $\xi \ne 0$. Since ${{\mathrm{rank}}}\widetilde{Z}_\alpha (t,\lambda )=n$ for all $t\in [a,\infty )$, it follows that $\widetilde{Z}_\alpha (\cdot ,\lambda )\,\xi$ is a nontrivial solution of ($\text {H}_\lambda$) and from identity (2.12) we get

$$\begin{aligned} 0= & {} \tfrac{i}{2}\,\,\xi ^*\,\big [\widetilde{X}_\alpha ^*(t,\lambda )\,\widetilde{U}_\alpha (t,\lambda )-\widetilde{U}_\alpha ^*(t,\lambda )\widetilde{X}_\alpha (t,\lambda )\big ]\,\xi \\= & {} {{\mathrm{im}}}(\lambda )\int _a^t \xi ^*\,\widetilde{Z}_\alpha ^*(\tau ,\lambda )\,\mathcal {W}(\tau )\,\widetilde{Z}_\alpha (\tau ,\lambda )\,\xi \,\mathrm {d}\tau , \end{aligned}$$

which contradicts Hypothesis 2.3. Similar argument shows the invertibility of $\widetilde{U}_\alpha (t,\lambda )$ for $t\ge b$. $\square$

The number of linearly independent square integrable solutions has been traditionally studied under the stronger hypothesis including all nontrivial solutions of system ($\text {H}_\lambda$), which we call the strong Atkinson condition (or definiteness condition), see [1, Inequality (9.1.6)]. The sufficiency of the weak Atkinson condition given in Hypothesis 2.3 was identified for the first time in [43, 44], and it enabled us to extend some related results to systems with jointly varying endpoints (such as the periodic or antiperiodic endpoints), see [44, Section 8].

Hypothesis 2.5

(Strong Atkinson condition) There exists $\lambda \in \mathbb {C}$ and $b\in (a,\infty )$ such that

$$\begin{aligned} \int _a^b z^*(t,\lambda )\,\mathcal {W}(t)\,z(t,\lambda )\,\mathrm {d}t>0 \end{aligned}$$

for any nontrivial solution of system ($\text {H}_\lambda$).

Again, if Hypothesis 2.5 is satisfied for some $\lambda \in \mathbb {C}$, then it is true for all $\lambda \in \mathbb {C}$ as shown in [3, Lemma 2.10]. It can be shown that the number of linearly independent square integrable solutions of ($\text {H}_\lambda$) is constant in the upper and lower half planes of $\mathbb {C}$, but these values may be different, i.e., it is possible $\dim \mathcal {N}(\lambda )\ne \dim \mathcal {N}(\bar{\lambda })$, see [26, Proposition 2.20] and [28]. It is easy to see that if system ($\text {H}_\lambda$) has only real-valued coefficients, then $z(t,\bar{\lambda })=\overline{z(t,\lambda )}$ and so $\dim \mathcal {N}(\lambda )=\dim \mathcal {N}(\bar{\lambda })$ for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$. This equality also holds in the case of complex-valued coefficients if system ($\text {H}_\lambda$) is in the limit circle case. This result is known as the invariance of the limit circle case. More specifically, if there exists $\nu \in \mathbb {C}$ such that system ($\text {H}_{\nu }$) is in the limit circle case and

$$\begin{aligned} \int _a^\infty |{{{\mathrm{im}}}(\nu )\times {{\mathrm{tr}}}\mathcal {J}\mathcal {W}(t)}|\,\mathrm {d}t<\infty , \end{aligned}$$

(2.13)

then system ($\text {H}_\lambda$) is in the limit circle case for any $\lambda \in \mathbb {C}$, see [1, Theorem 9.11.2], [48, Theorem 2], and [45, Theorem 4.1].

2.3 Principal and antiprincipal solutions at infinity

Now we discuss the (non-)oscillatory behavior of system ($\text {H}_\lambda$) and the concept of an (anti-)principal solution at infinity. System ($\text {H}_\lambda$) is said to be completely controllable (or identically normal), if the trivial solution is the only solution with $x(t)\equiv 0$ on a non-degenerate subinterval of $[a,\infty )$. In the following simple lemma we illustrate the relationship between the controllability of system ($\text {H}_\lambda$) and Hypothesis 2.5. The assumptions of the statement are satisfied, e.g., for system ($\text {H}_\lambda$) corresponding to the second-order Sturm–Liouville differential equation, see Remark 4.2(iii) with $m=1$.

Lemma 2.6

Let $\mathcal {W}(t)={{\mathrm{diag}}}\{W_1(t),W_4(t)\}$ with $W_1(t)>0$ on a non-degenerate subinterval of $[a,\infty )$ and let system ($\text {H}_{\nu }$) be completely controllable for some $\nu \in \mathbb {C}$. Then Hypothesis 2.5 holds for $\lambda =\nu$, and consequently it holds for all $\lambda \in \mathbb {C}$.

Proof

Let us assume that Hypothesis 2.5 does not hold for the given $\nu$, i.e., there exists a nontrivial solution $z(\cdot ,\nu )$ such that $\int _a^b z^*(t,\nu )\,\mathcal {W}(t)\,z(t,\nu )\,\mathrm {d}t=0$ for all $b\in (a,\infty )$. Since $W_1(t)>0$ on some non-degenerate subinterval of $[a,\infty )$, it follows that $x(t,\nu )\equiv 0$ on this subinterval. But then the controllability of system ($\text {H}_{\nu }$) implies $z(t,\nu )\equiv 0$ on $[a,\infty )$. Hence we get a contradiction, which means that Hypothesis 2.5 holds. The second part follows immediately from the comment following Hypothesis 2.5.$\square$

System ($\text {H}_\lambda$), for which the completely controllability assumption is not required, is called abnormal. In such case we define the maximal order of abnormality of ($\text {H}_\lambda$) as

$$\begin{aligned} d_\infty (\lambda ):= \max _{t\in [a,\infty )}d_\lambda [t,\infty )\le n, \end{aligned}$$

where $d_\lambda [c,\infty )$ denotes the dimension of the linear space of n-dimensional vector-valued functions $u(\cdot ,\lambda )\in \mathrm {C}_p^1$ such that the pair $x(t,\lambda )\equiv 0$ and $u(\cdot ,\lambda )$ solves system ($\text {H}_\lambda$) on $[c,\infty )$. Obviously, $d_\infty (\lambda )=0$ in the controllable case. A solution $Z(\cdot ,\lambda )$ with constant kernel of $X(t,\lambda )$ on some interval $[c,\infty )$ is called a minimal conjoined basis on $[c,\infty )$, if it is a conjoined basis and ${{\mathrm{rank}}}X(t,\lambda )=n-d_\lambda [c,\infty )$ on $[c,\infty )$. Similarly, $Z(\cdot ,\lambda )$ is a maximal conjoined basis on $[c,\infty )$, if it is a conjoined basis and ${{\mathrm{rank}}}X(t,\lambda )=n$ for all $t\in [c,\infty )$, see [35, Section 5] for more details.

For the rest of this section we fix $\lambda \in \mathbb {R}$ such that the Legendre condition ($\text {LC}_\lambda$) is satisfied. In this case for any conjoined basis $Z(\cdot ,\lambda )$ of ($\text {H}_\lambda$) the points $t_0\in [a,\infty )$, where the kernel of $X(\cdot ,\lambda )$ changes, are isolated, by [24, Theorem 3]. System ($\text {H}_\lambda$) is called nonoscillatory if there exists a conjoined basis $Z(\cdot ,\lambda )$ with finite number of proper focal points in $[a,\infty )$. Recall that $t_0\in (a,\infty )$ is a proper focal point of $Z(\cdot ,\lambda )$ if ${{\mathrm{Ker}}}X(t_0^-,\lambda )\subsetneqq {{\mathrm{Ker}}}X(t_0,\lambda )$, see [41, 42] and the references therein. In the opposite case system ($\text {H}_\lambda$) is called oscillatory. This notion then does not depend on the choice of the conjoined basis $Z(\cdot ,\lambda )$, by [41, Theorem 2.2].

In the oscillation theory the concept of a principal solution at infinity is used in order to indicate eventually the smallest one at infinity among all solutions of the given differential equation. We refer to [7, 25, 29] for this notion for second-order scalar differential equations, to [15] for second-order matrix differential equations, and to [31, 33] for system ($\text {H}_{0}$) under the controllability assumption. For abnormal system ($\text {H}_\lambda$) the principal solution at infinity is defined as a conjoined basis $Z(\cdot ,\lambda )$ with constant kernel of $X(t,\lambda )$ on $[c,\infty )$ for some $c\in [a,\infty )$ such that

$$\begin{aligned} \lim _{t\rightarrow \infty }S_c^\dagger (t,\lambda )=0,\quad \text {where}\quad S_c(t,\lambda ):=\int ^t_c X^\dagger (\tau ,\lambda )\,\mathcal {B}(\tau ,\lambda )\,X^{\dagger *}(\tau ,\lambda )\,\mathrm {d}\tau , \end{aligned}$$

(2.14)

see [37, Definition 7.1]. It is known in [37, Theorem 7.6] that the nonoscillation of ($\text {H}_\lambda$) is equivalent with the existence of a principal solution $Z_p(\cdot ,\lambda )$ of ($\text {H}_\lambda$), such that the rank $r_p(\lambda )$ of $X_p(t,\lambda )$ for t large enough lies between the numbers $n-d_\infty (\lambda )$ and n. In addition, the principal solution $Z_p(\cdot ,\lambda )$ is said to be minimal if $r_p(\lambda )=n-d_\infty (\lambda )$, it is said to be maximal if $r_p(\lambda )=n$, and for the remaining values $r_p(\lambda )\in \{n-d_\infty (\lambda )+1,\dots ,n-1\}$ it is called intermediate. The minimal and maximal principal solutions of system ($\text {H}_\lambda$) are, respectively, denoted by $Z^{[\mathrm {min}]}_p(\cdot ,\lambda )$ and $Z^{[\mathrm {max}]}_p(\cdot ,\lambda )$. The minimal principal solution $Z^{[\mathrm {min}]}_p(\cdot ,\lambda )$ is unique up to a right nonsingular constant multiple, by [35, Theorem 7.6]. One easily observes that in the controllable case all principal solutions coincide (up to a constant nonsingular multiple) with the minimal principal solution $Z^{[\mathrm {min}]}_p(\cdot ,\lambda )$.

Similarly, an antiprincipal solution at infinity of system ($\text {H}_\lambda$) is defined as a conjoined basis $Z(\cdot ,\lambda )$ of ($\text {H}_\lambda$) such that ${{\mathrm{Ker}}}X(t,\lambda )$ is constant on $[c,\infty )$ for some $c\in [a,\infty )$ and

$$\begin{aligned} {{\mathrm{rank}}}T(\lambda )=n-d_\infty (\lambda ),\quad \text {where}\quad T(\lambda ):=\lim _{t\rightarrow \infty }S_c^\dagger (t,\lambda ), \end{aligned}$$

see [36, Definition 5.1]. Since again the rank $r_a(\lambda )$ of $X(t,\lambda )$ for large t lies between $n-d_\infty (\lambda )$ and n, we obtain in the case of $r_a(\lambda )=n-d_\infty (\lambda )$ the notion of a minimal antiprincipal solution, while for $r_a(\lambda )=n$ we obtain the notion of a maximal antiprincipal solution. Minimal and maximal antiprincipal solutions of ($\text {H}_\lambda$) will be denoted by $Z^{[\mathrm {min}]}_a(\cdot ,\lambda )$ and $Z^{[\mathrm {max}]}_a(\cdot ,\lambda )$, respectively. By [36, Theorem 5.3 and Remark 5.4], antiprincipal solutions of ($\text {H}_\lambda$) are characterized by the property that

$$\begin{aligned} \lim _{t\rightarrow \infty }S_c(t,\lambda ) \quad \text {exists}, \end{aligned}$$

(2.15)

in which case this limit is necessarily equal to $T^\dagger (\lambda )$. Moreover, by [36, Theorem 5.8] the nonoscillation of system ($\text {H}_\lambda$) is equivalent with the existence of an antiprincipal solution $Z_a(\cdot ,\lambda )$ with $r_a(\lambda ):={{\mathrm{rank}}}X_a(t,\lambda )$ for large t lying between $n-d_\infty (\lambda )$ and n. Condition (2.15) then implies that

$$\begin{aligned} \Lambda _{\mathrm {max}}\big (\lim _{t\rightarrow \infty } S_c(t,\lambda )\big )<\infty ,\quad \text {i.e.,}\quad \Vert {\lim _{t\rightarrow \infty } S_c(t,\lambda )}\Vert _\sigma =\lim _{t\rightarrow \infty }\Vert {S_c(t,\lambda )}\Vert _\sigma <\infty \end{aligned}$$

(2.16)

for any antiprincipal solution $Z(\cdot ,\lambda )$ of ($\text {H}_\lambda$).

In the following statement we recall a limit characterization of the minimal principal solution $Z^{[\mathrm {min}]}_p(\cdot ,\lambda )$, which can be found in [38, Corollary 5.5].

Proposition 2.7

Let $\lambda \in \mathbb {R}$ be such that condition ($\text {LC}_\lambda$) holds and system ($\text {H}_\lambda$) is nonoscillatory. In addition, let $Z^{[\mathrm {min}]}(\cdot ,\lambda )$ and $Z^{[\mathrm {max}]}(\cdot ,\lambda )$ be minimal and maximal conjoined bases of system ($\text {H}_\lambda$) such that ${{\mathrm{Ker}}}X^{[\mathrm {min}]}(t,\lambda )$ is constant and $X^{[\mathrm {max}]}(t,\lambda )$ is invertible on $[c,\infty )$ for some $c\in [a,\infty )$. Define the constant $n\times n$ matrices $N:= W[Z^{[\mathrm {min}]}(t,\lambda ),Z^{[\mathrm {max}]}(t,\lambda )]$ for some (and hence) any $t\in [a,\infty )$ and $L:= \lim _{t\rightarrow \infty } S_c^{[\mathrm {max}]}(t,\lambda )[S_c^{[\mathrm {max}]}(t,\lambda )]^\dag$ with $S_c^{[\mathrm {max}]}(\cdot ,\lambda )$ given in (2.14) through $X^{[\mathrm {max}]}(\cdot ,\lambda )$. Then $Z^{[\mathrm {min}]}(\cdot ,\lambda )$ is the minimal principal solution and ${{\mathrm{rank}}}NL=n-d_\infty (\lambda )$ if and only if

$$\begin{aligned} \lim _{t\rightarrow \infty } [X^{[\mathrm {max}]}(t,\lambda )]^{-1}X^{[\mathrm {min}]}(t,\lambda )=0. \end{aligned}$$

(2.17)

In this case $Z^{[\mathrm {max}]}(\cdot ,\lambda )$ is an antiprincipal solution of ($\text {H}_\lambda$).

Finally, it was shown in [39, Theorem 1] that the minimal principal solution of system ($\text {H}_\lambda$) can be obtained by using the Reid construction as stated in the following theorem, see also [32, Theorem 6.1].

Proposition 2.8

Let $\lambda \in \mathbb {R}$ be such that condition ($\text {LC}_\lambda$) holds and system ($\text {H}_\lambda$) is nonoscillatory. Suppose that $Z^{[\mathrm {min}]}(\cdot ,\lambda )$ is a minimal conjoined basis of system ($\text {H}_\lambda$) on $[c,\infty )$, where $c\in [a,\infty )$ is such that $d_\lambda [c,\infty )=d_\infty (\lambda )$. If $Z(\cdot ,\lambda )$ is a conjoined basis of system ($\text {H}_\lambda$) satisfying

$$\begin{aligned} W[Z^{[\mathrm {min}]}(\cdot ,\lambda ),Z(t,\lambda )]\equiv I \quad \text {and}\quad [X^{[\mathrm {min}]}(c,\lambda )]^\dagger X(c,\lambda )=0, \end{aligned}$$

then $X(t,\lambda )$ is invertible for all t large enough and the solution $Z_\tau (\cdot ,\lambda )$ given by the initial conditions

$$\begin{aligned} X_\tau (\tau ,\lambda )=0,\quad U_\tau (\tau ,\lambda )=-X^{*-1}(\tau ,\lambda ) \end{aligned}$$

is a conjoined basis satisfying

$$\begin{aligned} Z^{[\mathrm {min}]}_p(t,\lambda )=\lim _{\tau \rightarrow \infty } Z_\tau (t,\lambda ) \quad \text {for all }\ t\in [a,\infty ). \end{aligned}$$

For completeness we note that the conjoined basis $Z(\cdot ,\lambda )$ from Proposition 2.8 is a maximal antiprincipal solution of system ($\text {H}_\lambda$), see [39, Proposition 1].

3 Weyl solution on real line

In this section we study the problem of extending the Weyl solution $\mathcal {X}(\cdot ,\lambda )$ to the real values of $\lambda$, when system ($\text {H}_\lambda$) is in the limit point case on $\mathbb {C}\backslash \mathbb {R}$. This means that we show a connection of the Weyl solution with the minimal principal solution $Z_p^{[\mathrm {min}]}(\cdot ,\lambda )$. In addition, we investigate the square integrability of the columns of $Z_p^{[\mathrm {min}]}(\cdot ,\lambda )$.

First we present a natural extension of the Reduction of Order Theorem to system ($\text {H}_\lambda$), compare with [8, Proposition 1, pg. 35].

Theorem 3.1

(Reduction of order) Let $\lambda \in \mathbb {C}$ be fixed. Let $Z(\cdot ,\lambda )$ be a solution of system ($\text {H}_\lambda$) such that (2.4) holds and suppose that $X(t,\lambda )$ and $X(t,\bar{\lambda })$ are invertible on $[c,\infty )$ for some $c\in [a,\infty )$. Then the $2n\times n$ matrix function $\underline{Z}(\cdot ,\lambda )$ determined for $t\in [c,\infty )$ by the blocks

$$\begin{aligned} \underline{X}(t,\lambda ):= X(t,\lambda )\,[M+S_c(t,\lambda )\,N],\quad \underline{U}(t,\lambda ):= U(t,\lambda )\,[M+S_c(t,\lambda )\,N]+X^{*-1}(t,\bar{\lambda })\,N,\nonumber \\ \end{aligned}$$

(3.1)

where $M,N\in \mathbb {C}^{n\times n}$ are arbitrary constant matrices and $S_c(t,\lambda )$ is defined in (2.14), solves system ($\text {H}_\lambda$) and satisfies

$$\begin{aligned} W[Z(t,\bar{\lambda }),\underline{Z}(t,\lambda )]\equiv N,\quad W[\underline{Z}(t,\bar{\lambda }),\underline{Z}(t,\lambda )]\equiv M^*N-N^*M. \end{aligned}$$

Proof

The proof is based on straightforward calculations similarly as in the classical Reduction of Order Theorem, see [8, Proposition 1, pg. 35]. In the proof we utilize the first and third equalities in (2.2) and the identity $U(t,\bar{\lambda })\,X^{-1}(t,\bar{\lambda })=X^{*-1}(t,\lambda )\,U^*(t,\lambda )$, which follows from (2.4).$\square$

Next we derive two formulas for the half-line Weyl–Titchmarsh function $M_+(\lambda )$ from (2.11) and discuss their form for the Sturm–Liouville differential equations. If Hypothesis 2.3 holds, then the solution $\widetilde{Z}_\alpha (\cdot ,\lambda )$ defined in (2.5) satisfies the assumptions of Theorem 3.1 with $c:=b$ and $\lambda \in \mathbb {C}\backslash \mathbb {R}$, as it is shown in Lemma 2.4. Then by (3.1) with $Z(\cdot ,\lambda ):=\widetilde{Z}_\alpha (\cdot ,\lambda )$ and $M:=0$, $N:=I$ we get

$$\begin{aligned} \underline{X}(t,\lambda ):= \widetilde{X}_\alpha (t,\lambda )\,\widetilde{S}_b(t,\lambda ),\quad \underline{U}(t,\lambda ):= \widetilde{U}_\alpha (t,\lambda )\,\widetilde{S}_b(t,\lambda )+\widetilde{X}_\alpha ^{*-1}(t,\bar{\lambda }),\quad t\ge b, \end{aligned}$$

where $\widetilde{S}_b(\cdot ,\lambda )$ denotes the S-matrix defined similarly as in (2.14) through the matrix $\widetilde{X}_\alpha (\cdot ,\lambda )$. In this case the (columns of the) solutions $\underline{Z}(\cdot ,\lambda )$ and $\widetilde{Z}_\alpha (\cdot ,\lambda )$ are linearly independent. Indeed, let us assume that the matrix

$$\begin{aligned} \Omega (t):=\left( \begin{array}{ll}\underline{X}(t,\lambda ) &{} \widetilde{X}_\alpha (t,\lambda )\\ \underline{U}(t,\lambda ) &{} \widetilde{U}_\alpha (t,\lambda )\end{array}\right) \end{aligned}$$

is not invertible. Then there exists $\xi =(\xi _1^\top ,\xi _2^\top )^\top \in \mathbb {C}^{2n}\backslash \{0\}$ such that $\Omega (t)\,\xi =0$ for some $t\ge b$, which is equivalent with the conditions

$$\begin{aligned} \widetilde{X}_\alpha (t,\lambda )\,[\widetilde{S}_b(t,\lambda )\,\xi _1+\xi _2]=0 \quad \text {and}\quad \widetilde{U}_\alpha (t,\lambda )\,[\widetilde{S}_b(t,\lambda )\,\xi _1+\xi _2]+\widetilde{X}_\alpha ^{*-1}(t,\bar{\lambda })\,\xi _1=0. \end{aligned}$$

However, the invertibility of $\widetilde{X}_\alpha (t,\lambda )$ implies $\widetilde{S}_b(t,\lambda )\,\xi _1+\xi _2=0$, which upon substitution into the second equation yields that $\xi _1=0$, and consequently $\xi _2=0$. Therefore, the pair $\underline{Z}(\cdot ,\lambda ),\widetilde{Z}_\alpha (\cdot ,\lambda )$ forms a fundamental system of solutions of ($\text {H}_\lambda$), which means that there exist matrices $V_1,V_2\in \mathbb {C}^{n\times n}$ such that

$$\begin{aligned} \widehat{Z}_\alpha (t,\lambda )\equiv \underline{Z}(t,\lambda )\,V_1+\widetilde{Z}_\alpha (t,\lambda )\,V_2,\quad t\ge b. \end{aligned}$$

(3.2)

If we put $t=b$ in (3.2), we obtain by using (2.6) that $V_1=-I$ and $V_2=\widetilde{X}_\alpha ^{-1}(b,\lambda )\,\widehat{X}_\alpha (b,\lambda )$. Therefore, the function $M_+(\lambda )$ given in (2.11) can be equivalently written as

$$\begin{aligned} M_+(\lambda )&=-\lim _{t\rightarrow \infty }\widetilde{X}_\alpha ^{-1}(t,\lambda )\,[\underline{X}(t,\lambda )\,V_1+\widetilde{X}_\alpha (t,\lambda )\,V_2] =-V_2+\lim _{t\rightarrow \infty }\widetilde{S}_b(t,\lambda )\nonumber \\&=-\widetilde{X}_\alpha ^{-1}(b,\lambda )\,\widehat{X}_\alpha (b,\lambda ) +\int _b^\infty \widetilde{X}_\alpha ^{-1}(t,\lambda )\,\mathcal {B}(t,\lambda )\,\widetilde{X}_\alpha ^{*-1}(t,\bar{\lambda })\,\mathrm {d}t,\qquad \lambda \in \mathbb {C}\backslash \mathbb {R}. \end{aligned}$$

(3.3)

Similarly, the associated conjoined basis $\widehat{Z}_\alpha (\cdot ,\lambda )$ satisfies the assumptions of Theorem 3.1 for example when Hypothesis 2.5 is satisfied. In that case we get from (3.1) with the choices $Z(\cdot ,\lambda ):=\widehat{Z}_\alpha (\cdot ,\lambda )$ and $M:=0$, $N:=I$ that

$$\begin{aligned} \underline{X}(t,\lambda ):= \widehat{X}_\alpha (t,\lambda )\,\widehat{S}_b(t,\lambda ),\quad \underline{U}(t,\lambda ):= \widehat{U}_\alpha (t,\lambda )\,\widehat{S}_b(t,\lambda )+\widehat{X}_\alpha ^{*-1}(t,\bar{\lambda }),\quad t\ge b, \end{aligned}$$

where $\widehat{S}_b(\cdot ,\lambda )$ is defined as in (2.14) through $\widehat{X}_\alpha (\cdot ,\lambda )$. Then the solutions $\underline{Z}(\cdot ,\lambda )$ and $\widehat{Z}_\alpha (\cdot ,\lambda )$ are linearly independent, which means that $\widetilde{Z}_\alpha (t,\lambda )\equiv \underline{Z}(t,\lambda )\,V_1+\widehat{Z}_\alpha (t,\lambda )\,V_2$ for all $t\ge b$ and some matrices $V_1,V_2\in \mathbb {C}^{n\times n}$. Since by using (2.7) we obtain $V_1=I$ and $V_2=\widehat{X}_\alpha ^{-1}(b,\lambda )\,\widetilde{X}_\alpha (b,\lambda )$, the function $M_+(\lambda )$ given in (2.11) has also the form

$$\begin{aligned} M_+(\lambda )&=-\lim _{t\rightarrow \infty }[\underline{X}(t,\lambda )\,V_1+\widehat{X}_\alpha (t,\lambda )\,V_2]^{-1}\widehat{X}_\alpha (t,\lambda ) =-\lim _{t\rightarrow \infty }[V_2+\widehat{S}_b(t,\lambda )]^{-1}\nonumber \\&=-\lim _{t\rightarrow \infty }\bigg [\widehat{X}_\alpha ^{-1}(b,\lambda )\,\widetilde{X}_\alpha (b,\lambda ) +\int _b^t \widehat{X}_\alpha ^{-1}(\tau ,\lambda )\,\mathcal {B}(\tau ,\lambda )\,\widehat{X}_\alpha ^{*-1}(\tau ,\lambda )\,\mathrm {d}\tau \bigg ]^{-1}, \quad \lambda \in \mathbb {C}\backslash \mathbb {R}. \end{aligned}$$

(3.4)

Formulas (3.3) and (3.4) are new in the context of abnormal as well as completely controllable linear Hamiltonian systems ($\text {H}_\lambda$).

Remark 3.2

From identities (3.3) and (3.4) we can derive new formulas for the function $M_+(\lambda )$ associated with the even order n-vector-valued Sturm–Liouville differential equation

$$\begin{aligned} \left. \begin{aligned}&(-1)^m\,[P_m(t)\,y^{(m)}(t,\lambda )]^{(m)}+(-1)^{m-1}\,[P_{m-1}(t)\, y^{(m-1)}(t,\lambda )]^{(m-1)}\\&+\dots +[P_2(t)\,y''(t,\lambda )]''-[P_1(t)\,y'(t,\lambda )]'+P_0(t)\,y(t,\lambda ) =\lambda W(t)\,y(t,\lambda ), \end{aligned} \,\right\} \end{aligned}$$

(3.5)

where $m\in \mathbb {N}$, $P_0(t),\dots ,P_m(t),W(t)\in \mathbb {C}^{n\times n}$, $W(t)=W^*(t)\ge 0$, $P_0(t),\dots ,P_m(t)$ are Hermitian, $P_m(t)$ is invertible, and $y(t,\lambda )\in \mathbb {C}^n$ for all $t\in [a,\infty )$ and $\lambda \in \mathbb {C}$. Equation (3.5) is equivalent with system ($\text {H}_\lambda$), where

$$\begin{aligned} \left. \begin{array}{ll} A(t)\equiv \left( \begin{array}{lllll}0 &{}\quad I &{}\quad 0 &{}\quad \dots &{}\quad 0\\ 0 &{}\quad 0 &{}\quad I &{}\quad \dots &{}\quad 0\\ \vdots &{}\quad \vdots &{}\quad \ddots &{}\quad \ddots &{}\quad \vdots \\ 0 &{}\quad 0 &{}\quad 0 &{}\quad \dots &{}\quad I\\ 0 &{}\quad 0 &{}\quad 0 &{}\quad \dots &{}\quad 0\end{array}\right) , &{} \begin{array}{rl} B(t) &{}={{\mathrm{diag}}}\{0,\dots ,0,P^{-1}_m(t)\}, \\ C(t) &{}={{\mathrm{diag}}}\{P_0(t),P_1(t),\dots ,P_{m-1}(t)\},\\ W_1(t) &{}={{\mathrm{diag}}}\{W(t),0,\dots ,0\},\\ W_2(t) &{}=W_4(t)\equiv 0 \end{array} \end{array} \right\} \qquad \quad \end{aligned}$$

(3.6)

with $A(t),B(t),C(t),W_1(t),W_2(t),W_4(t)\in \mathbb {C}^{nm\times nm}$, see e.g., [8, Section 2.7] for more details. Observe that in this special case the corresponding Legendre condition ($\text {LC}_\lambda$) does not depend on $\lambda$. If $W(t)>0$ on $[a,\infty )$, then Hypothesis 2.5 is satisfied for any $b>a$, which implies that $\widetilde{X}_{(0,I)}(t,\lambda )$ is invertible on $[a,\infty )$ for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$, see Lemma 2.4. Then for the system ($\text {H}_\lambda$), and consequently also for equation (3.5), being in the limit point case, we obtain from (3.3) that

$$\begin{aligned} M_+(\lambda ) =\int _a^\infty \widetilde{X}_{(0,I)}^{-1}(t,\lambda )\,{{\mathrm{diag}}}\big \{0,\dots ,0,P^{-1}_m(t)\big \}\,\widetilde{X}_{(0,I)} ^{*-1}(t,\bar{\lambda })\,\mathrm {d}t, \quad \lambda \in \mathbb {C}\backslash \mathbb {R}, \end{aligned}$$

because $\widetilde{X}_{(0,I)}^{-1}(a,\lambda )\,\widehat{X}_{(0,I)}(a,\lambda )=0$. Similarly, the matrix $\widehat{X}_I(t,\lambda )=\widehat{X}_{(I,0)}(t,\lambda )$ is invertible on $[a,\infty )\times \mathbb {C}\backslash \mathbb {R}$. Hence in the limit point case we obtain from (3.4) that

$$\begin{aligned} M_+(\lambda )=-\lim _{t\rightarrow \infty } \bigg [\int _a^t \widehat{X}_I^{-1}(\tau ,\lambda )\,{{\mathrm{diag}}}\{0,\dots ,0,P^{-1}_m(t)\}\,\widehat{X}_I^{*-1} (\tau ,\lambda )\,\mathrm {d}\tau \bigg ]^{-1}, \quad \lambda \in \mathbb {C}\backslash \mathbb {R}, \end{aligned}$$

(3.7)

because $\widehat{X}_I^{-1}(a,\lambda )\,\widetilde{X}_I(a,\lambda )=0$. In particular, identity (3.7) reduces to [7, Identity (117)] in the case of $m=1$, and to [7, Identity (58)] if $m=n=1$.

In the following result we describe a connection between the Weyl solution and the minimal principal solution of system ($\text {H}_\lambda$). This result will allow to extend formulas (3.3) and (3.4) for $M_+(\lambda )$ to certain real values of $\lambda$. Denote by $\mathcal {X}_+(\cdot ,\lambda )$ the Weyl solution defined as in (2.8) with the matrix $M:=M_+(\lambda )$. Then Hypothesis 2.3 guarantees that $\mathcal {X}_+(\cdot ,\lambda )$ is well defined for any $\lambda \in \mathbb {C}\backslash \mathbb {R}$. In this case we have all basic information about the number and structure of square integrable solutions as well as about the functions $\mathcal {X}_+(\cdot ,\lambda )$ and $M_+(\lambda )$. But the situation starts to be more complicated when $\lambda$ approaches the real line. This problem is solved in the limit point case in the following theorem, which generalizes [7, Theorems 2.13 and 3.11]. We also note that the construction utilized in the proof is slightly different than the one used in [7, Theorem 3.11].

Theorem 3.3

Let system ($\text {H}_\lambda$) be in the limit point case for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$. Let $\nu \in \mathbb {R}$ be such that ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is nonoscillatory. Moreover, let $\alpha \in \Gamma$ be such that Hypothesis 2.3 holds and the associated solution $\widehat{Z}_\alpha (\cdot ,\nu )$ defined in (2.5) is a minimal conjoined basis of ($\text {H}_{\nu }$) on $[c,\infty )$ with $\widehat{X}_\alpha ^\dag (c,\nu )\,\widetilde{X}_\alpha (c,\nu )=0$ for some $c\in [a,\infty )$. Then the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ can be extended to $\lambda =\nu$ and $\mathcal {X}_+(\cdot ,\nu )$ is the minimal principal solution of system ($\text {H}_{\nu }$).

Proof

Let the assumptions hold. Then for any $\lambda \in \mathbb {C}\backslash \mathbb {R}$ the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ satisfies

(3.8)

where, by Lemma 2.4, the solution $Z_\tau (\cdot ,\lambda )$ satisfies

$$\begin{aligned} Z_\tau (\tau ,\lambda )=\left( \begin{array}{ll}0\\ U_\alpha (\tau ,\lambda )-\widetilde{U}_\alpha (\tau ,\lambda )\,\widetilde{X}^{-1}(\tau ,\lambda )\,X_\alpha (\tau ,\lambda )\end{array}\right) \overset{(2.6)}{=}\left( \begin{array}{ll}0\\ -\widetilde{X}_\alpha ^{*-1}(\tau ,\bar{\lambda })\end{array}\right) \end{aligned}$$

for $\tau \ge b$. This means that $Z_\tau (\cdot ,\lambda )$ is the same solution as the one in Proposition 2.8. At the same time the assumptions of the theorem imply that the limit on the right-hand side of (3.8) exists also for $\lambda =\nu$ and is equal to $Z^{[\mathrm {min}]}_p(\cdot ,\nu )$, by Proposition 2.8. Therefore, by the continuous dependence of solutions on the spectral parameter $\lambda$, the function $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ represents an extension of the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ to $\lambda =\nu$.$\square$

Remark 3.4

• (i) The proof of Theorem 3.3 shows that the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ extends to the value $\lambda =\nu$, at which it coincides with the minimal principal solution $Z^{[\mathrm {min}]}_p(\cdot ,\nu )$ obtained from the Reid construction in Proposition 2.8.

• (ii) If the assumptions of Theorem 3.3 are satisfied for all $\nu \in (\lambda _1,\lambda _2)$, where $-\infty \le \lambda _1<\lambda _2\le \infty$, then the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ can be analytically extended to $\lambda \in (\lambda _1,\lambda _2)$ and for these values of $\lambda$ it is the minimal principal solution of ($\text {H}_\lambda$), compare with [33, Problem I.10.4, pg. 79].

The result in Theorem 3.3 is new even for the controllable system ($\text {H}_{\nu }$), in which case we obtain the following statement.

Corollary 3.5

Let system ($\text {H}_\lambda$) be in the limit point case for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$. Let $\nu \in \mathbb {R}$ be such that ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is completely controllable and nonoscillatory. Moreover, let $\alpha \in \Gamma$ be such that Hypothesis 2.3 holds and the associated conjoined basis $\widehat{Z}_\alpha (\cdot ,\nu )$ defined in (2.5) is such that $\widehat{X}_\alpha (t,\nu )$ is invertible on $[c,\infty )$ for some $c\in [a,\infty )$ and $\widetilde{X}_\alpha (c,\nu )=0$. Then the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ can be extended to $\lambda =\nu$ and $\mathcal {X}_+(\cdot ,\nu )$ is the minimal principal solution of system ($\text {H}_{\nu }$). Consequently, if the above assumptions are satisfied for all $\nu \in (\lambda _1,\lambda _2)$, where $-\infty \le \lambda _1<\lambda _2\le \infty$, then the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ can be analytically extended to $\lambda \in (\lambda _1,\lambda _2)$ and for these values of $\lambda$ it is the principal solution of ($\text {H}_\lambda$).

Remark 3.6

• (i) The application of Proposition 2.8 in the latter proof clarifies the importance of the role of $\alpha$ in the initial values for $\widehat{Z}_\alpha (\cdot ,\lambda )$ and $\widetilde{Z}_\alpha (\cdot ,\lambda )$. If $\nu \in \mathbb {R}$ is as in Theorem 3.3, then in general it is not possible to anticipate that for a particular choice of $\alpha$ the solution $\widehat{Z}_\alpha (\cdot ,\nu )$ will be a minimal conjoined basis. This fact is always true only in the controllable case. In addition, if system ($\text {H}_{\nu }$) is controllable, disconjugate, and $\det \widehat{X}_I(t,\nu )\ne 0$ for all $t\in [a,\infty )$ as in [7, Section 3], then we can take without lost of generality $\alpha =(I,0)$.

• (ii) If the assumptions of Theorem 3.3 and Proposition 2.7 are satisfied for the conjoined bases $Z^{[\mathrm {min}]}(\cdot ,\nu ):= Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ and $Z^{[\mathrm {max}]}(\cdot ,\nu ):=\widetilde{Z}_\alpha (\cdot ,\nu )$, then from the equality $Z_p^{[\mathrm {min}]}(t,\nu )=\mathcal {X}_+(t,\nu )$ and identity (2.17) we can derive the corresponding value of $M_+(\nu )$. More precisely, we have

  $$\begin{aligned} M_+(\nu )=-\lim _{t\rightarrow \infty } \widetilde{X}_\alpha ^{-1}(t,\nu )\,\widehat{X}_\alpha (t,\nu ). \end{aligned}$$

  (3.9)

  This implies together with [39, Identity (10)] that

  $$\begin{aligned} M_+(\nu )=-\lim _{t\rightarrow \infty } \widehat{S}_c^{\;\dag }(t,\nu ) =-\lim _{t\rightarrow \infty } \bigg [\int ^t_c \widehat{X}_\alpha ^{\;\dagger }(\tau ,\nu )\,\mathcal {B}(\tau ,\nu )\,\widehat{X}_\alpha ^{\;\dagger *}(\tau ,\nu )\,\mathrm {d}\tau \bigg ]^\dag , \end{aligned}$$

  (3.10)

  compare with (3.4). Let us also note that the condition ${{\mathrm{rank}}}NL=n-d_\infty (\nu )$ in Proposition 2.7 is trivially satisfied if $\det N\ne 0$, i.e., if the solutions $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ and $\widetilde{Z}_\alpha (\cdot ,\nu )$ are linearly independent. Especially, if $Z_p^{[\mathrm {min}]}(\cdot ,\nu )=\widehat{Z}_\alpha (\cdot ,\nu )$, then $M_+(\nu )=0$, which obviously agrees with formula (3.10), see (2.17) and (2.14).

• (iii) However, the additional assumptions in part (ii) are never satisfied in the controllable case. Fortunately, in that case it suffices to assume only the linear independence of $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ and $\widetilde{Z}_\alpha (\cdot ,\nu )$. Since $\widetilde{X}_\alpha (\cdot ,\nu )$ is invertible on $[d,\infty )$ for some $d\in [a,\infty )$, it follows that (3.9) holds by the definition of the (minimal) principal solution. The symplectic property of the fundamental matrix $\Phi _\alpha (t,\nu )$ then implies the equality

  $$\begin{aligned} \big [\widetilde{X}_\alpha ^{-1}(t,\nu )\,\widehat{X}_\alpha (t,\nu )\big ]'=-\widetilde{X}^{-1}_\alpha (t,\nu )\,\mathcal {B}(t,\nu )\,\widetilde{X}^{*-1}_\alpha (t,\nu ). \end{aligned}$$

  (3.11)

  Upon combining (3.9) and (3.11) we obtain the formula

  $$\begin{aligned} M_+(\nu )=-\widetilde{X}_\alpha ^{-1}(d,\nu )\,\widehat{X}_\alpha (d,\nu ) +\lim _{t\rightarrow \infty } \int ^t_d \widetilde{X}_\alpha ^{-1}(\tau ,\nu )\,\mathcal {B}(\tau ,\nu )\,\widetilde{X}_\alpha ^{*-1}(\tau ,\nu )\,\mathrm {d}\tau . \end{aligned}$$

  (3.12)

• (iv) Similar conclusion as in Theorem 3.3 is hidden, in some sense, in the exponential dichotomy for a class of linear Hamiltonian differential systems studied in [10,11,12, 18, 19] and recently in [21, Section 5.6].

The result of Theorem 3.3 is illustrated in the following example, where system ($\text {H}_\lambda$) is the $2\times 2$ Dirac system with constant real-valued coefficients.

Example 3.7

Let us consider the scalar system ($\text {H}_\lambda$) on $[0,\infty )$ with

$$\begin{aligned} \mathcal {H}(t,\lambda )=\left( \begin{array}{ll}p &{}\quad q+\lambda \\ r-\lambda &{}\quad -p\end{array}\right) , \quad \text {i.e.,}\quad \mathcal {H}(t)\equiv \left( \begin{array}{ll}p &{}\quad q\\ r &{}\quad -p \end{array}\right) , \quad \mathcal {W}(t)\equiv {{\mathrm{diag}}}\{1,1\}, \end{aligned}$$

(3.13)

where $p,q,r\in \mathbb {R}$ are given constants with $p>0$ and $q\ne -r$, compare with [47, Example 3.1]. If we choose $\alpha =(0,1)$, then we obtain the fundamental matrix $\Phi _{(0,1)}(t,\lambda )=\big (\widehat{Z}_{(0,1)}(t,\lambda ),\widetilde{Z}_{(0,1)}(t,\lambda )\big )$ of ($\text {H}_\lambda$) with (3.13) in the form

$$\begin{aligned} \Phi _{(0,1)}(t,\lambda ) =\left( \begin{array}{ll}\frac{q+\lambda }{\omega (\lambda )}\sinh (\omega (\lambda )\,t) &{}\quad -\cosh (\omega (\lambda )\,t)-\frac{p}{\omega (\lambda )}\sinh (\omega (\lambda )\,t)\\ \cosh (\omega (\lambda )\,t)-\frac{p}{\omega (\lambda )}\sinh (\omega (\lambda )\,t) &{}\quad -\frac{r-\lambda }{\omega (\lambda )}\sinh (\omega (\lambda )\,t)\end{array}\right) \end{aligned}$$

for all $\lambda \in \mathbb {C}\backslash \{\lambda _{1,2}\}$, where

$$\begin{aligned} \omega (\lambda ):= \sqrt{p^2+(q+\lambda )(r-\lambda )}, \quad \lambda _{1,2}:=\big (-q+r\pm \sqrt{(q+r)^2+4p^2}\big )/2, \end{aligned}$$

i.e., $\lambda _{1,2}\in \mathbb {R}$ are the zeros of $\omega (\lambda )$ and the principal square root is taken in $\omega (\lambda )$. Since we have $\widetilde{Z}_{(0,1)}^*(t,\lambda )\,\mathcal {W}(t)\,\widetilde{Z}_{(0,1)}(t,\lambda )>0$ on $[0,\infty )$, Hypothesis 2.3 is satisfied with any $b>0$. Moreover, one can easily see that $\widehat{Z}(\cdot ,\lambda ),\widetilde{Z}(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$ for any $\lambda \in \mathbb {C}\backslash \mathbb {R}$, so that system ($\text {H}_\lambda$) with (3.13) is in the limit point case for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$. Then by (2.11) we get for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$ that

$$\begin{aligned} M_+(\lambda )&=\lim _{t\rightarrow \infty }\frac{q+\lambda }{\omega (\lambda )\coth (\omega (\lambda )\,t)+p} =\frac{q+\lambda }{p+\omega (\lambda )}, \end{aligned}$$

because ${{\mathrm{re}}}\omega (\lambda )>0$ for $\lambda \in \mathbb {C}\backslash \mathbb {R}$ implies that $\lim _{t\rightarrow \infty }\coth (\omega (\lambda )\,t)=1$. Hence, by (2.8) we have for $\lambda \in \mathbb {C}\backslash \mathbb {R}$ the Weyl solution

$$\begin{aligned} \mathcal {X}_+(t,\lambda ) = \left( \begin{array}{ll}-\frac{q+\lambda }{\omega (\lambda )+p} \\ 1\end{array}\right) \,{{\mathrm{e}}}^{-\omega (\lambda )\,t},\quad \mathcal {X}_+(\cdot ,\lambda )\in \mathcal {L}^2_{\mathcal {W}}. \end{aligned}$$

(3.14)

Since the principal square root is not well defined for negative real numbers, the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ can be extended to $\mathbb {C}\backslash \big ((\infty ,\lambda _1)\cup (\lambda _2,\infty )\big )$ and on the interior of this region the function $\mathcal {X}_+(\cdot ,\lambda )$ is analytic and square integrable. On the other hand, for $\alpha =(1,0)$ we have

$$\begin{aligned} \Phi _I(t,\lambda ) =\left( \begin{array}{ll}\cosh (\omega (\lambda )\,t)+\frac{p}{\omega (\lambda )}\sinh (\omega (\lambda )\,t) &{}\quad \frac{q+\lambda }{\omega (\lambda )}\sinh (\omega (\lambda )\,t)\\ \frac{r-\lambda }{\omega (\lambda )}\sinh (\omega (\lambda )\,t) &{}\quad \cosh (\omega (\lambda )\,t) -\frac{p}{\omega (\lambda )}\sinh (\omega (\lambda )\,t)\end{array}\right) , \end{aligned}$$

(3.15)

which yields by (2.11) and (2.8) the functions

$$\begin{aligned} {\hat{M}}_+(\lambda )=-\frac{p+\omega (\lambda )}{q+\lambda }, \quad {\hat{\mathcal {X}}}_+(t,\lambda )=-\frac{\omega (\lambda )+p}{q+\lambda }\,\mathcal {X}_+(t,\lambda ). \end{aligned}$$

(3.16)

We can see that the Weyl solution ${\hat{\mathcal {X}}}_+(t,\lambda )$ is not defined for $\lambda =-q$.

Since the Legendre condition for ($\text {H}_\lambda$) is satisfied when $\lambda \ge -q$, the conclusion of Theorem 3.3 will hold on the interval $[-q,\lambda _2]\subseteq [\lambda _1,\lambda _2]$. System ($\text {H}_\lambda$) with (3.13) is controllable for $\lambda \in \mathbb {R}\backslash \{-q\}$ and nonoscillatory for $\lambda \in \mathbb {R}$ such that $\omega (\lambda )\ge 0$, i.e., for $\lambda \in [\lambda _1,\lambda _2]$.

If $\lambda \in (-q,\lambda _2)$ and $\alpha =(1,0)$, then $\widehat{Z}_I(\cdot ,\lambda )$, which is the first column of $\Phi _I(\cdot ,\lambda )$ in (3.15), satisfies the assumptions of Theorem 3.3 with $c=0$. This means that for these values of $\lambda$ the minimal principal solution $Z_p^{[\mathrm {min}]}(t,\lambda )$ coincides (up to a constant nonzero multiple) with the Weyl solution ${\hat{\mathcal {X}}}_+(t,\lambda )$ in (3.16).

Similarly, the assumptions of Theorem 3.3 are satisfied for $\lambda =-q$ with $c=0$ and $\alpha =(0,1)$. In this case, the order of abnormality is $d_\infty (-q)=1$ and the corresponding minimal principal solution is $Z_p^{[\mathrm {min}]}(t,-q)=(0,\,1)^{\top }{{\mathrm{e}}}^{-p\,t}$, which coincides with the Weyl solution $\mathcal {X}_+(t,-q)$ in (3.14).

Finally, for $\lambda =\lambda _2\ne -q$ we obtain the same result with the choice of $\alpha =(1,0)$, i.e., with

$$\begin{aligned} \Phi _I(t,\lambda _2) =\left( \begin{array}{ll}pt+1 &{}\quad (\lambda _2 +q)\,t\\ -\frac{p^2}{\lambda _2 +q}\,t &{}\quad 1-pt \end{array}\right) . \end{aligned}$$

In this case the minimal principal solution $Z_p^{[\mathrm {min}]}(t,\lambda _2)=(\lambda _2 +q,\,-p)^\top$ is again a constant multiple of the Weyl solution ${\hat{\mathcal {X}}}_+(t,\lambda _2)$ in (3.16), but $Z_p^{[\mathrm {min}]}(\cdot ,\lambda _2)\not \in \mathcal {L}^2_{\mathcal {W}}$. Let us also note that the previous choices of the matrices $\alpha$ are the only possibilities satisfying the conditions of Theorem 3.3 for $c=0$.

In addition, we can calculate the $M_+(\lambda )$ function according to formulas (3.10) and (3.12) from Remark 3.6(ii), because the additional assumptions are satisfied in all three cases. In particular, for $\lambda \in (-q,\lambda _2)$ we have $M_+(\lambda )=-[p+\omega (\lambda )]/(q+\lambda )$ by (3.12) with $d=1$, for $\lambda =\lambda _2$ we get $M_+(\lambda _2)=-p/(q+\lambda _2)$ by (3.12) with $d=1$, and for $\lambda =-q$ we get $M_+(-q)=0$ by (3.10), which agrees with the equality between $\widehat{Z}_{(0,1)}(t,-q)$ and $Z_p^{[\mathrm {min}]}(t,-q)$.

We note for completeness that for $\lambda \in [\lambda _1,-q)$ we cannot discuss principal solutions, as the Legendre condition ($\text {LC}_\lambda$) is not satisfied in this case. However, the corresponding Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ possesses similar properties as above.$\square$

In the last part of this section we focus on the square integrability of the columns of the minimal principal solution $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$. From the above result we know that $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ is essentially equal to the Weyl solution $\mathcal {X}_+(\cdot ,\nu )$, whenever the assumptions of Theorem 3.3 are satisfied. In the limit point case and under Hypothesis 2.3, the columns of the Weyl solution $\mathcal {X}_+(\cdot ,\lambda )$ for $\lambda \in \mathbb {C}\backslash \mathbb {R}$ form a basis of the space $\mathcal {N}(\lambda )$ of all square integrable solutions of ($\text {H}_\lambda$). At the same time, for $\lambda \in \mathbb {R}$ the number of square integrable solutions of ($\text {H}_\lambda$) can be less (but not greater) than n, see [26, Lemma 2.25]. In some very special cases, such as for equation (3.5) with $m=n=1$ in the limit point case, it is possible to obtain an information about the square integrability of the (minimal) principal solution whenever a square integrable solution exists. The proof of this fact utilizes the following three tools: (i) the equality between the Weyl solution and the (minimal) principal solution, (ii) any solution is either minimal (and simultaneously maximal) principal or maximal (and simultaneously minimal) antiprincipal, and (iii) the limit characterization of principal solutions, see [7, Theorem 2.13] and also [49, Theorem 3.5].

In the next result we establish an analogous result for system ($\text {H}_\lambda$) with $W_2(t)=W_4(t)\equiv 0$ on $[a,\infty )$. This restriction is a direct consequence of the fact that the properties of (anti-)principal solutions are related to the block $X(\cdot ,\lambda )$, while the square integrability depends in general also on the block $U(\cdot ,\lambda )$. We note that the result below does not rely on Theorem 3.3 or on the limit point case. Its last part can be used as a simple limit circle test for ($\text {H}_\lambda$).

Theorem 3.8

Let $W_2(t)=W_4(t)\equiv 0$ on $[a,\infty )$. Suppose that $\nu \in \mathbb {R}$ is such that ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is nonoscillatory. Let $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ and $Z_a^{[\mathrm {max}]}(\cdot ,\nu )$ be the minimal principal solution and a maximal antiprincipal solution of ($\text {H}_{\nu }$) such that ${{\mathrm{rank}}}NL=n-d_\infty (\nu )$, where the matrices N and L are defined in Proposition 2.7. If all columns of $Z_a^{[\mathrm {max}]}(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$, then all columns of $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ do as well and consequently, system ($\text {H}_{\nu }$) has at least $2n-d_\infty (\nu )$ linearly independent solutions in $\mathcal {L}^2_{\mathcal {W}}$, i.e., $\dim \mathcal {N}(\nu )\ge 2n-d_\infty (\nu )$. Especially, if the (columns of the) two solutions $Z_p^{[\mathrm {min}]}(\cdot ,\nu )$ and $Z_a^{[\mathrm {max}]}(\cdot ,\nu )$ are linearly independent, then system ($\text {H}_\lambda$) is in the limit circle case for all $\lambda \in \mathbb {C}$ if and only if all columns of $Z_a^{[\mathrm {max}]}(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$.

Proof

For simplicity we abbreviate $Z_p(t,\nu ):= Z_p^{[\mathrm {min}]}(t,\nu )$ and $Z_a(t,\nu ):= Z_a^{[\mathrm {max}]}(t,\nu )$ for $t\in [a,\infty )$. Then Proposition 2.7 yields that

$$\begin{aligned} \lim _{t\rightarrow \infty } X_a^{-1}(t,\nu )\,X_p(t,\nu )=0, \quad \text {or equivalently}\quad \lim _{t\rightarrow \infty } \Vert {X_a^{-1}(t,\nu )\,X_p(t,\nu )}\Vert _\sigma =0. \end{aligned}$$

This means that there exists $\omega >0$ such that

$$\begin{aligned} \Vert {X_a^{-1}(t,\nu )\,X_p(t,\nu )}\Vert _\sigma \le \omega , \quad \text {or equivalently}\quad \Vert {X_p^*(t,\nu )\,X_a^{*-1}(t,\nu )}\Vert _\sigma \le \omega , \end{aligned}$$

for all $t\in [c,\infty )$ with $c\in [a,\infty )$ large enough. If we define the Hermitian matrix

$$\begin{aligned} \Upsilon (t) := X_a^{-1}(t,\nu )\,X_p(t,\nu )\,X_p^*(t,\nu )\,X_a^{*-1}(t,\nu ),\quad t\in [c,\infty ), \end{aligned}$$

then we have $\Lambda _{\mathrm {max}}[\Upsilon (t)]\le \omega ^2$. This implies that $\Upsilon (t)\le \Lambda _{\mathrm {max}}[\Upsilon (t)]\,I\le \omega ^2 I$. Thus, it holds

$$\begin{aligned} X_p(t,\nu )\,X_p^*(t,\nu )\le \omega ^2 X_a(t,\nu )\,X_a^*(t,\nu ). \end{aligned}$$

Upon multiplying the latter equality by $W_1^{1/2}(t)\ge 0$ from both sides and using the fact that the value of $\Lambda _{\mathrm {max}}(\cdot )$ preserves the ordering of Hermitian matrices, we obtain

$$\begin{aligned} \Vert {W^{1/2}(t)\,X_p(t,\nu )}\Vert _\sigma \le \Vert {W^{1/2}(t)\,X_a(t,\nu )}\Vert _\sigma . \end{aligned}$$

(3.17)

As a consequence of the special structure of $\mathcal {W}(\cdot )$, it follows that all columns of $Z_a(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$ if and only if $\int _a^\infty \Vert {W^{1/2}(t)\,X_a(t,\nu )}\Vert _\sigma \,\mathrm {d}t<\infty$. But in that case also $\int _a^\infty \Vert {W^{1/2}(t)\,X_p(t,\nu )}\Vert _\sigma \,\mathrm {d}t<\infty$, by (3.17), which yields that all columns of $Z_p(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$. Since ${{\mathrm{rank}}}NL=n-d_\infty (\nu )$ is assumed, it follows that

$$\begin{aligned} {{\mathrm{rank}}}W[Z_p(t,\nu ),Z_a(t,\nu )]\equiv {{\mathrm{rank}}}N\ge n-d_\infty (\nu ). \end{aligned}$$

This means that at least $n-d_\infty (\nu )$ columns of $Z_p(\cdot ,\nu )$ and $Z_a(\cdot ,\nu )$ are linearly independent, and by the previous part these columns belong to $\mathcal {L}^2_{\mathcal {W}}$. Therefore, the number of square integrable solutions of system ($\text {H}_{\nu }$) is at least $2n-d_\infty (\nu )$. In particular, if the solutions $Z_p(\cdot ,\nu )$ and $Z_a(\cdot ,\nu )$ are linearly independent and if all columns of $Z_a(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$, then all columns of $Z_p(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$ as well and hence $\dim \mathcal {N}(\nu )=2n$. The invariance of the limit circle case then yields $\dim \mathcal {N}(\lambda )=2n$ for all $\lambda \in \mathbb {C}$. The converse of the stated equivalence follows from the definition of the limit circle case. $\square$

For controllable system ($\text {H}_{\nu }$) we obtain from Theorem 3.8 the following result.

Corollary 3.9

Let $W_2(t)=W_4(t)\equiv 0$ on $[a,\infty )$. Suppose that $\nu \in \mathbb {R}$ is such that ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is completely controllable and nonoscillatory. Let $Z_a(\cdot ,\nu )$ be an antiprincipal of ($\text {H}_{\nu }$) such that all its columns belong to $\mathcal {L}^2_{\mathcal {W}}$. Then system ($\text {H}_\lambda$) is in the limit circle case for all $\lambda \in \mathbb {C}$.

Proof

If system ($\text {H}_{\nu }$) is completely controllable and nonoscillatory, then its principal and antiprincipal solutions $Z_p(\cdot ,\nu )$ and $Z_a(\cdot ,\nu )$ are linearly independent. Therefore, if all columns of $Z_a(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$, then system ($\text {H}_\lambda$) is in the limit circle case for all $\lambda \in \mathbb {C}$ by the last part of Theorem 3.8.$\square$

In the scalar and uncontrollable case, i.e., for $n=1$ and $d_\infty (\nu )=1$, the conclusion of Theorem 3.8 is trivial. On the other hand, in the scalar and controllable case, i.e., for $n=1$ and $d_\infty (\nu )=0$, the result in Theorem 3.8 or in Corollary 3.9 yields that the existence of a square integrable solution of ($\text {H}_{\nu }$) always implies the square integrability of the principal solution of this system. Especially, in the limit point case we obtain the following generalization of [7, Theorem 2.13], see also equation (3.5) with $m=n=1$.

Corollary 3.10

Let $n=1$ and $W_2(t)=W_4(t)\equiv 0$ on $[a,\infty )$. Suppose that $\nu \in \mathbb {R}$ is such that ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is completely controllable, nonoscillatory, and in the limit point case. If ($\text {H}_{\nu }$) possesses a square integrable solution, then it is its principal solution.

4 Non-limit circle case criterion

In this section we establish a non-limit circle criterion for system ($\text {H}_\lambda$), i.e., a sufficient condition for the existence of a solution which does not belong to $\mathcal {L}^2_{\mathcal {W}}$. The presented result is new even in the controllable case and it generalizes one of the classical limit point criteria for the second-order Sturm–Liouville differential equations in [30, Theorem 4.1], see also [14] and [9, Theorem 11.6]. Here we assume that the weight matrix $\mathcal {W}(t)$ is block diagonal, i.e., $W_2(t)\equiv 0$ on $[a,\infty )$, which is enforced by the used method, see Remark 4.2(ii) for more details. We note that such system ($\text {H}_\lambda$) satisfies the limit circle invariance, since condition (2.13) is trivially satisfied in this case. Also, system ($\text {H}_\lambda$) of this form is general enough to include several important equations, such as equation (3.5), the Schrödinger system (for $W_1(t)\equiv I$ and $W_4(t)\equiv 0$), see Remark 4.2(iii) and, e.g., [5, 13, 27], or the Dirac system (for $\mathcal {W}(t)\equiv I$). The latter system is, however, known to be in the limit point case, by [6, Lemma 2.15].

Theorem 4.1

Let $W_2(t)\equiv 0$ on $[a,\infty )$. If there exists $\nu \in \mathbb {R}$ such that system ($\text {H}_{\nu }$) is nonoscillatory, the Legendre condition ($\text {LC}_{\nu }$) holds, and

$$\begin{aligned} \lim _{t\rightarrow \infty } \Lambda _{\mathrm {max}}\Bigg (\int _a^t \Big [W_1^{1/2}(\tau )\,\mathcal {B}(\tau ,\nu )\,W_1^{1/2}(\tau )\Big ]^{1/2}\,\mathrm {d}\tau \Bigg )=\infty , \end{aligned}$$

(4.1)

then system ($\text {H}_\lambda$) is not in the limit circle case for any $\lambda \in \mathbb {C}$. That is, for every $\lambda \in \mathbb {C}$ there exists a solution $z(\cdot ,\lambda )$ of ($\text {H}_\lambda$) such that $z(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$.

Proof

The assumption $W_2(t)\equiv 0$ on $[a,\infty )$ guarantees by (2.13) the invariance of the limit circle case, which implies that it suffices to prove the existence of a non-square integrable solution of ($\text {H}_\lambda$) for some $\lambda \in \mathbb {C}$. We will show that system ($\text {H}_{\nu }$) is not in the limit circle case, where $\nu$ is the value from the assumptions of the theorem. By contradiction, suppose that all solutions of ($\text {H}_{\nu }$) belong to $\mathcal {L}^2_{\mathcal {W}}$. First we note that for any $2n\times n$ solution $Z(\cdot ,\nu )=\big (z_1(\cdot ,\nu ),\dots ,z_n(\cdot ,\nu )\big )$ of ($\text {H}_{\nu }$) the matrix $Z^*(t,\nu )\,\mathcal {W}(t)\,Z(t,\nu )$ is positive semidefinite, so that for $i,j\in \{1,\dots ,n\}$ we have

$$\begin{aligned} 0\le |{z_i^*(t,\nu )\,\mathcal {W}(t)\,z_j(t,\nu )}|&\le \big [z_i^*(t,\nu )\,\mathcal {W}(t)\,z_i(t,\nu )\times z_j^*(t,\nu )\, \mathcal {W}(t)\,z_j(t,\nu )\big ]^{1/2} \nonumber \\&\le \tfrac{1}{2}\,\big [z_i^*(t,\nu )\,\mathcal {W}(t)\,z_i(t,\nu )+ z_j^*(t,\nu )\,\mathcal {W}(t)\,z_j(t,\nu )\big ], \end{aligned}$$

(4.2)

on $[a,\infty )$ by the inequality of the arithmetic and geometric means. Thus, the assumption that the solutions $z_1(\cdot ,\nu ),\dots ,z_n(\cdot ,\nu )$ belong to $\mathcal {L}^2_{\mathcal {W}}$ implies that

$$\begin{aligned} \int _a^\infty \Vert {Z^*(t,\nu )\,\mathcal {W}(t)\,Z(t,\nu )}\Vert _\sigma \,\mathrm {d}t<\infty . \end{aligned}$$

(4.3)

Since the Legendre condition ($\text {LC}_{\nu }$) holds and system ($\text {H}_{\nu }$) is nonoscillatory, there exists a maximal antiprincipal solution $Z_a(\cdot ,\nu ):=Z_{a}^{[\mathrm {max}]}(\cdot ,\nu )$ of ($\text {H}_{\nu }$), i.e., $\det X_a^{[\mathrm {max}]}(t,\nu )\ne 0$ for all $t\in [c,\infty )$ with some $c\in [a,\infty )$ large enough. From the definition of the spectral norm, the fact $W_4(t)\ge 0$ on $[a,\infty )$, the block diagonal form of $\mathcal {W}(t)$, and inequalities (2.1)(i), (2.1)(iii), and (4.3) we obtain

$$\begin{aligned}&\int _a^\infty \Big \Vert W_1^{1/2}(t)\,X_a(t,\nu )\Big \Vert _\sigma ^2\,\mathrm {d}t\nonumber \\&\quad = \int _a^\infty \big \Vert X^*_a(t,\nu )\,W_1(t)\,X_a(t,\nu )\big \Vert _\sigma \,\mathrm {d}t\nonumber \\&\quad \le \int _a^\infty \big \Vert X^*_a(t,\nu )\,W_1(t)\,X_a(t,\nu )+U^*_a(t,\nu )\,W_4(t)\,U_a(t,\nu )\big \Vert _\sigma \,\mathrm {d}t\nonumber \\&\quad =\int _a^\infty \big \Vert Z^*_a(t,\nu )\,\mathcal {W}(t)\,Z_a(t,\nu )\big \Vert _\sigma \,\mathrm {d}t<\infty . \end{aligned}$$

(4.4)

Now let $t\in [c,\infty )$ and denote the rows of $X_a^{-1}(t,\nu )$ as $\xi _1(t),\dots ,\xi _n(t)$. Then it follows from (2.16) that the diagonal elements of $X_a^{-1}(t,\nu )\,\mathcal {B}(t,\nu )\,X_a^{*-1}(t,\nu )$, i.e., the functions $\xi _i(t)\,\mathcal {B}(t,\nu )\,\xi _i^*(t)$, satisfy

$$\begin{aligned} \int _c^\infty \xi _i(t)\,\mathcal {B}(t,\nu )\,\xi _i^*(t)\,\mathrm {d}t<\infty . \end{aligned}$$

Since condition ($\text {LC}_{\nu }$) implies the positive semidefiniteness of $X_a^{-1}(t,\nu )\,\mathcal {B}(t,\nu )\,X_a^{*-1}(t,\nu )$, we obtain similarly as in (4.2) the estimate

$$\begin{aligned} |{\xi _i(t)\,\mathcal {B}(t,\lambda )\,\xi _j^*(t)}| \le \tfrac{1}{2}\, \big [\xi _i(t)\,\mathcal {B}(t,\nu )\xi _i^*(t)+\xi _j(t)\,\mathcal {B}(t,\nu )\,\xi _j^*(t,\nu )\big ] \quad \text {for all i,j=1,...,n. }\ \end{aligned}$$

This implies in addition to (2.16) that

$$\begin{aligned} \int _c^\infty \big \Vert X_a^{-1}(t,\nu )\,\mathcal {B}(t,\nu )\,X_a^{*-1}(t,\nu )\big \Vert _\sigma \,\mathrm {d}t<\infty . \end{aligned}$$

(4.5)

Consequently for any $t\ge c$ we have

$$\begin{aligned}&\Lambda _{\mathrm {max}}\Bigg (\int _c^t\Big [W_1^{1/2}(\tau )\,B(\tau ,\nu )\,W_1^{1/2}(\tau )\Big ]^{1/2}\mathrm {d}\tau \Bigg ) \overset{(2.1)(i)}{=} \bigg \Vert \int _c^t\Big [W_1^{1/2}(\tau )\,B(\tau ,\nu )\,W_1^{1/2}(\tau )\Big ]^{1/2}\mathrm {d}\tau \bigg \Vert _\sigma \nonumber \\&\quad \le \int _c^t\bigg \Vert \Big [W_1^{1/2}(\tau )\,B(\tau ,\nu )\,W_1^{1/2}(\tau )\Big ]^{1/2}\bigg \Vert _\sigma \mathrm {d}\tau \overset{(2.1)(ii)}{=} \int _c^t\Big \Vert W_1^{1/2}(\tau )\,B(\tau ,\nu )\,W_1^{1/2}(\tau )\Big \Vert _\sigma ^{1/2}\mathrm {d}\tau \nonumber \\&\quad =\int _c^t\Big \Vert W_1^{1/2}(\tau )\,X_a(\tau ,\nu )\,X_a^{-1}(\tau ,\nu )\,B(\tau ,\nu )\, X_a^{*-1}(\tau ,\nu )\,X_a^*(\tau ,\nu )\,W_1^{1/2}(\tau )\Big \Vert _\sigma ^{1/2}\mathrm {d}\tau \nonumber \\&\quad \le \int _c^t\bigg (\Big \Vert W_1^{1/2}(\tau )\,X_a(\tau ,\nu )\Big \Vert _\sigma \, \Big \Vert X_a^{-1}(\tau ,\nu )\,B(\tau ,\nu )\,X_a^{*-1}(\tau ,\nu )\Big \Vert _\sigma \, \Big \Vert X_a^*(\tau ,\nu )\,W_1^{1/2}(\tau )\Big \Vert _\sigma \bigg )^{\!1/2}\mathrm {d}\tau \nonumber \\&\quad = \int _c^t\bigg (\Big \Vert W_1^{1/2}(\tau )\,X_a(\tau ,\nu )\Big \Vert _\sigma ^2\bigg )^{\!1/2}\, \bigg (\Big \Vert X_a^{-1}(\tau ,\nu )\,B(\tau ,\nu )\,X_a^{*-1}(\tau ,\nu )\Big \Vert _\sigma \bigg )^{\!1/2}\mathrm {d}\tau \nonumber \\&\quad \le \frac{1}{2}\int _c^t \bigg (\Big \Vert W_1^{1/2}(\tau )\,X_a(\tau ,\nu )\Big \Vert _\sigma ^2+ \Big \Vert X_a^{-1}(\tau ,\nu )\,B(\tau ,\nu )\,X_a^{*-1}(\tau ,\nu )\Big \Vert _\sigma \bigg )\mathrm {d}\tau , \end{aligned}$$

(4.6)

where we used also the submultiplicative property of $\Vert {\cdot }\Vert _\sigma$ and the inequality of arithmetic and geometric means. But as $t\rightarrow \infty$ in (4.6) we obtain that the right-hand side of (4.6) tends to a finite limit, by (4.4) and (4.5). This is a contradiction with (4.1), and hence system ($\text {H}_{\nu }$) possesses at least one solution $z(\cdot ,\nu )\not \in \mathcal {L}^2_{\mathcal {W}}$. The proof is complete. $\square$

Remark 4.2

• (i) In the previous proof we utilized the maximal antiprincipal solution $Z_a(\cdot ,\nu )$, which has $X_a(t,\nu )$ eventually invertible. Nevertheless, the same arguments can be used when $Z_a(\cdot ,\nu )$ is any antiprincipal solution of ($\text {H}_{\nu }$). In this case we replace $X_a^{-1}(\cdot ,\nu )$ by $X_a^\dagger (\cdot ,\nu )$ and use the identities

  $$\begin{aligned} X_a(t,\nu )\,X_a^\dagger (t,\nu )\,\mathcal {B}(t,\nu )=\mathcal {B}(t,\nu )=\mathcal {B}^*(t,\nu ) =\mathcal {B}^*(t,\nu )X_a^{\dagger *}(t,\nu )\,X_a^*(t,\nu ) \end{aligned}$$

  on $[c,\infty )$, see [24, Lemma 2] and [35, Theorem 4.2], where $c\in [a,\infty )$ is such that ${{\mathrm{Ker}}}X_a(t,\nu )$ is constant on $[c,\infty )$.

• (ii) The block diagonal structure of $\mathcal {W}(t)$ is required to guarantee the validity of inequality (4.4), which is crucial in the proof of the convergence of (4.6). In the general case we have

  

  and this integral may be convergent even if $\int _a^\infty x^*(t,\lambda )\,W_1(t)\,x(t,\lambda )\,\mathrm {d}t$ is divergent. Indeed, consider system ($\text {H}_\lambda$) with $\mathcal {H}(t)\equiv \left( \begin{array}{ll}0 &{} 1\\ 1 &{} 0\end{array}\right)$ and the weight matrix $\mathcal {W}(t)={{\mathrm{e}}}^{-2t}\left( \begin{array}{ll}\cosh t &{} -\sinh t\\ -\sinh t &{} \cosh t\end{array}\right) \ge 0$ for $t\in [0,\infty )$. For $\lambda =0$ the principal and antiprincipal solutions are $z_p(t,0)=({{\mathrm{e}}}^{-t},-{{\mathrm{e}}}^{-t})^\top$ and $z_a(t,0)=({{\mathrm{e}}}^{t},{{\mathrm{e}}}^{t})^\top$. Since

  $$\begin{aligned} \int _0^\infty z_p^*(t,0)\,\mathcal {W}(t)\,z_p(t,0)\,\mathrm {d}t&=\int _0^\infty 2{{\mathrm{e}}}^{-4t}\,(\cosh t+\sinh t)\,\mathrm {d}t=\int _0^\infty 2{{\mathrm{e}}}^{-3t}\mathrm {d}t=2/3,\\ \int _0^\infty z_a^*(t,0)\,\mathcal {W}(t)\,z_a(t,0)\,\mathrm {d}t&=\int _0^\infty 2\,(\cosh t-\sinh t)\,\mathrm {d}t=\int _0^\infty 2{{\mathrm{e}}}^{-t}\mathrm {d}t=2, \end{aligned}$$

  and condition (2.13) is trivially satisfied for $\nu =0$, this system is in the limit circle case for all $\lambda \in \mathbb {C}$. But in contrast with (4.4) we have

  $$\begin{aligned} \int _0^\infty x_a^*(t,0)\,W_1(t)\,x_a(t,0)\,\mathrm {d}t=\int _0^\infty \cosh t\,\mathrm {d}t=\infty . \end{aligned}$$

• (iii) For the second-order Sturm–Liouville differential equation, i.e., equation (3.5) with $m=n=1$, Theorem 4.1 reduces to the limit point criterion in [30, Theorem 4.1] mentioned at the beginning of this section. In particular, condition (4.1) has the form

  $$\begin{aligned} \int _a^\infty \sqrt{\frac{W(t)}{P_1(t)}}\,\mathrm {d}t=\infty . \end{aligned}$$

  On the other hand, for the second-order vector-valued Sturm–Liouville equations, i.e., for (3.5) with $m=1$ and $n\ge 2$, Theorem 4.1 yields a new result, in which condition (4.1) reads as

  $$\begin{aligned} \lim _{t\rightarrow \infty }\Lambda _{\mathrm {max}}\Bigg (\int _a^t\Big [W^{1/2}(\tau )\,P_m^{-1}(\tau )\,W^{1/2} (\tau )\Big ]^{1/2}\,\mathrm {d}\tau \Bigg )=\infty . \end{aligned}$$

  But, surprisingly, the criterion in Theorem 4.1 is not applicable when $m\ge 2$, because in that case assumption (4.1) is not satisfied, as $W_1^{1/2}(t)\,\mathcal {B}(t,\nu )\,W_1^{1/2}(t)\equiv 0$ on $[a,\infty )$ by (3.6). This fact also shows that condition (4.1) is not necessary for the existence of a non-square integrable solution. For completeness we note that condition (4.1) for (3.5) can be satisfied only when the right-hand side of (3.5) is replaced by the expression $\lambda \,W(t)\,y^{(m-1)}(t,\lambda )$.$\square$

Now we provide several examples illustrating the result of Theorem 4.1. In the first example we consider a $4\times 4$ controllable system with constant coefficients satisfying the assumptions of Theorem 4.1 and show that there exist at most two linearly independent square integrable solutions for any $\lambda \in \mathbb {C}$. This observation opens the question if it is possible to state Theorem 4.1 as a limit point criterion analogously to the case of the second-order Sturm–Liouville equation.

Example 4.3

Let $[a,\infty )=[0,\infty )$ and consider the system

$$\begin{aligned} (\mathrm{H}_\lambda )\ \ \text {with }\ A(t)\equiv 0,\ \ B(t)\equiv {{\mathrm{diag}}}\{p,q\},\ \ C(t)\equiv {{\mathrm{diag}}}\{r,s\},\ \ \mathcal {W}(t)={{\mathrm{diag}}}\{1,1,1,0\}, \end{aligned}$$

(4.7)

where p, r, s are nonnegative and $q>0$. Then $W_1^{1/2}(t)\,\mathcal {B}(t,\lambda )\,W_1^{1/2}(t)=\mathcal {B}(t,\lambda )={{\mathrm{diag}}}\{p+\lambda ,\,q\}$ and condition (4.1) is satisfied with $\nu =0$, because

$$\begin{aligned} \lim _{t\rightarrow \infty } \Lambda _{\mathrm {max}}\Bigg (\int _0^t\Big [W_1^{1/2}(\tau )\,\mathcal {B}(\tau ,0)\,W_1^{1/2} (\tau )\Big ]^{1/2}\,\mathrm {d}\tau \Bigg )&=\lim _{t\rightarrow \infty } \Lambda _{\mathrm {max}}\Bigg (\int _0^t \left( \begin{array}{ll}\sqrt{p} &{} 0\\ 0 &{} \sqrt{q}\end{array}\right) \,\mathrm {d}\tau \Bigg )\\&=\lim _{t\rightarrow \infty } \max \{\sqrt{p},\sqrt{q}\}\,t=\infty . \end{aligned}$$

Assume first that $\lambda \not \in \{-p,r,s\}$. Then system (4.7) corresponds to the pair of equations

$$\begin{aligned} y_1''(t,\lambda )=(p+\lambda )(r-\lambda )\,y_1(t,\lambda ), \quad y_2''(t,\lambda )=q(s-\lambda )\,y_2(t,\lambda ), \end{aligned}$$

and the functions

$$\begin{aligned} z_{1,2}(t,\lambda )=\left( \begin{array}{ll}{{\mathrm{e}}}^{\pm \sqrt{(p+\lambda )(r-\lambda )}\,t}\\ 0\\ \frac{\pm \sqrt{(p+\lambda )(r-\lambda )}}{p+\lambda }{{\mathrm{e}}}^{\pm \sqrt{(p+\lambda )(r-\lambda )}\,t}\\ 0\end{array}\right) , \quad z_{3,4}(t,\lambda )=\left( \begin{array}{ll}0\\ {{\mathrm{e}}}^{\pm \sqrt{q(s-\lambda )}\,t}\\ 0 \\ \pm \sqrt{\frac{s-\lambda }{q}}{{\mathrm{e}}}^{\pm \sqrt{q(s-\lambda )}\,t}\end{array}\right) \end{aligned}$$

form its fundamental system of solutions. For $\lambda =-p$ the linearly independent solutions of (4.7) are

$$\begin{aligned} z_{1}(t,-p)=\left( \begin{array}{ll}0\\ 0\\ 1\\ 0\end{array}\right) ,\quad z_{2}(t,-p)=\left( \begin{array}{ll}1\\ 0\\ (r+p)\,t\\ 0\end{array}\right) , \quad z_{3,4}(t,-p)=\left( \begin{array}{ll}0\\ {{\mathrm{e}}}^{\pm \sqrt{q(s+p)}\,t}\\ 0 \\ \pm \sqrt{\frac{s+p}{q}}{{\mathrm{e}}}^{\pm \sqrt{q(s+p)}\,t}\end{array}\right) . \end{aligned}$$

Similar situation occurs for $\lambda \in \{r,s\}$. These calculations show that system (4.7) is nonoscillatory for all $\lambda \in [-p,r]\cap (-\infty ,s]$. Therefore, the assumptions of Theorem 4.1 are satisfied with $\nu =0$ and it follows that system (4.7) possesses at least one non-square integrable solution for all $\lambda \in \mathbb {C}$. This fact can be verified directly, because $z_1(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$ when $\lambda \in (-p,r)$, and $z_1(\cdot ,\lambda ),z_2(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$ when $\lambda \in (-\infty ,-p]\cup [r,\infty )$. Similarly, for $\lambda \in (-\infty ,s)$ we have $z_3(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$ and for $\lambda \in [s,\infty )$ we obtain $z_3(\cdot ,\lambda ),z_4(\cdot ,\lambda )\not \in \mathcal {L}^2_{\mathcal {W}}$. Hence there exist at most two linearly independent square integrable solutions of system (4.7), i.e., $\dim \mathcal {N}(\lambda )\le 2$ for all $\lambda \in \mathbb {R}$. Moreover, system (4.7) is in the limit point case for all $\lambda \in \mathbb {C}\backslash \mathbb {R}$. This fact follows from the calculation of the limiting matrix radius $R_+(\lambda )$ for the fundamental matrix $\Phi _I(t,\lambda )$. Indeed, for $\lambda \in \mathbb {C}\backslash \mathbb {R}$ the second $4\times 2$ block of $\Phi _I(t,\lambda )$ is

and the value $R_+(\lambda )=0$ is obtained by the formulas in Sect. 2.2.$\square$

Now we consider system ($\text {H}_\lambda$), which is not controllable. We show that condition (4.1) does not guarantee the limit point case for system ($\text {H}_\lambda$), which justifies the fact that the result in Theorem 4.1 is “only” a non-limit circle case criterion, similarly to [2, Theorem 13].

Example 4.4

Motivated by [36, Example 7.4], let us consider system ($\text {H}_\lambda$) with

This system is not controllable for $\lambda =0$ with $d_\infty (0)=d_0[0,\infty )=2$. Since the assumptions of Theorem 4.1 are satisfied with $\nu =0$, the system possesses at least one non-square integrable solution for any $\lambda \in \mathbb {C}$. The corresponding minimal and maximal principal and antiprincipal solutions are

Then the first column of each of the above solutions does not belong to $\mathcal {L}^2_{\mathcal {W}}$, so that the invariance of the limit circle case implies that ($\text {H}_\lambda$) is not in the limit circle case for every $\lambda \in \mathbb {C}$. Moreover, in this case there exist four linearly independent square integrable solutions – the second and third columns of the solutions $Z^{[\mathrm {max}]}_p(\cdot ,0)$ and $Z^{[\mathrm {min}]}_a(\cdot ,0)$.

In addition, we obtain a similar conclusion also for the choice of $W_1(t)\equiv {{\mathrm{diag}}}\{0,1,0\}$, although condition (4.1) does not hold in this case. Finally, we note that system ($\text {H}_\lambda$) is in the limit circle case, e.g., when $W_1(t)\equiv 0$ and $W_4(t)\equiv {{\mathrm{diag}}}\{0,0,1\}$,$\square$

In the last example we show that the Legendre condition required in Theorem 4.1 is not necessary for the existence of a non-square integrable solution. This condition is used in the proof for the existence of an antiprincipal solution, as well as for the correct formulation of condition (4.1).

Example 4.5

Let $[a,\infty )=[0,\infty )$ and let the coefficients of system ($\text {H}_\lambda$) be given as

$$\begin{aligned} A(t)\equiv 0,\quad B(t)\equiv \left( \begin{array}{ll}0 &{} p\\ p &{} q \end{array}\right) ,\quad C(t)\equiv {{\mathrm{diag}}}\{0,s\},\quad W_1(t)\equiv \left( \begin{array}{ll}1 &{} 1\\ 1 &{} 1\end{array}\right) ,\quad W_4(t)\equiv {{\mathrm{diag}}}\{0,1\}, \end{aligned}$$

where p, q, s are positive. For $\lambda =0$ we get the fundamental system of solutions

$$\begin{aligned} z_{1,2}(t,0) =\left( \begin{array}{ll}(p/q)\,{{\mathrm{e}}}^{\pm \sqrt{qs}\,t}\\ {{\mathrm{e}}}^{\pm \sqrt{qs}\,t}\\ 0 \\ \pm \sqrt{s/q}\,{{\mathrm{e}}}^{\pm \sqrt{qs}\,t}\end{array}\right) ,\quad z_3(t,0)=\left( \begin{array}{ll}\phantom {-}pt\\ \phantom {-}0 \\ -q/p\\ \phantom {-}1\end{array}\right) ,\quad z_4(t,0)=\left( \begin{array}{ll}1\\ 0 \\ 0\\ 0 \end{array}\right) , \end{aligned}$$

which implies that the system is nonoscillatory. Moreover, one observes that only $z_2(\cdot ,0)\in \mathcal {L}^2_{\mathcal {W}}$, which means that we have at least one solution not in $\mathcal {L}^2_{\mathcal {W}}$ for all $\lambda \in \mathbb {C}$, according to the invariance of the limit circle case. Nevertheless, the assumptions of Theorem 4.1 are not fulfilled, because the matrix $\mathcal {B}(t,0)=B(t)$ is indefinite.$\square$

Finally, from Theorem 4.1 we obtain the following oscillation criterion for system ($\text {H}_\lambda$).

Corollary 4.6

Let $W_2(t)\equiv 0$ on $[a,\infty )$. If system ($\text {H}_\lambda$) is in the limit circle case for some (and hence for all) $\lambda \in \mathbb {C}$ and there exists $\nu \in \mathbb {R}$ such that conditions ($\text {LC}_{\nu }$) and (4.1) are satisfied, then system ($\text {H}_{\nu }$) is oscillatory.

Remark 4.7

If in addition to $W_2(t)\equiv 0$ we also have $W_4(t)\equiv 0$ on $[a,\infty )$, then $\mathcal {B}(t,\nu )= B(t)$, which yields the independence of conditions ($\text {LC}_{\nu }$) and (4.1) on $\nu \in \mathbb {R}$. In this special case we obtain from Corollary 4.6 the following statement. If system ($\text {H}_\lambda$) is in the limit circle case for some (and hence for all) $\lambda \in \mathbb {C}$ and

$$\begin{aligned} B(t)\ge 0 \quad t\in [a,\infty ), \quad \lim _{t\rightarrow \infty } \Lambda _{\mathrm {max}}\Bigg (\int _a^t \Big [W_1^{1/2}(\tau )\,B(\tau )\,W_1^{1/2}(\tau )\Big ]^{1/2}\,\mathrm {d}\tau \Bigg )=\infty , \end{aligned}$$

then ($\text {H}_\lambda$) is oscillatory for all $\lambda \in \mathbb {R}$. This can be regarded as a criterion for the invariance of the limit-circle-oscillatory behavior (usually denoted by “LCO”) for system ($\text {H}_\lambda$), compare with [50, Theorem 7.3.1] for equation (3.5) with $m=n=1$.