Generalized Schur–Nevanlinna functions and their realizations

Abstract

Pontryagin space operator valued generalized Schur functions and generalized Nevanlinna functions are investigated by using discrete-time systems, or operator colligations, and state space realizations. It is shown that generalized Schur functions have strong radial limit values almost everywhere on the unit circle. These limit values are contractive with respect to the indefinite inner product, which allows one to generalize the notion of an inner function to Pontryagin space operator valued setting. Transfer functions of self-adjoint systems such that their state spaces are Pontryagin spaces, are generalized Nevanlinna functions, and symmetric generalized Schur functions can be realized as transfer functions of self-adjoint systems with Kreĭn spaces as state spaces. A criterion when a symmetric generalized Schur function is also a generalized Nevanlinna function is given. The criterion involves the negative index of the weak similarity mapping between an optimal minimal realization and its dual. In the special case corresponding to the generalization of an inner function, a concrete model for the weak similarity mapping can be obtained by using the canonical realizations.

1 Introduction

Let ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ be separable Pontryagin spaces with the same finite negative index, and let ${\mathcal {L}}({{\mathcal {U}}},{{\mathcal {Y}}})$ be the class of bounded linear operators from ${{\mathcal {U}}}$ to ${{\mathcal {Y}}}.$ An ${\mathcal {L}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function $\theta$ belongs to generalized Schur class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})},$ if it is holomorphic at the origin and the Schur kernel

$$\begin{aligned} K_\theta (w,z)=\frac{1-\theta (z)\theta ^*(w)}{1-z{\bar{w}}}, \qquad w,z \in \rho (\theta ), \end{aligned}$$

(1.1)

where $\theta ^*(w)=\left( \theta (w)\right) ^*$, has $\kappa$ negative squares. This means that for any finite sets of points $\{w_1,\ldots ,w_n\}\subset \rho (\theta ),$ where $\rho (\theta )$ is maximal domain of analyticity of $\theta$, and vectors $\{f_1,\ldots ,f_n\} \subset {{\mathcal {Y}}},$ the Hermitian matrix

$$\begin{aligned} \left( \left\langle K_\theta (w_j,w_i)f_j,f_i \right\rangle _{{{\mathcal {Y}}}}\right) _{i,j=1}^{n}, \end{aligned}$$

(1.2)

where $\left\langle \cdot , \cdot \right\rangle _{{{\mathcal {Y}}}}$ is the inner product of ${{\mathcal {Y}}},$ has no more than $\kappa$ negative eigenvalues, and there exists a matrix of the form (1.2) which has exactly $\kappa$ negative eigenvalues. On the other hand, an ${\mathcal {L}}({{\mathcal {U}}})$-valued function $\theta ,$ where ${{\mathcal {U}}}$ is a Pontryagin space, belongs to generalized Nevanlinna class ${{\mathbf {N}}_{\kappa }({{\mathcal {U}}})}$ if it is meromorphic on ${{\mathbb {C}}}\setminus {{\mathbb {R}}},$ real, or symmetric, in a sense that $\theta (z)=\theta ^\#(z)$ for every $z \in \rho (\theta ),$ where $\theta ^\#(z)$ is defined to be $\theta ^*({\bar{z}})$, and the Nevanlinna kernel

$$\begin{aligned} N_\theta (w,z)=\frac{\theta (z)-\theta ^*(w)}{z-{\bar{w}}}, \qquad w,z \in \rho (\theta ) , \end{aligned}$$

(1.3)

has $\kappa$ negative squares. If ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, the classes ${\mathbf {S}}_0({{\mathcal {U}}},{{\mathcal {Y}}})$ and ${\mathbf {N}}_0({{\mathcal {U}}})$, which are denoted as ${\mathbf {S}}({{\mathcal {U}}},{{\mathcal {Y}}})$ and ${\mathbf {N}}({{\mathcal {U}}}),$ coincide with the ordinary Schur and Nevanlinna classes. That is, ${\mathbf {S}}({{\mathcal {U}}},{{\mathcal {Y}}})$ consists of ${\mathcal {L}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued functions holomorphic and bounded by one in ${{\mathbb {D}}},$ and ${\mathbf {N}}({{\mathcal {U}}})$ consists of ${\mathcal {L}}({{\mathcal {U}}})$-valued functions holomorphic and symmetric in ${{\mathbb {C}}}\setminus {{\mathbb {R}}}$ such that their imaginary parts are nonnegative in the upper half plane. The classes of generalized Schur and Nevanlinna functions were first studied by Kreĭn and Langer in series of papers [26, 27, 27, 29, 30], first in the scalar case (${{\mathcal {U}}}={{\mathcal {Y}}}={{\mathbb {C}}}$) and later in the operator valued case.

The study of the (generalized) Schur functions in infinite dimensional spaces naturally leads to contractive operators and passive linear discrete-time systems; or what is the same thing, contractive operator colligations. An operator colligation

$$\begin{aligned} \Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}},{{\mathcal {Y}}}) \end{aligned}$$

(1.4)

consists of a Kreĭn space ${{\mathcal {X}}}$ (the state space), Pontryagin spaces ${{\mathcal {U}}}$ (the incoming space) and ${{\mathcal {Y}}}$ (the outgoing space) with the same negative index, and the system operator $T_{\Sigma } \in {{\mathcal {L}}}({{\mathcal {X}}}\oplus {{\mathcal {U}}},{{\mathcal {X}}}\oplus {{\mathcal {Y}}}),$ where the direct orthogonal sum ${{\mathcal {X}}}\oplus {{\mathcal {U}}}$ or $\begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {U}}}\end{pmatrix}$ is with respect to the indefinite inner product. Here $T_{\Sigma }$ is bounded and everywhere defined, and has the block representation of the form

$$\begin{aligned} T_{\Sigma }=\begin{pmatrix} A&{}B \\ C&{}D \end{pmatrix} : \begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {U}}}\end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {Y}}}\end{pmatrix}, \end{aligned}$$

(1.5)

where $A \in {{\mathcal {L}}}({{\mathcal {X}}})$ is the main operator, and $B \in {{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {X}}})$, $C \in {{\mathcal {L}}}({{\mathcal {X}}},{{\mathcal {Y}}})$, and $D \in {{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$ . The colligation will be usually called as a system, since it can be seen as a linear discrete-time system, and the system is identified with its operator expression (1.5). The system $\Sigma$ is passive (isometric, co-isometric, conservative, self-adjoint), if the system operator $T_\Sigma$ in (1.5) is contractive (isometric, co-isometric, unitary, self-adjoint) with respect to the indefinite inner product. The transfer function of the system (1.5), or characteristic function of the operator colligation, is defined by

$$\begin{aligned} \theta _\Sigma (z):=D+z C(I-z A)^{-1}B, \end{aligned}$$

(1.6)

whenever $I-z A$ is invertible. Especially, $\theta _\Sigma$ is defined and holomorphic on a neighbourhood of the origin. The values $\theta _{\Sigma }(z)$ are bounded operators from ${{\mathcal {U}}}$ to ${{\mathcal {Y}}}.$ Conversely, if $\theta$ is an operator valued function, and the transfer function of a system $\Sigma$ coincides with it in a neighbourhood of the origin, then $\Sigma$ is a (scattering) realization of $\theta ,$ and a realization problem for the operator valued function $\theta$ analytic at the origin is to find system of the form (1.5) such that its transfer function coincides with $\theta .$

For ordinary Schur functions, this connection was discovered and studied, for instance, by Arov [7, 8], de Branges and Rovnyak [17, 18], Brodskiĭ [19] and Sz.-Nagy and Foias [36]. The standard Hilbert space theory of ordinary Schur functions has a counterpart for the generalized Schur functions, and this will led to replacing the Hilbert state space, or all of the spaces, by Pontryagin, or in some cases, even by Kreĭn spaces. In the case where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, the generalized Schur class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ and its connections to unitary colligations were studied, for instance, by Dijksma, Langer and de Snoo [22]. Arov’s approach to use passive systems was utilized by Saprikin [34], Arov and Saprikin [13], Arov, Rovnyak and Saprikin [12] and by the author in [31] to study the class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces. If ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, one encounters operator colligations, or systems, such that all the spaces are indefinite. Theory of canonical isometric, co-isometric and conservative systems in that case is considered, for instance, in [2, 3, 20, 23], along with the other properties of the generalized Schur functions. Especially, symmetric generalized Schur functions, with a little bit more general definition than in this paper, were studied in [3]. The results about the unitary similarities between the canonical realizations obtained therein will be used.

Theory of passive systems and generalized Schur functions in the case where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, was studied by the author in [32]. On the other aspects, in the case where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are finite dimensional, the class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is closely related to generalized Potapov class and generalized ${\mathcal {J}}$-inner functions; see for instance [1, 6, 8, 21, 37].

On the other hand, the generalized Nevanlinna functions have been studied alongside with the Schur functions, mainly with scalar, matrix and Hilbert space operator valued cases. Instead of unitary and contractive operators, the study of the generalized Nevanlinna functions involves dissipative and self-adjoint linear operators and relations; see for instance [25, 29].

The aim of this paper is to study connections of discrete-time systems, transfer functions and operator valued analytic functions which are both generalized Schur and generalized Nevanlinna functions for some indices, that is, which belongs to the class ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ where ${{\mathcal {U}}}$ is a Pontryagin space. Before involving the realization theory, the structural properties of the generalized Schur functions and generalized Nevanlinna functions are studied by using the Potapov–Ginzburg transformation. Especially, in the case where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are finite dimensional anti-Hilbert spaces, the behaviour of the functions in the classes ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}})$ and ${\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ is reciprocal to the Hilbert space case, see Corollary 2.3 and Proposition 2.6. Moreover, in Theorem 2.8, when ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative indices, it will be proved that for $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, the strong radial limit value $\theta (\zeta ):=\lim _{r \rightarrow 1^-}\theta (r\zeta )$ where $\zeta$ belongs to the unit circle ${{\mathbb {T}}}$, exists almost everywhere (a.e.), and their values are contractive with respect to the underlying indefinite inner products. Theorem 2.8 also gives rise to a notion of a generalized ${{\mathcal {J}}}$-inner function in infinite dimensional spaces.

In realization theory, the study of the class ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ where ${{\mathcal {U}}}$ is a Pontryagin space, naturally leads to self-adjoint systems. For the ordinary Schur and Nevanlinna functions, these connections were studied by Arlinskiĭ, Hassi and de Snoo in [4] and by Arlinskiĭ and Hassi in [5]. One of their main results was that $\theta \in {\mathbf {S}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$, where ${{\mathcal {U}}}$ is a Hilbert space, if and only if $\theta$ has a minimal passive self-adjoint realization of the form (1.5) such that the state space is a Hilbert space [4, Theorem 5.4]. In the case $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ one can obtain a similar realization which is self-adjoint, but not passive in the general case; see Theorem 3.5, Remark 3.6 and Proposition 3.7.

On the other hand, every $\theta \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}}, {{\mathcal {Y}}})$ has a minimal passive realization $\Sigma ,$ and it can be chosen such that it is optimal or $^*$-optimal [32, Theorem 3.5]; for the case where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, see also [34, Theorem 5.3]. For a symmetric $\theta \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}}),$ these realizations have special properties. Namely, the dual system of the optimal minimal passive realization of $\theta$ is a $^*$-optimal minimal passive realization of $\theta ,$ and vice versa. One can form a weak similarity mapping Z between those systems such that Z is everywhere defined, contractive and self-adjoint. If $\theta$ has a meromorphic continuation to ${{\mathbb {C}}}\setminus {{\mathbb {R}}}$, then the negative index of the mapping Z with respect to the indefinite inner product in question determines the number of the negative squares of the Nevanlinna kernel (1.3); see Theorem 3.10. That is, the negative index of the Z, which roughly speaking tells that how much Z behaves like a positive operator with respect to the indefinite inner product in question, can be used to determine whether $\theta$ is also a generalized Nevanlinna function. If, in addition, the boundary values of $\theta$ on the unit disc ${{\mathbb {T}}}$ are unitary, then Z is also unitary and can be represented in an explicit form by using the canonical realizations from [2, 3].

It is a classical problem to determine if an ordinary Schur function $\theta$ can represented as a corner of a bi-inner dilation of the form

$$\begin{aligned} \Theta =\begin{pmatrix} \theta &{} \theta _{2} \\ \theta _{3} &{} \theta _{4} \end{pmatrix} ; \end{aligned}$$

(1.7)

see, for an instance, [8, 14]. Arlinskiĭ and Hassi showed in [5] that every $\theta \in {\mathbf {S}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}}),$ where ${{\mathcal {U}}}$ is a Hilbert space, has a bi-inner dilation, and moreover, a dilation (1.7) can be chosen such that it is an ordinary Nevanlinna function. In the last section of this paper, similar results will be obtained for the subclasses of ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ where ${{\mathcal {U}}}$ is a Pontryagin space. In particular, functions in ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ with the property that their minimal passive realizations are unitarily similar, always have a dilation with unitary boundary values almost everywhere on ${{\mathbb {T}}}$, and those functions in ${\mathbf {S}}_{\kappa }({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa }({{\mathcal {U}}})$ which have a minimal passive self-adjoint realization, always have a dilation $\Theta$ with unitary boundary values almost everywhere on ${{\mathbb {T}}}$. Moreover, $\Theta$ can be chosen such that it is a generalized Nevanlinna function with the index $\kappa$; see Theorem 4.1.

2 Structural properties of the generalized Schur and generalized Nevanlinna functions

When ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, the full structure of the functions in ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ and ${\mathbf {N}}_{\kappa }({{\mathcal {U}}})$ is somewhat more complicated than in the better known Hilbert space case. For instance, when ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, Kreĭn–Langer factorizations shows that a function in ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ has exactly $\kappa$ poles, counting multiplicities; see Lemma 2.5. This does not hold anymore when the negative index of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ is not zero; a function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ may has any countable number of poles, see Corollary 2.3 and Example 2.7 below. However, some properties of the function $\theta$ in ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ or ${\mathbf {N}}_{\kappa }({{\mathcal {U}}})$ can be analyzed by using a suitable transformation $\theta \mapsto \theta '$, where $\theta ' \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}}',{{\mathcal {Y}}}')$ or ${\mathbf {N}}_{\kappa }({{\mathcal {U}}}')$ for some Hilbert spaces ${{\mathcal {U}}}'$ and ${{\mathcal {Y}}}'$.

In what follows, all notions of continuity and convergence are understood to be with respect to the strong topology, which is induced by any fundamental decomposition of the space in question. Let $\theta$ be an ${{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function holomorphic on a set $\rho (\theta )$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index. Let ${{\mathcal {U}}}={{\mathcal {U}}}_{+}\oplus {{\mathcal {U}}}_{-}$ and ${{\mathcal {Y}}}={{\mathcal {Y}}}_{+}\oplus {{\mathcal {Y}}}_{-}$ be some fixed fundamental decompositions of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$. Represent $\theta$ as

$$\begin{aligned} \theta (z)=\begin{pmatrix} \theta _{11}(z) &{} \theta _{12}(z) \\ \theta _{21}(z) &{} \theta _{22}(z) \end{pmatrix}: \begin{pmatrix} {{\mathcal {U}}}_+ \\ {{\mathcal {U}}}_- \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {Y}}}_+ \\ {{\mathcal {Y}}}_- \end{pmatrix}, \end{aligned}$$

(2.1)

and define ${{\mathcal {U}}}'={{\mathcal {U}}}_+ \oplus |{{\mathcal {U}}}_-|$ and ${{\mathcal {Y}}}'={{\mathcal {Y}}}_+ \oplus |{{\mathcal {Y}}}_-|$, where $|{{\mathcal {U}}}_-|$ and $|{{\mathcal {Y}}}_-|$ are antispaces of ${{\mathcal {U}}}_-$ and ${{\mathcal {Y}}}_-$. The antispace of an inner product space ${{\mathcal {H}}}$ is by definition the space that coincides with ${{\mathcal {H}}}$ as a vector space and is endowed with an inner product $-\langle \cdot , \cdot \rangle _{{{\mathcal {H}}}}$. Denote

$$\begin{aligned} {\left\{ \begin{array}{ll} \sigma : {{\mathcal {U}}}_- \rightarrow |{{\mathcal {U}}}_-| &{}, \tau : {{\mathcal {Y}}}_- \rightarrow |{{\mathcal {Y}}}_-|, \\ \sigma ^* =-\sigma ^{-1} &{}, \tau ^* =-\tau ^{-1}, \end{array}\right. } \end{aligned}$$

(2.2)

for the identity mappings. The Potapov–Ginzburg transformation; see [2, Sect. 4.3] and [15, Sect. 5.§1], of $\theta$ is then defined to be an ${{\mathcal {L}}}({{\mathcal {U}}}',{{\mathcal {Y}}}')$-valued function

$$\begin{aligned} \begin{aligned} \theta _P(z)&=\begin{pmatrix} \theta _{11}(z) -\theta _{12}(z)\theta _{22}^{-1}(z)\theta _{21}(z) &{} \theta _{12}(z)\theta _{22}^{-1}(z) \tau ^{-1}\\ -\sigma \theta _{22}^{-1}(z) \theta _{21} (z) &{}\sigma \theta _{22}^{-1}(z) \tau ^{-1} \end{pmatrix} \\&=\begin{pmatrix} {\theta _P}_{11}(z) &{} {\theta _P}_{12}(z) \\ {\theta _P}_{21}(z) &{} {\theta _P}_{22}(z) \end{pmatrix}, \end{aligned} \end{aligned}$$

(2.3)

whose domain $\rho (\theta _P)$ consists of all the points $z\in \rho (\theta )$ such that $\theta _{22}(z)$ is invertible. A calculation shows that

$$\begin{aligned} \theta (z)=\begin{pmatrix} {\theta _P}_{11}(z) -{\theta _P}_{12}(z){\theta _P}_{22}^{-1}(z){\theta _P}_{21}(z) &{}{\theta _P}_{12}(z){\theta _P}_{22}^{-1}(z) \sigma \\ -\tau ^{-1} {\theta _P}_{22}^{-1}(z) {\theta _P}_{21} (z) &{}\tau ^{-1} {\theta _P}_{22}^{-1}(z)\sigma \end{pmatrix} \end{aligned}$$

(2.4)

holds for every $z \in \rho (\theta _P).$ Note that the values of ${\theta _P}_{22}$ are invertible whenever they exist. Define, respectively, ${{\mathcal {L}}}({{\mathcal {Y}}}',{{\mathcal {Y}}})$ and ${{\mathcal {L}}}({{\mathcal {U}}}',{{\mathcal {U}}})$-valued functions

$$\begin{aligned} \Phi (z) = \begin{pmatrix} I_{{{\mathcal {Y}}}_+} &{} -\theta _{12}(z)\sigma ^{-1} \\ 0 &{} -\theta _{22}(z) \sigma ^{-1} \end{pmatrix} \quad \text {and}\quad \Psi (z) = \begin{pmatrix} I_{{{\mathcal {U}}}_+} &{} -\theta _{21}^\#(z) \tau ^{-1} \\ 0 &{} -\theta _{22}^\#(z) \tau ^{-1} \end{pmatrix}. \end{aligned}$$

(2.5)

Proposition 2.1

Let ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ be Pontryagin spaces with the same negative index $\pi \ge 1$, and let $\theta$ be an ${{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function holomorphic on a set $\rho (\theta )$ and meromorphic on a set ${\mathcal {D}}$.

• (i) If $\theta _P$ exists, it is meromorphic on ${{\mathcal {D}}}$, and if $\theta _P$ is meromorphic on a set ${{\mathcal {D}}}_P,$ then so is $\theta$.

• (ii) The Potapov–Ginzburg transformation $\left( \theta ^\#\right) _P$ of $\theta ^\#$ is $\left( \theta _P\right) ^\#$.

• (iii) The identities

  $$\begin{aligned} I - \theta (z)\theta ^*(w)= & {} \Phi (z) \big ( I - \theta _P(z)\theta _P^*(w) \big )\Phi ^*(w) \end{aligned}$$

  (2.6)
  $$\begin{aligned} I - \theta ^\#(z){\theta ^\#}^*(w)= & {} \Psi (z) \left( I - \theta _P^\#(z){\theta _P^\#}^*(w) \right) \Psi ^*(w) \end{aligned}$$

  (2.7)
  $$\begin{aligned} \theta (z)-\theta ({\bar{w}})= & {} \Phi (z) \big ( \theta _P(z)-\theta _P({\bar{w}}) \big ) \Psi ^*(w) \end{aligned}$$

  (2.8)
  $$\begin{aligned} \theta ^\#(z)-\theta ^\#({\bar{w}})= & {} \Psi (z) \left( \theta ^\#_P(z)-\theta ^\#_P({\bar{w}}) \right) \Phi ^*(w) \end{aligned}$$

  (2.9)

  hold whenever the corresponding functions are defined.

• (iv) If $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, then $\rho (\theta _P)$ is of the form $\rho (\theta )\setminus \Xi ,$ where $\Xi$ contains at most $\kappa$ points.

• (v) If $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ and $\theta _{22}^{-1}(0)$ exists, then $\theta _P\in {\mathbf {S}}_\kappa ({{\mathcal {U}}}',{{\mathcal {Y}}}')$. If $\theta _P\in {\mathbf {S}}_\kappa ({{\mathcal {U}}}',{{\mathcal {Y}}}')$ then $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$.

• (vi) If ${{\mathcal {U}}}={{\mathcal {Y}}}$ and $\theta$ is a symmetric function such that $\theta _{22}$ is invertible for some $\alpha \in {{\mathbb {C}}}\setminus {{\mathbb {R}}}$, then $\theta \in {\mathbf {N}}_\kappa ({{\mathcal {U}}})$ if and only if $\theta _P\in {\mathbf {N}}_\kappa ({{\mathcal {U}}}')$.

• (vii) If ${{\mathcal {U}}}={{\mathcal {Y}}}$, $\theta =\theta ^\#$ and $\theta _{22}^{-1}(0)$ exists, then $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ if and only if $\theta _P \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}') \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}')$.

Proof

(i) Suppose $\theta _P$ exists, i.e. $\theta _{22}$ in decomposition (2.1) is invertible for some point $\alpha \in \rho (\theta )$. Since $\theta$ is meromorphic on ${{\mathcal {D}}}$, so are all the entries in (2.1). To prove that $\theta _P$ is meromorphic on ${{\mathcal {D}}}$, it is now sufficient to show that $\theta _{22}^{-1}$ is meromorphic on ${{\mathcal {D}}}$, since then all the entries in (2.3) are meromorphic. To this end, note that the values of $\theta _{22}$ are operators between the spaces with the same finite dimension. Therefore, $\theta _{22}(z)$ can be identified as a square matrix, and $\theta _{22}^{-1}(z)$ has a representation $\theta _{22}^{-1}(z)=\frac{\mathrm {cof}(\theta _{22}(z))}{\det ( \theta _{22}(z))},$ where $\det (\theta _{22}(z))$ and $\mathrm {cof}(\theta _{22}(z))$ are, respectively, the determinant and the cofactor matrix of $\theta _{22}(z)$. The function $\det ( \theta _{22} )$ is not identically zero since $\theta _{22}(\alpha )$ is invertible. Since $\theta _{22}$ is meromorphic on ${{\mathcal {D}}}$, so are the functions $\det ( \theta _{22}(z))$ and $\mathrm {cof}(\theta _{22}(z))$. It follows now that $\theta _{22}^{-1}$ exists and it is meromorphic on ${{\mathcal {D}}}$, and so is $\theta _P$.

If $\theta _P$ is meromorphic on ${{\mathcal {D}}}_P$, by using the same argument as above, one can show that ${\theta _P}_{22}^{-1}$ is meromorphic on ${{\mathcal {D}}}_P$, and then it follows from (2.4) that $\theta$ is meromorphic on ${{\mathcal {D}}}_P$.

For the proof of (ii), (iii) and (iv), see [2, Lemmas 4.3.1 and 4.3.2 and Theorem 4.3.3].

(v) From the part (iv) it follows that $\theta _P$ exists. By (2.6), it holds

$$\begin{aligned} K_{\theta }(w,z)= \Phi (z)K_{\theta _P}(w,z)\Phi ^*(w) \end{aligned}$$

(2.10)

for the Schur kernels $K_{\theta }$ and $K_{\theta _P}$ of the form (1.1), whenever the functions are defined. Let $\Omega$ be a region such that $\theta$ and $\theta _P$ both are holomorphic on $\Omega$. Then, the values of $\theta _{22}$ are bijective in $\Omega ,$ and it easily follows from this fact that $\Phi ^*(w)$ is onto for every $w \in \Omega .$ Then it follows from (2.10) that $K_{\theta _P}$ restricted to $\Omega$ has the same number of negative squares than $K_{\theta }$ restricted to $\Omega$. Now an application of [2, Theorem 1.1.4] shows that unrestricted $K_{\theta _P}$ and $K_{\theta }$ have the same number of negative squares. Therefore, if $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ and $\theta _{22}^{-1}(0)$ exists, $\theta _P$ is holomorphic at the origin and $K_{\theta _P}$ has exactly $\kappa$ negative squares, so $\theta _P\in {\mathbf {S}}_\kappa ({{\mathcal {U}}}',{{\mathcal {Y}}}')$. Conversely if $\theta _P\in {\mathbf {S}}_\kappa ({{\mathcal {U}}}',{{\mathcal {Y}}}')$, the function $\theta _P$ and then also $\theta$ are holomorphic at the origin, $K_{\theta }$ has exactly $\kappa$ negative squares, so $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$.

(vi) It follows from the assumption ${{\mathcal {U}}}={{\mathcal {Y}}}$ that ${{\mathcal {U}}}'={{\mathcal {Y}}}'$, and the assumption that $\theta _{22}$ is invertible for some point guarantees that $\theta _P$ exists. Moreover, the function $\theta _P$ is also symmetric by part (ii). From these symmetry conditions it follows that $\sigma =\tau$ in (2.2) and $\Psi (z)=\Phi (z)$ in (2.5). By (2.8), it then holds

$$\begin{aligned} N_{\theta }(w,z)= \Psi (z)N_{\theta _P}(w,z)\Psi ^*(w) \end{aligned}$$

for the Nevanlinna kernels $N_{\theta }$ and $N_{\theta _P}$ of the form (1.3), whenever the functions are defined. Now the same argument as used in the proof of part (iii) shows that $N_{\theta }$ and $N_{\theta _P}$ have the same number of negative squares. Moreover, part (i) shows that if either $\theta$ or $\theta _P$ is meromorphic on ${{\mathbb {C}}}\setminus {{\mathbb {R}}}$, then so is the other. The claim now follows.

(vii) This follows straightforwardly from the parts (v) and (vi). $\square$

Remark 2.2

The assumption that $\theta _{22}^{-1}(0)$ exists in parts (v) and (vii) of Proposition 2.1 is technical; it is needed because the generalized Schur function must be analytic at the origin. If $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ and $\theta _{22}(0)$ is not invertible, it follows from part (iv) that $\theta _{22}(\alpha )$ is invertible for some $\alpha \in {{\mathbb {D}}}$. The conclusions of the part (v) of Proposition 2.1 then hold if $\theta _P(z)$ is replaced by $\theta _P(\eta (z))$, where $\eta (z)=\frac{\alpha -z}{1-{\bar{\alpha }} z}$; see [2, Sect. 2.5 B]. The same is true in the part (vii) of Proposition 2.1, if $\alpha \in (-1,1)$, since then $\eta ({\bar{z}})=\overline{\eta (z)}$ and $\theta _P(\eta ({\overline{z}}))= (\theta _P(\eta (z)))^*$. By part (iv), $\alpha$ can be chosen to be real.

In one dimensional cases, that is, when ${{\mathcal {U}}}={{\mathcal {Y}}}=-{{\mathbb {C}}}$, where $-{{\mathbb {C}}}$ is the antispace of the complex numbers, the Potapov–Ginzburg transformation reduces to transformation of the form $\theta \mapsto \theta ^{-1}$.

Corollary 2.3

A function $\theta _1$ such that $\theta _1(0)\ne 0$ belongs to ${\mathbf {S}}_{\kappa _1}(-{{\mathbb {C}}})$ if and only if $\theta _1^{-1} \in {\mathbf {S}}_{\kappa _1}({{\mathbb {C}}})$, and a function $\theta _2$ which is not identically zero belongs to ${\mathbf {N}}_{\kappa _2}(-{{\mathbb {C}}})$ if and only if $\theta _2^{-1} \in {\mathbf {N}}_{\kappa _2}({{\mathbb {C}}})$. Moreover, a function $\theta$ such that $\theta (0)\ne 0$ belongs to ${\mathbf {S}}_{\kappa _1}(-{{\mathbb {C}}}) \cap {\mathbf {N}}_{\kappa _2}(-{{\mathbb {C}}})$ if and only if $\theta ^{-1} \in {\mathbf {S}}_{\kappa _1}({{\mathbb {C}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathbb {C}}})$.

Proof

The claims follow from parts (v)–(vii) of Proposition 2.1 by choosing ${{\mathcal {U}}}={{\mathcal {Y}}}=-{{\mathbb {C}}}$, since then $\theta _P=\theta ^{-1}$ and ${{\mathcal {U}}}'={{\mathcal {Y}}}'={{\mathbb {C}}}$. $\square$

Remark 2.4

In Corollary 2.3, the roles of $-{{\mathbb {C}}}$ and ${{\mathbb {C}}}$ could be interchanged; it still holds, if one replaces $-{{\mathbb {C}}}$ by ${{\mathbb {C}}}$ and ${{\mathbb {C}}}$ by $-{{\mathbb {C}}}$. Moreover, Corollary 2.3 holds as stated, if one replaces the spaces $-{{\mathbb {C}}}$ and ${{\mathbb {C}}}$, respectively, by $-{{\mathbb {C}}}^n$ and ${{\mathbb {C}}}^n$, and changes the assumptions “$\theta _1$ not identically zero” and “$\theta _2(0)\ne 0$”, respectively, by “$\det (\theta _1)$ not identically zero” and “$\det (\theta _2(0))\ne 0$”. However, in that case, the roles of $-{{\mathbb {C}}}$ and ${{\mathbb {C}}}$ could not be interchanged, since if $n\ge 2,$ there are matrix functions in ${\mathbf {S}}_{\kappa }({{\mathbb {C}}}^n)$ such that their values are not invertible anywhere on ${{\mathbb {D}}}$.

When ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, the class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ has characterizations which do not involve the Schur kernel (1.1). For a proof of the following lemma, combine [22, Proposition 7.11] and [2, Theorem 4.2.1].

Lemma 2.5

Let ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ be Hilbert spaces, and let $\theta$ be an ${{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function holomorphic at the origin and meromorphic on ${{\mathbb {D}}}$. Then the following statements are equivalent:

• (i) $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$;

• (ii) $\theta$ has finite pole multiplicity $\kappa$ and

  $$\begin{aligned} \lim _{r \rightarrow 1^-} \sup _{|z|=r}\Vert \theta (z) \Vert \le 1 \end{aligned}$$

  holds;

• (iii) $\theta$ has factorizations of the form

  $$\begin{aligned} \theta (z)=\theta _r(z)B^{-1}_r(z) =B^{-1}_l(z)\theta _l(z), \end{aligned}$$

  where $\theta _r, \theta _l \in {{{\mathbf {S}}}( {{\mathcal {U}}},{{\mathcal {Y}}})}$, $B_r$ and $B_r$ are Blaschke products of degree $\kappa$ with values, respectively, in ${\mathcal {L}}({\mathcal {U}})$ and ${\mathcal {L}}({\mathcal {Y}})$, such that $B_r(w)f=0$ and $\theta _r(w)f=0$ for some $w \in {{\mathbb {D}}}$ only if $f=0$, and $B_l^*(w)g=0$ and $\theta _l^*(w)g=0$ for some $w \in {{\mathbb {D}}}$ only if $g=0.$

When ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are finite dimensional anti-Hilbert spaces with the same negative index, i.e. ${{\mathcal {U}}}={{\mathcal {Y}}}=-{{\mathbb {C}}}^n$, the results of Lemma 2.5 have counterparts; in particular, the analog for Lemma 2.5(ii) will be stated and proved in proposition below.

For a meromorphic function $\theta$ such that the values of $\theta$ are operators between the spaces with the same finite dimension, z is called a zero of $\theta$ if it is a pole of $\theta ^{-1}$. If ${{\mathcal {X}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, the lower bound of an operator $T:{{\mathcal {X}}}\rightarrow {{\mathcal {Y}}}$ is a value $L\ge 0$ satisfying $\Vert Tx \Vert _{{{\mathcal {Y}}}} \ge L$ for all $x \in {{\mathcal {X}}}$ such that $\Vert x\Vert _{{{\mathcal {X}}}}=1$. The operator T is called bounded below if a non-zero lower bound exists, and the best possible choice of all the lower bounds, i.e. the greatest one, is denoted as $\gamma (T)$.

Proposition 2.6

An $n \times n$-matrix valued function $\theta$ meromorphic on ${{\mathbb {D}}}$ and holomorphic at the origin belongs to ${\mathbf {S}}_{\kappa }(-{{\mathbb {C}}}^n)$ if and only if $\theta$ has exactly $\kappa$ zeros in ${{\mathbb {D}}}$, counting multiplicities, and

$$\begin{aligned} \lim _{r \rightarrow 1^{-}} \inf _{|z|=r} \gamma (\theta (z)) \ge 1. \end{aligned}$$

(2.11)

where $\gamma (\theta (z))$ is taken with respect to the usual norm of ${{\mathcal {L}}}({{\mathbb {C}}}^n)$.

Proof

The values of $\theta$ can be considered as the operators in ${{\mathcal {L}}}(-{{\mathbb {C}}}^n)$. Then, the Potapov–Ginzburg transformation $\theta _P$ of $\theta$ is $\theta ^{-1}$. Suppose $\theta \in {\mathbf {S}}_{\kappa }(-{{\mathbb {C}}}^n)$. Then, by Proposition 2.1, $\theta ^{-1}$ is meromorphic on ${{\mathbb {D}}}$. It can be assumed that $\theta ^{-1}$ exists at the origin, since if not, one only has to consider $\theta ^{-1}(\eta (z))$ as in Remark 2.2. Then $\theta ^{-1} \in {\mathbf {S}}_{\kappa }({{\mathbb {C}}}^n)$ by Proposition 2.1. It follows from Lemma 2.5 that $\theta ^{-1}$ has exactly $\kappa$ poles in ${{\mathbb {D}}}$, counting multiplicities, and it holds

$$\begin{aligned} \lim _{r \rightarrow 1^{-}} \sup _{|z|=r} \Vert \theta ^{-1}(z)\Vert \le 1. \end{aligned}$$

(2.12)

It follows now that $\theta$ has exactly $\kappa$ zeros, counting multiplicities, in ${{\mathbb {D}}},$ and (2.11) holds.

Assume then that $\theta$ has $\kappa$ zeros in ${{\mathbb {D}}}$ and (2.11) holds. It can be again assumed that $z=0$ is not a zero of $\theta$. Then $\theta ^{-1}$ is meromorphic on ${{\mathbb {D}}}$ and holomorphic at the origin, it has $\kappa$ poles and (2.12) holds. It follows from Lemma 2.5 that $\theta ^{-1} \in {\mathbf {S}}_{\kappa }({{\mathbb {C}}}^n)$, and then by Proposition 2.1 that $\theta \in {\mathbf {S}}_{\kappa }(-{{\mathbb {C}}}^n)$. $\square$

Lemma 2.5 and Proposition 2.6 show that when ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are definite, that is, Hilbert spaces or anti-Hilbert spaces, functions in the class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ can have only finite number of poles or zeros, respectively. This does not hold in general, when the spaces ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are indefinite. In that case, it is possible that $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ has infinite number of zeros and poles, as Example 2.7 below shows. However, a function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ still has some properties similar to (2.11) or (2.12). Indeed, the radial limit values of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ exists a.e. on ${{\mathbb {T}}}$, and they are contractive with respect to the indefinite inner product of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$; see Theorem 2.8 below.

Example 2.7

Let $b_1$ and $b_2$ be scalar infinite Blaschke products such that $b_2(0)\ne 0$. Consider an ${{\mathcal {L}}}\left( {{\mathbb {C}}}\oplus -{{\mathbb {C}}}\right)$-valued function $\theta (z)= {\tiny \begin{pmatrix} b_1(z) &{} 0\\ 0 &{} b_2^{-1}(z) \end{pmatrix}}$. A calculation shows that the Potapov–Ginzburg transformation $\theta _P$ of the function $\theta$ is the ${{\mathcal {L}}}\left( {{\mathbb {C}}}^2 \right)$-valued function $\theta _P(z)= {\tiny \begin{pmatrix} b_1(z) &{} 0\\ 0 &{} b_2(z) \end{pmatrix}}$. It easily follows from Lemma 2.5 that $\theta _P \in {\mathbf {S}}({{\mathbb {C}}}^2)$, and then by Proposition 2.1 that $\theta \in {\mathbf {S}}({{\mathbb {C}}}\oplus -{{\mathbb {C}}})$. Moreover, $\theta$ has infinite number of zeros and poles.

Theorem 2.8

Let ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ be Pontryagin spaces with the same negative index.

• (i) If $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, then strong radial limit values $\lim _{r \rightarrow 1^-}\theta (r\zeta )$ exist for a.e. $\zeta \in {{\mathbb {T}}},$ and the limit values are contractive with respect to the indefinite inner products of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$.

• (ii) If $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, strong radial limit values of the function $\theta$ are isometric (co-isometric) a.e. on ${{\mathbb {T}}}$ if and only if strong radial limit values of $\theta _P$ are isometric (co-isometric) a.e. on ${{\mathbb {T}}}$.

Proof

(i)The Hilbert space case is known. For ordinary Schur functions, the result is classical, see [36, Chapter V]. If $\kappa >0$ and ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, $\theta$ has Kreĭn–Langer factorizations of the form (2.5). Since inverse Blaschke products are rational functions with unitary values everywhere on ${{\mathbb {T}}}$, the result now follows from the case $\kappa =0$.

Assume then that the negative index of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ is not zero. By Proposition 2.1, the Potapov–Ginzburg transform $\theta _P$ of $\theta$ exists. It can be assumed that $\theta _{22}(0)$ invertible; if not, one only need to consider $\theta _P(\eta (z))$, where $\eta$ is as in Remark 2.2. By Proposition 2.1, $\theta _P \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}}',{{\mathcal {Y}}}')$, and since ${{\mathcal {U}}}'$ and ${{\mathcal {Y}}}'$ are Hilbert spaces, $\theta _P$ is meromorphic on ${{\mathbb {D}}}$, has strong contractive radial limit values almost everywhere on ${{\mathbb {T}}}$, and the same holds for the entries $\theta _{P_{11}}$, $\theta _{P_{12}}$, $\theta _{P_{21}}$ and $\theta _{P_{22}}$ in (2.3). By Lemma 2.5, $\theta _P$ has exactly $\kappa$ poles in ${{\mathbb {D}}}$, counting multiplicities, and therefore $\theta _{P_{22}}$ has no more than $\kappa$ poles in ${{\mathbb {D}}}$. It now follows again from Lemma 2.5 that $\theta _{P_{22}} \in {\mathbf {S}}_{\kappa '}(|{{\mathcal {U}}}_-|, |{{\mathcal {Y}}}_-|),$ where $\kappa '\le \kappa$. Then, $\theta _{P_{22}}$ has the Kreĭn–Langer factorization of the form

$$\begin{aligned} \theta _{P_{22}}= B^{-1}\theta _0, \end{aligned}$$

(2.13)

where $B^{-1}$ is an inverse Blaschke product and $\theta _0 \in {\mathbf {S}}_{0}(|{{\mathcal {U}}}_-|, |{{\mathcal {Y}}}_-|)$. The values of $\theta _{P_{22}}$ are operators between the spaces with same finite dimension, and they can be identified with square matrices. Moreover, the values of $\theta _{P_{22}}$ are by construction and Lemma 2.1 invertible at least on $\rho (\theta )\setminus \Xi$, where $\Xi$ contains at most $\kappa$ points. Since the values of $B^{-1}$ are invertible whenever they exists, it follows that the values of $\theta _0$ are invertible on $\rho (\theta )\setminus \Xi$. In particular, the function $\mathrm {det}(\theta _0)$, is not identically zero. These facts combined with (2.13) show that $\theta _{P_{22}}^{-1}(z)$ has a representation

$$\begin{aligned} \theta _{P_{22}}^{-1}(z)= \frac{\mathrm {cof}(\theta _{P_{22}}(z))}{\det (\theta _{P_{22}}(z))} =\frac{\mathrm {cof}(\theta _{P_{22}}(z))}{\det (B^{-1}(z)) \det (\theta _0(z))}, \end{aligned}$$

(2.14)

where $\mathrm {cof}$ means the cofactor matrix. The function $\theta _{P_{22}}$ is meromorphic on ${{\mathbb {D}}}$ and has strong contractive radial limit values a.e. on ${{\mathbb {T}}}$, so clearly $\mathrm {cof}(\theta _{P_{22}})$ is meromorhic in ${{\mathbb {D}}}$ and has strong radial limit values a.e. on ${{\mathbb {T}}}$. Since the values of Blaschke product $B^{-1}$ are unitary everywhere on the unit cirle, $|\det (B^{-1}(\zeta ))|=1$ for every $\zeta \in {{\mathbb {T}}}$. The values of $\theta _0$ are contractive everywhere on ${{\mathbb {D}}}$, and therefore $\det (\theta _0)$ is bounded holomorphic function in ${{\mathbb {D}}}.$ This implies that radial limit values of $\det (\theta _0)$ exist, and since $\det (\theta _0)$ is not identically zero, the radial limit values also differ from zero a.e. on ${{\mathbb {T}}}$. It now follows from (2.14) that $\theta _{P_{22}}^{-1}$ is meromorphic on ${{\mathbb {D}}}$ and has radial limit values a.e. on ${{\mathbb {T}}}$. It has been proved that all the entries in the representation of $\theta$ in (2.4) are meromorphic in ${{\mathbb {D}}}$ and have strong radial limit values a.e. on ${{\mathbb {T}}}$, so the same holds for $\theta$. The fact that the radial limit values of $\theta$ are contractive with the respect to the inner products of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ follows now easily from the identity (2.6) in Proposition 2.1, since the radial limit values of $\theta _P$ are contractive.

(ii) Consider the identities (2.6) and (2.7) from Proposition 2.1. The claims follow from these identities if one proves that the strong radial limit values of $\Phi$ and $\Psi$ exist and are onto a.e. on ${{\mathbb {T}}}$. It follows from the part (i) that all the entries in the definition of $\Phi$ in (2.5) have strong radial limit values a.e. on ${{\mathbb {T}}}$, so the same holds for $\Phi$. Since $\theta _{22}^{-1}=\sigma ^{-1} {\theta _P}_{22}\tau$ and the strong radial limit values of ${\theta _P}_{22}$ exist a.e. on ${{\mathbb {T}}}$, the strong radial limit values of ${\theta }_{22}^{-1}$ also exist a.e. on ${{\mathbb {T}}}$. Especially, the strong radial limit values of ${\theta }_{22}$ are invertible a.e. on ${{\mathbb {T}}}$. An easy calculation then shows that the strong radial limit values of $\Phi$ are onto a.e. on ${{\mathbb {T}}}$. Similar argument shows that the same holds for $\Psi$, and the claims follow. $\square$

In the special case where ${{\mathcal {U}}}={{\mathcal {Y}}}$ and ${{\mathcal {U}}}$ is finite dimensional, Theorem 2.8 above could be derived from [1, Theorem 6.8]. A function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, is called inner (co-inner, bi-inner), if the radial limit values of $\theta$ are isometric (co-isometric, unitary) a.e. on ${{\mathbb {T}}}$. By using a similar notion as in [1, 6, 8, 21], a function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, is called a generalized ${\mathcal {J}}$-inner (co-${\mathcal {J}}$-inner, bi-${\mathcal {J}}$-inner ) function, if the radial limit values of $\theta$ are isometric (co-isometric, unitary) a.e. on ${{\mathbb {T}}}$, with respect to the inner product of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$. Following [12]; see also [31, Sect. 4], the class ${\mathbf {U}}_\kappa ({{\mathcal {U}}},{{\mathcal {Y}}})$ is defined to be the subclass of the generalized bi-${\mathcal {J}}$-inner functions in ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}.$ The class ${\mathbf {U}}_\kappa ({{\mathcal {U}}},{{\mathcal {U}}})$ is written as ${\mathbf {U}}_\kappa ({{\mathcal {U}}}).$ For a symmetric function, it is evident that if the radial values are isometric or co-isometric a.e., they are also unitary.

3 Linear systems, self-adjoint realizations and similarity mappings in state spaces

If needed, the colligation, or the system, of the form (1.4) will be written as $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}},{{\mathcal {Y}}}).$ Often in this paper, ${{\mathcal {U}}}={{\mathcal {Y}}}$ and it will be then written $\Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}})$. In what follows, unless otherwise stated, the state space ${{\mathcal {X}}}$ and the spaces ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are assumed to be Pontryagin spaces, which will be indicated by the notation $\Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}}, {{\mathcal {Y}}}; \kappa )$ where $\kappa$ is reserved for the negative index of ${{\mathcal {X}}}.$ Note that the common negative index of ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ is not assumed to be related to $\kappa$. The adjoint or dual of the system $\Sigma$ is the system $\Sigma ^*$ such that its system operator is the indefinite adjoint $T_{\Sigma }^*$ of $T_\Sigma .$ That is, $\Sigma ^*=(T_{\Sigma }^*;{{\mathcal {X}}},{{\mathcal {Y}}},{{\mathcal {U}}}).$ In this paper, all the adjoints are with respect to the indefinite inner products in question. The identity $\theta _{\Sigma ^*}(z)={\theta _{\Sigma }}^\#(z)$ holds for the transfer function $\theta _{\Sigma ^*}$ of the dual system $\Sigma ^*.$

The following subspaces

$$\begin{aligned}&{{\mathcal {X}}}^c:={\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,A^{n}B:\,n=0,1,\ldots \} \end{aligned}$$

(3.1)

$$\begin{aligned}&{{\mathcal {X}}}^o:= {\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,A^{*n}C^*:\,n=0,1,\ldots \} \end{aligned}$$

(3.2)

$$\begin{aligned}&{{\mathcal {X}}}^{s}:= {\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,A^{n}B,\mathrm{ran}\,A^{*m}C^*:\,n,m=0,1,\ldots \}, \end{aligned}$$

(3.3)

are called, respectively, controllable, observable and simple subspaces. The system is said to be controllable (observable, simple) if ${{\mathcal {X}}}^c={{\mathcal {X}}}({{\mathcal {X}}}^o={{\mathcal {X}}},{{\mathcal {X}}}^s={{\mathcal {X}}})$ and minimal if it is both controllable and observable. When $\Omega \ni 0$ is some symmetric neighbourhood of the origin, that is, ${\bar{z}} \in \Omega$ whenever $z\in \Omega ,$ then also

$$\begin{aligned} {{\mathcal {X}}}^c= & {} {\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,(I-zA)^{-1}B: z \in \Omega \} \end{aligned}$$

(3.4)

$$\begin{aligned} {{\mathcal {X}}}^o= & {} {\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,(I-zA^*)^{-1}C^*: z \in \Omega \} \end{aligned}$$

(3.5)

$$\begin{aligned} {{\mathcal {X}}}^{s}= & {} {\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,(I-zA)^{-1}B,\mathrm{ran}\,(I-wA^*)^{-1}C^*: z,w \in \Omega \} \end{aligned}$$

(3.6)

In the case where all the spaces are Hilbert spaces, it is well known; see for instance [8, Proposition 8], that the transfer function of the passive system is an ordinary Schur function. In general case where ${{\mathcal {X}}}$, ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces such that ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ have the same negative index, the transfer function of the passive system $\Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}};\kappa )$ is a generalized Schur function, with the index not larger that the negative index of the state space [32, Proposition 2.4]. Conversely, every $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ has a realization of the form (1.5), and the realization can be chosen such that it is controllable isometric (observable co-isometric, simple conservative, minimal passive) [2, Chapter 2], [32, Lemma 2.8]. Any two controllable isometric (observable co-isometric, simple conservative) realizations of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ are unitarily similar [2, Theorem 2.1.3]. Two given realizations $\Sigma _1=(A_1,B_1,C_1,D_1;{{\mathcal {X}}}_1,{{\mathcal {U}}},{{\mathcal {Y}}};\kappa _1)$ and $\Sigma _2=(A_2,B_2,C_2,D_2;{{\mathcal {X}}}_2,{{\mathcal {U}}},{{\mathcal {Y}}};\kappa _2)$ of the same ${{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function $\theta$ analytic at the origin are called unitarily similar if $D_1=D_2$ and there exists a unitary operator $U: {{\mathcal {X}}}_1 \rightarrow {{\mathcal {X}}}_2$ such that

$$\begin{aligned} A_1=U^{-1}A_2U, \quad B_1=U^{-1}B_2, \quad C_1=C_2U. \end{aligned}$$

(3.7)

Unitary similarity preserves dynamical properties of the system and also the spectral properties of the main operator. Moreover, it easily follows that if the realizations are unitarily similar, their state spaces have the same negative index.

The realizations $\Sigma _1$ and $\Sigma _2$ above are said to be weakly similar if $D_1=D_2$ and there exists an injective closed densely defined possible unbounded linear operator $Z: {{\mathcal {X}}}_1 \rightarrow {{\mathcal {X}}}_2$ with the dense range such that

$$\begin{aligned} ZA_1x=A_2Zx, \quad C_1x=C_2Zx, \quad x\in {{\mathcal {D}}}(Z),\quad \text{ and } \quad ZB_1=B_2, \end{aligned}$$

(3.8)

where ${{\mathcal {D}}}(Z)$ is the domain of Z. It is known that two minimal realizations of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})},$ or more generally, any ${{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function holomorphic at the origin, are weakly similar; see [32, Proposition 2.2], [31, Theorem 2.5] and [35, p. 702].

For a generalized Nevanlinna function $\theta \in {{\mathbf {N}}_{\kappa }({{\mathcal {U}}})}$ in the special case where ${{\mathcal {U}}}$ is a Hilbert space, the realization of $\theta$ usually means a representation of the form

$$\begin{aligned} \theta (z)=\theta (z_0)^* + (z-\overline{z_0})\Gamma ^*( I +(z-z_0)(H-z)^{-1} )\Gamma , \end{aligned}$$

(3.9)

such that ${{\mathcal {X}}}$ is a Pontryagin space, $\Gamma \in {{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {X}}}),$ H is a self-adjoint linear relation in ${{\mathcal {X}}}$ and $z_0$ is some fixed point in $\rho (H) \cap {{\mathbb {C}}}^+,$ where $\rho (H)$ is the field of regularity of H. In fact, $\theta$ is a generalized Nevanlinna function if and only if it has a representation of the form (3.9) [25, 29]. The realization can be chosen such that the negative index of ${{\mathcal {X}}}$ coincides with the index $\kappa$ of $\theta \in {{\mathbf {N}}_{\kappa }({{\mathcal {U}}})}$, and it holds

$$\begin{aligned} {{\mathcal {X}}}= {\overline{{\mathrm{span}}}}\,\left\{ ( I +(z-z_0)(H-z)^{-1} )\Gamma u: \quad z \in \rho (H), \quad u \in {{\mathcal {U}}}\right\} . \end{aligned}$$

In that case, the realization is unique up to unitary equivalence.

In general, a function $\theta \in {{\mathbf {N}}_{\kappa }({{\mathcal {U}}})}$ is not necessary holomorphic at the origin, and therefore it cannot be realized in the form (1.6). However, a self-adjoint system with a Pontryagin state space always induces some generalized Nevanlinna function.

Proposition 3.1

Let $\Sigma =(A,B,B^*,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa )$ be a self-adjoint system. Then the transfer function $\theta$ of $\Sigma$ belongs to the generalized Nevanlinna class ${\mathbf {N}}_{\kappa '}({{\mathcal {U}}}),$ where $\kappa '$ is the dimension of a maximal negative subspace of

$$\begin{aligned} \mathrm {span}\{\mathrm{ran}\,(I-zA)^{-1}B: z \in \Omega \}, \end{aligned}$$

(3.10)

where $\Omega$ is some sufficiently small symmetric neighbourhood of the origin.

Proof

Since $\Sigma$ is self-adjoint, A and D must be self-adjoint operators, ${{\mathcal {U}}}={{\mathcal {Y}}}$, $\theta (z)=\theta ^\#(z),$ and $B^*=C$. Then the spaces (3.1)–(3.3) coincide. It follows from [15, Corollary 3.15, pp. 106] that the non-real spectrum of A consists of not more than $2\kappa$ (counting multiplicities) eigenvalues situated symmetrically with respect to the real axis. Since $(I-zA)^{-1}$ exists whenever 1/z is in the resolvent set $\rho (A)$ of A,  it follows that $\theta (z)=D+zB^*(I-zA)^{-1}B$ is meromorphic on ${{\mathbb {C}}}\setminus {{\mathbb {R}}}$ with at most $2\kappa$ non-real poles. By using the resolvent identity; cf. also [2, Theorem 1.2.4], and the fact that the system operator is self-adjoint, one deduces that the Nevanlinna kernel of $\theta$ can be represented as

$$\begin{aligned} N_{\theta }(w,z) = \frac{\theta (z)-\theta ^*(w)}{z-{\bar{w}}} =B^*(I-zA^*)^{-1} (I-{\bar{w}}A)^{-1}B. \end{aligned}$$

(3.11)

Therefore, it follows from [2, Lemma 1.1.1’] that the number of negative eigenvalues of the Gram matrix of the form

$$\begin{aligned} \left( \left\langle N_\theta (w_j,w_i)f_j,f_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n} = \left( \left\langle (I-\bar{w_j}A)^{-1}B f_j, (I-\bar{w_i}A)^{-1}B f_i \right\rangle _{{{\mathcal {X}}}}\right) _{i,j=1}^{n}, \end{aligned}$$

where $f_i \in {{\mathcal {U}}}$ and $w_i \in {{\mathbb {C}}}\setminus {{\mathbb {R}}}$, $i=1,\ldots ,n$, coincides with the dimension of a maximal negative subspace of the span of $\{ (I-\bar{w_i}A)^{-1}B f_i \}_{i=1}^n$. It now follows that the Nevanlinna kernel $N_\theta$ has $\kappa '$ negative squares, where $\kappa$ is the dimension of a maximal negative subspace of (3.10), and the proof is complete. $\square$

By using the fact that the transfer function of the passive system (1.5) is a generalized Schur function with the index not larger than the negative index of the state space of $\Sigma$, it follows from Proposition 3.1 that the transfer function of a passive self-adjoint system is both a generalized Schur function and a generalized Nevanlinna function. Moreover, if ${{\mathcal {U}}}$ is a Hilbert space, the negative indices coincide. Some further machinery from the Kreĭn space operator theory will be needed to prove this.

Let ${{\mathcal {X}}}$ be a Kreĭn space. The negative index $\mathrm {ind}_{-}(H)$, with respect to the inner product of ${{\mathcal {X}}}$, of the bounded self-adjoint operator $H \in {{\mathcal {L}}}({{\mathcal {X}}})$ is defined to be the supremum of all positive integers n such that there exists an invertible and nonpositive matrix of the form $\left( \left\langle Hx_j,x_i \right\rangle _{{{\mathcal {X}}}}\right) _{i,j=1}^{n},$ where $\{x_k\}_{k=1}^n\subset {{\mathcal {X}}}.$ If such a matrix does not exists for any n,  then $\mathrm {ind}_{-}(H)$ is defined to be zero. In that case, the operator H is nonnegative with respect to the inner product of ${{\mathcal {X}}}.$. In general, the negative index of the self adjoint operator measures how much the operator behaves like a positive operator. For an arbitrary $T\in {{\mathcal {L}}}({{\mathcal {X}}},{{\mathcal {Y}}}),$ the operator $T^*T$ is a bounded self adjoint operator in ${{\mathcal {L}}}({{\mathcal {X}}}),$ and it is easy to deduce that T is contractive if and only if $\mathrm {ind}_{-}(I-T^*T)=0.$

It well known; cf. [15, Theorem 3.4 on p. 267.], [23, Lecture 2], that every bounded linear operator between Kreĭn spaces can be dilated to unitary operator. In this paper, the following version of that result, which can be derived from [23, Theorems 2.3 and 2.4], is needed.

Theorem 3.2

Suppose that $A \in {{\mathcal {L}}}( {{\mathcal {X}}}_1 , {{\mathcal {X}}}_2)$ where ${{\mathcal {X}}}_1$ and ${{\mathcal {X}}}_2$ are Pontryagin spaces with the same negative index. Then there exist Kreĭn spaces ${{\mathfrak {D}}}_{A}$ and ${{\mathfrak {D}}}_{A^*}$ with

$$\begin{aligned} \mathrm {ind}_{-}(I-A^*A)=\mathrm {ind}_{-}({{\mathfrak {D}}}_{A})=\mathrm {ind}_{-}({{\mathfrak {D}}}_{A^*})= \mathrm {ind}_{-}(I-AA^*) , \end{aligned}$$

and linear operators $D_A \in {{\mathcal {L}}}({{\mathfrak {D}}}_{A},{{\mathcal {X}}}_1)$ and $D_{A^*}\in {{\mathcal {L}}}( {{\mathfrak {D}}}_{A^*} , {{\mathcal {X}}}_2)$ with zero kernels and a linear operator $L \in {{\mathcal {L}}}({{\mathfrak {D}}}_{A} , {{\mathfrak {D}}}_{A^*})$ such that it holds

$$\begin{aligned} I-A^*A=D_{A}D_{A}^*, \qquad I-AA^*=D_{A^*}D_{A^*}^*. \end{aligned}$$

Furthermore, the operator

$$\begin{aligned} U_A:= \begin{pmatrix} A &{} D_{A^*} \\ D^*_{A} &{} -L^* \end{pmatrix}: \begin{pmatrix} {{\mathcal {X}}}_1 \\ {{\mathfrak {D}}}_{A^*} \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}_2 \\ {{\mathfrak {D}}}_{A} \end{pmatrix} \end{aligned}$$

(3.12)

is unitary. Moreover, if $\mathrm {ind}_{-}(I-A^*A)= \mathrm {ind}_{-}(I-AA^*)$ is finite, then $U_A$ is essentially unique.

The operator $U_A$ in Theorem 3.2 is called as a Julia operator of A,  the operators $D_A$ and $D_{A^*}$ are called, respectively, defect operators of A and $A^*$, and the spaces ${{\mathfrak {D}}}_A$ and ${{\mathfrak {D}}}_{A^*}$ are called, respectively, defect spaces of A and $A^*$. In general, any bounded operator V with the zero kernel is called as a defect operator of A if it holds $I-A^*A=VV^*.$ Julia operator of A is essentially unique, if for any other Julia operator

$$ U_A'= \begin{pmatrix} A &{} {D_{A^*}}' \\ {{D_{A}}'}^* &{} -{{L}'}^* \end{pmatrix}: \begin{pmatrix} {{\mathcal {X}}}_1 \\ {{{\mathfrak {D}}}_{A^*}}' \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}_2 \\ {{{\mathfrak {D}}}_{A}}' \end{pmatrix}, $$

of A, there exists unitary operators $H_1: {{{\mathfrak {D}}}_{A^*}}\rightarrow {{{\mathfrak {D}}}_{A^*}}'$ and $H_2: {{{\mathfrak {D}}}_{A}} \rightarrow {{{\mathfrak {D}}}_{A}}'$ such that

$$ {D_{A^*}} ={D_{A^*}}'H_1, \qquad {D_{A}} ={D_{A}}'H_2, \qquad H_1L =L' H_2. $$

If $\theta$ is the transfer function of the system (1.5), the Schur kernel of the form (1.1) can be represented as a sum of two kernels. This can be done by using the defect operators of the system operator and its adjoint. A special case, where the system is passive, i.e. the system operator is contractive, is proved in [32, Lemma 2.4]; see also the proof of [34, Theorem 2.2]. The proofs given therein can be applied word by word to get the next result, since the existence of defect operator is guaranteed by Theorem 3.2. Therefore, the proof will not be repeated here.

Lemma 3.3

Let $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}},{{\mathcal {Y}}};\kappa )$ be a system with the transfer function $\theta$. Denote the system operator of $\Sigma$ as T. If

$$\begin{aligned} D_{T} = \begin{pmatrix} D_{T_{,1}}\\ D_{T_{,2}} \end{pmatrix}:{{\mathcal {D}}}_{T} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {U}}}\end{pmatrix}\quad D_{T^*} = \begin{pmatrix} D_{T_{,1}^*}\\ D_{T_{,2}^*} \end{pmatrix}: {{\mathcal {D}}}_{T^*} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {Y}}}\end{pmatrix}, \end{aligned}$$

(3.13)

are defect operators of T and $T^*$, respectively, then the identities

$$\begin{aligned} I_{{{\mathcal {Y}}}}- \theta (z) \theta ^*(w)= & {} (1 -z{\bar{w}}) G(z)G^*(w) +\psi (z) \psi ^*(w) ,\end{aligned}$$

(3.14)

$$\begin{aligned} I_{{{\mathcal {U}}}}-\theta ^* (w) \theta (z)= & {} (1 -z{\bar{w}}) F^*(w)F(z) +\varphi ^*(w) \varphi (z), \end{aligned}$$

(3.15)

with

$$\begin{aligned} G(z)= & {} C(I_{{{\mathcal {X}}}}-zA)^{-1}, \qquad \psi (z)= D_{T_{,2}^*}+zC(I_{{{\mathcal {X}}}}-zA)^{-1} D_{T^*_{,1}}, \end{aligned}$$

(3.16)

$$\begin{aligned} F(z)= & {} (I_{{{\mathcal {X}}}}-zA)^{-1}B, \qquad \varphi (z)= D_{T_{,2}}^* +zD_{T_{,1}}^*(I_{{{\mathcal {X}}}}-zA)^{-1}B, \end{aligned}$$

(3.17)

hold for every z and w in a sufficiently small symmetric neighbourhood of the origin.

The system (1.5) can be expanded to a larger system such that the state space and the main operator will not change. This expansion is called an embedding. The embedding of the system (1.5) is any system determined by the system operator

$$\begin{aligned} T_{{\widetilde{\Sigma }}}&= \begin{pmatrix} A &{} {\widetilde{B}} \\ {\widetilde{C}} &{} {\widetilde{D}} \end{pmatrix} = \begin{pmatrix} A&\begin{pmatrix} B &{} B_1 \end{pmatrix} \\ \begin{pmatrix} C \\ C_1 \end{pmatrix} &{} \begin{pmatrix} D &{} D_{12} \\ D_{21} &{} D_{22} \end{pmatrix} \end{pmatrix}: \begin{pmatrix} {{\mathcal {X}}}\\ \begin{pmatrix} {{\mathcal {U}}}\\ {{\mathcal {U}}}' \end{pmatrix} \end{pmatrix}\rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ \begin{pmatrix} {{\mathcal {Y}}}\\ {{\mathcal {Y}}}' \end{pmatrix} \end{pmatrix}, \end{aligned}$$

where ${{\mathcal {U}}}'$ and ${{\mathcal {Y}}}'$ are Hilbert spaces. The transfer function of the embedded system is

$$\begin{aligned} \begin{aligned} \theta _{{\widetilde{\Sigma }}}(z ) = \begin{pmatrix} D + zC(I_{{{\mathcal {X}}}}-zA)^{-1}B &{} D_{12}+zC(I_{{{\mathcal {X}}}}-zA)^{-1}B_1 \\ D_{21} +zC_1(I_{{{\mathcal {X}}}}-zA)^{-1}B &{} D_{22} +zC_1(I_{{{\mathcal {X}}}}-zA)^{-1}B_1 \end{pmatrix} = \begin{pmatrix} \theta _{\Sigma } (z) &{} \theta _{12}(z)\\ \theta _{21}(z) &{} \theta _{22}(z) \end{pmatrix}, \end{aligned} \end{aligned}$$

where $\theta _{\Sigma }$ is the transfer function of the original system.

Proposition 3.4

If $\Sigma =(A,B,B^*,D;{{\mathcal {U}}};\kappa )$ is a passive self-adjoint system, its transfer function $\theta$ belongs to ${\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ where $\kappa _1\le \kappa _2$ and $\kappa _2$ is the dimension of a maximal negative subspace of

$$ \mathrm {span}\{\mathrm{ran}\,(I-zA)^{-1}B: z \in \Omega \}, $$

where $\Omega$ is a sufficiently small symmetric neighbourhood of the origin . Moreover, if ${{\mathcal {U}}}$ is a Hilbert space, then $\kappa _1=\kappa _2.$

Proof

It follows from Proposition 3.1 that $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}).$ Moreover, since $\Sigma$ is passive, $\theta$ is also a generalized Schur function with the negative index $\kappa _1$, which is not larger than the negative index $\kappa$ of the state space ${{\mathcal {X}}}.$ By using Lemma 3.3, the equation (3.14) and a result from [2, Theorem 1.5.5], it follows that $\kappa _1 \le \kappa _1' +\kappa _2',$ where $\kappa _1'$ and $\kappa _2'$ are the negative indices of the kernels $\left( 1-z{\bar{w}} \right) ^{-1} \left( \psi (z) \psi ^*(w)\right)$ and $G(z)G^*(w)$ in (3.14), respectively. Since $\Sigma$ is self-adjoint system, $A=A^*$ and $C=B^*$. Then the same argument as in the proof of Proposition 3.1 shows that $\kappa _2'=\kappa _2$. Since $\Sigma$ is passive, the system operator of T and its adjoint $T^*$ are contractive. Therefore, $\psi ^*(w)$ is an operator in a Hilbert space ${\mathfrak {D}}_{T^*}$, and it follows that the kernel $\left( 1-z{\bar{w}} \right) ^{-1} \left( \psi (z) \psi ^*(w)\right)$ has no negative square; for details, see the proof of [32, Proposition 2.4]. Therefore $\kappa _1\le \kappa _2$.

Assume then that ${{\mathcal {U}}}$ is a Hilbert space. Denote the system operator of $\Sigma$ as T. Theorem 3.2 guarantees the existence of the defect operator $D_{T}$ of T with the properties described therein. Since $T=T^*$ is contractive, the domain ${{\mathcal {D}}}_{T}$ of $D_T$ is a Hilbert space. Moreover, (3.12) shows that it holds

$$ \begin{pmatrix} T&D_T \end{pmatrix} \begin{pmatrix} T&D_T \end{pmatrix}^* = TT^* + D_TD_T^* =T^2+ D_TD_T^*=I. $$

That is, the operator $\begin{pmatrix} T&D_T \end{pmatrix}$ is co-isometric. Therefore the system ${\widetilde{\Sigma }}$ with the system operator

$$\begin{aligned} T_{{\widetilde{\Sigma }}}= \begin{pmatrix} A&\begin{pmatrix} B &{} D_{T_{,1}} \end{pmatrix} \\ B^* &{} \begin{pmatrix} D &{} D_{T_{,2}} \end{pmatrix} \end{pmatrix} : \begin{pmatrix} {{\mathcal {X}}}\\ \begin{pmatrix} {{\mathcal {U}}}\\ {{\mathcal {D}}}_{T} \end{pmatrix} \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ {{\mathcal {U}}}\end{pmatrix} \end{aligned}$$

is a co-isometric embedding of $\Sigma .$ The transfer function of ${\widetilde{\Sigma }}$ is given by

$$\begin{aligned} {\widetilde{\theta }}= \begin{pmatrix} D&D_{T_{,2}} \end{pmatrix} + zB^* (I-zA^*)^{-1} \begin{pmatrix} B&D_{T_{,1}} \end{pmatrix} = \begin{pmatrix} \theta&\psi \end{pmatrix} \end{aligned}$$

(3.18)

where $\psi$ is defined as in (3.16). Since the system $\Sigma$ is self-adjoint, the identity (3.11) holds. By applying (3.14) from Lemma 3.3, it follows that

$$\begin{aligned} K_{{\widetilde{\theta }}}(w,z) =B^*(I-zA^*)^{-1} (I-{\bar{w}}A)^{-1}B= N_{{\theta }}(w,z). \end{aligned}$$

Since $N_{{\theta }}$ has $\kappa _2$ negative squares, so has $K_{{\widetilde{\theta }}}$, and therefore ${\widetilde{\theta }}$ is a generalized Schur function with the index $\kappa _2.$ The first identity in (3.18) shows that the total number of poles, counting multiplicities, of ${\widetilde{\theta }}$ and $\theta$ are equal. It then follows from Lemma 2.5 that ${\widetilde{\theta }}$ and $\theta$ have the same index, and the proof is complete. $\square$

Theorem 3.5

If ${{\mathcal {U}}}$ is a Hilbert space, then $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ has a minimal self-adjoint realization $\Sigma =(A,B,B^*,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa _2).$

Proof

Define ${\check{\theta }}(z)=-\theta \left( 1/z \right) +\theta (0)$. Then ${\check{\theta }}$ clearly is meromorphic on ${{\mathbb {C}}}\setminus {{\mathbb {R}}},$ analytic at the infinity with ${\check{\theta }}(\infty )=0$ and it holds ${\check{\theta }}(z)={\check{\theta }}^\#(z).$ Moreover, for any finite set of points $\{w_1,\ldots ,w_n\}\subset \rho ({\check{\theta }}),$ and vectors $\{f_1,\ldots ,f_n\} \subset {{\mathcal {U}}},$ it holds

$$\begin{aligned} \left( \left\langle {N}_{{\check{\theta }}}(w_j,w_i)f_j,f_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n}&=\left( \left\langle \frac{ \theta ^*\left( \frac{1}{{w_j}}\right) -\theta \left( \frac{1}{w_i}\right) }{w_i-{\overline{w_j}}} f_j,f_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n} \\&= \left( \left\langle \frac{ \theta \left( \frac{1}{w_i}\right) -\theta ^*\left( \frac{1}{{w_j}}\right) }{ \frac{1}{ w_i} - \frac{1}{{\overline{w_j}}}} w_if_j,w_jf_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n} \\&= \left( \left\langle {N}_{{\theta }}\left( \frac{1}{w_j},\frac{1}{w_i}\right) w_if_j,w_jf_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n} \\&= \left( \left\langle {N}_{{\theta }}\left( {w_j'},{w_i'}\right) f_j',f_i' \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n}. \end{aligned}$$

The identity above yields that ${N}_{{\theta }}$ and ${N}_{{\check{\theta }}}$ have the same number of negative squares, and therefore ${\check{\theta }} \in {\mathbf {N}}_{\kappa _1}({{\mathcal {U}}}) .$ Since $\theta$ is holomorphic at the origin, it has the Neumann series of the form $\theta (z)=\sum _{n=0}^\infty \theta _nz^n,$ for every $z \in \Omega ,$ where $\Omega$ is a sufficiently small symmetric neighbourhood of the origin. Therefore $\lim _{z \rightarrow 0}z^{-1} \left( \theta (z)-\theta (0)\right) =\theta _1,$ and also

$$ \lim _{z \rightarrow \infty }z {\check{\theta }}(z) = - \lim _{z \rightarrow \infty }z \left( \theta \left( \frac{1}{z}\right) -\theta (0)\right) =- \lim _{z' \rightarrow 0}z^{'-1} \left( \theta \left( {z'}\right) -\theta (0)\right) =-\theta _1. $$

This implies $y \lim _{y \rightarrow \infty } \left| \left\langle {\check{\theta }}(iy)f,f\right\rangle _{{{\mathcal {U}}}} \right| <\infty$ for every $f\in {{\mathcal {U}}}.$ Therefore, ${\check{\theta }}$ has the realization of the form (3.9) which reduces to ${\check{\theta }}(z)= {\widehat{B}}^*({\widehat{A}}-z)^{-1}{\widehat{B}}$, where ${\widehat{B}} \in {{\mathcal {L}}}({{\mathcal {U}}},{{\mathcal {X}}})$ and ${\widehat{A}}$ is a self-adjoint operator in a Pontryagin space ${\widehat{{{\mathcal {X}}}}}$ with the negative index $\kappa _2$ [28, 33, pp. 348–349]. But then $\theta$ can be realized as

$$ \theta (z)=D - {\widehat{B}}^*\left( {\widehat{A}}-\frac{1}{z}\right) ^{-1}{\widehat{B}} = D + z{\widehat{B}}^*\left( I -z{\widehat{A}}\right) ^{-1}{\widehat{B}}, $$

where $D=\theta (0)=\theta ^*(0).$ That is, ${\widehat{\Sigma }}=({\widehat{A}},{\widehat{B}},{\widehat{B}}^*,D;{\widehat{{{\mathcal {X}}}}},{{\mathcal {U}}};\kappa _2)$ is a self-adjoint realization of $\theta .$ It follows from Proposition 3.1 that the dimension of the maximal negative subspace of $\mathrm {span}\{\mathrm{ran}\,(I-z{\widehat{A}})^{-1}{\widehat{B}}: z \in \Omega \}:={\mathfrak {S}}$, where $\Omega$ is some sufficiently small symmetric neighbourhood of the origin, is $\kappa _2$, the negative index of ${\widehat{{{\mathcal {X}}}}}$. Then, the closure ${\widehat{{{\mathcal {X}}}}}^c$ of ${\mathfrak {S}}$ must be a regular subspace of ${\widehat{{{\mathcal {X}}}}}.$ Therefore, ${\widehat{{{\mathcal {X}}}}} = {\widehat{{{\mathcal {X}}}}}^c \oplus \left( {\widehat{{{\mathcal {X}}}}}^c\right) ^\perp ,$ and $\left( {\widehat{{{\mathcal {X}}}}}^c\right) ^\perp$ is a Hilbert subspace of ${\widehat{{{\mathcal {X}}}}}.$ Since $\Sigma$ is self-adjoint system, the spaces ${\widehat{{{\mathcal {X}}}}}^c$, ${\widehat{{{\mathcal {X}}}}}^o$ and ${\widehat{{{\mathcal {X}}}}}^s$ coincide. These facts and (3.1) imply that the system operator ${\widehat{T}}$ of ${\widehat{\Sigma }}$ can be represented as

$$\begin{aligned} {{\widehat{T}}} = \begin{pmatrix} \begin{pmatrix} A_1 &{} 0 \\ 0 &{} A \end{pmatrix} &{} \begin{pmatrix} 0 \\ B \end{pmatrix}\\ \begin{pmatrix} 0 &{} B^* \end{pmatrix}&D \end{pmatrix}: \begin{pmatrix} \begin{pmatrix} ({\widehat{{{\mathcal {X}}}}}^c)^\perp \\ {\widehat{{{\mathcal {X}}}}}^c \end{pmatrix} \\ {{\mathcal {U}}}\end{pmatrix} \rightarrow \begin{pmatrix} \begin{pmatrix} ({\widehat{{{\mathcal {X}}}}}^c)^\perp \\ {\widehat{{{\mathcal {X}}}}}^c\end{pmatrix} \\ {{\mathcal {U}}}\end{pmatrix}. \end{aligned}$$

(3.19)

Define ${{\mathcal {X}}}= {\widehat{{{\mathcal {X}}}}}^c.$ A calculation shows that ${\widehat{A}}^n{\widehat{B}}=A^nB$ for every $n \in {{\mathbb {N}}}_0= \{0,1,2,\ldots \}$ and therefore ${\overline{{\mathrm{span}}}}\,\{\mathrm{ran}\,A^{n}B:\,n=0,1,\ldots \} ={{\mathcal {X}}}.$ That is, $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa _1)$ is a self-adjoint minimal realization of $\theta .$ $\square$

Remark 3.6

The realization $\Sigma$ in Theorem 3.5 is not shown to be passive. In the case where ${{\mathcal {U}}}$ is a Hilbert space and $\theta \in {\mathbf {S}}({{\mathcal {U}}})\, \cap \, {\mathbf {N}}({{\mathcal {U}}})$, that is, when $\theta$ is an ordinary Schur and Nevanlinna function, it is known from [4, Theorem 5.1] that there exists a minimal self-adjoint passive realization $\Sigma$ of $\theta .$ In general, if $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$, where ${{\mathcal {U}}}$ is Pontryagin space, it follows from Proposition 3.4 that a self-adjoint minimal realization $\Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}};\kappa )$ of $\theta$ can be passive only if $\kappa _1 \le \kappa _2=\kappa ,$ and in the case where ${{\mathcal {U}}}$ is a Hilbert space, only if $\kappa _1 = \kappa _2=\kappa .$

The conditions of Theorem 3.5 that ${{\mathcal {U}}}$ is a Hilbert space there and $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ can be relaxed slightly; with a cost of weakened conclusions.

Proposition 3.7

Let ${{\mathcal {U}}}$ be a Pontryagin space. Then a symmetric function $\theta \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}})$ has a self-adjoint realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}})$ where ${{\mathcal {X}}}$ is a Kreĭn space.

Proof

Let $\Sigma _1=(A,B,C,D;{{\mathcal {X}}}_1,{{\mathcal {U}}};\kappa )$ be a simple conservative realization of $\theta .$ Since $\theta$ is symmetric in sense $\theta (z)=\theta ^\#(z)$, it holds $D=D^*$ and the dual system $\Sigma _1^*=(A^*,C^*,B^*,D;{{\mathcal {X}}}_1,{{\mathcal {U}}};\kappa )$ of $\Sigma _1$ is also a simple conservative realization of $\theta$. Therefore $\Sigma _1$ and $\Sigma _1^*$ are unitarily similar, that is, there exists a unitary mapping $J: {{\mathcal {X}}}_1 \rightarrow {{\mathcal {X}}}_1$ such that, see (3.7),

$$\begin{aligned} A=J^*A^*J, \qquad JB=C^*, \qquad C=B^*J, \qquad J^{-1}=J^*. \end{aligned}$$

(3.20)

The letter J is used, because the operator J is now also self-adjoint in ${{\mathcal {X}}}_1$. Indeed, let $N \in {{\mathbb {N}}}_0$. Easy calculations show that it holds

$$\begin{aligned} J \left( \sum _{n=0}^N A^nB \right) = \sum _{n=0}^N {A^{^*}}^nC^* = \sum _{n=0}^N {A^{^*}}^nJ^*B = J^* \left( \sum _{n=0}^N A^nB \right) , \end{aligned}$$

(3.21)

and similarly

$$\begin{aligned} J \left( \sum _{n=0}^N {A^{^*}}^nC^* \right) =J^* \left( \sum _{n=0}^N {A^{^*}}^nC^* \right) . \end{aligned}$$

(3.22)

Since $\Sigma _1$ is simple, it follows from (3.21) and (3.22) that J and $J^*$ coincide on a dense lineal of ${{\mathcal {X}}}_1,$ and then by continuity, everywhere. That is, J is unitary and self-adjoint. Now introduce the inner product space ${{\mathcal {X}}},$ which coincides with ${{\mathcal {X}}}_1$ as a vector space but which is endowed with the inner product $\langle x, y\rangle _{{{\mathcal {X}}}}= \langle Jx, y\rangle _{{{\mathcal {X}}}_1}$. Then ${{\mathcal {X}}}$ is a Kreĭn space. Moreover, it holds

$$\begin{aligned} \langle A x, y\rangle _{{{\mathcal {X}}}}&= \langle JAx, y\rangle _{{{\mathcal {X}}}_1} = \langle x, A^*Jy\rangle _{{{\mathcal {X}}}_1} = \langle x, JAy\rangle _{{{\mathcal {X}}}_1}=\langle x, Ay\rangle _{{{\mathcal {X}}}},\\ \langle B u, x\rangle _{{{\mathcal {X}}}}&= \langle JB u, x\rangle _{{{\mathcal {X}}}_1} = \langle u, B^*Jx\rangle _{{{\mathcal {U}}}} =\langle u, B^*Jx\rangle _{{{\mathcal {U}}}}=\langle u, Cx\rangle _{{{\mathcal {U}}}}. \end{aligned}$$

This implies that A is self-adjoint in the Kreĭn space ${{\mathcal {X}}},$ and the adjoint of $B: {{\mathcal {U}}}\rightarrow {{\mathcal {X}}}$ is C viewed as operator from ${{\mathcal {X}}}$ to ${{\mathcal {U}}}$. Then, A, B, C and their adjoints all are everywhere defined, and therefore bounded also with respect to the topology induced by ${{\mathcal {X}}}$ [16, Chapter VI 2]. Define $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}})$. Then, $\Sigma$ is a self-adjoint realization of $\theta ,$ and the proof is complete. $\square$

In Proposition 3.7, the self-adjoint realization $\Sigma =(T_{\Sigma };{{\mathcal {X}}},{{\mathcal {U}}})$ with the Kreĭn space ${{\mathcal {X}}}$ was constructed from a simple conservative realization $\Sigma _1=(T_{\Sigma _1};{{\mathcal {X}}}_1,{{\mathcal {U}}};\kappa )$. If ${{\mathcal {X}}}$ is a Pontryagin space with the negative index $\kappa ''$, it follows from Proposition 3.1 that the transfer function $\theta \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}})$ belongs also in the class ${\mathbf {N}}_{\kappa '}({{\mathcal {U}}}),$ where $\kappa '\le \kappa ''.$ One might conjecture that this happens for every $\theta \in {\mathbf {S}}_{\kappa }({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa '}({{\mathcal {U}}}).$ However, Theorem 3.8 below shows that this is not true; it happens only when $\theta \in {\mathbf {U}}_{\kappa }({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa '}({{\mathcal {U}}})$. That is, the values of $\theta$ must also be unitary for all but finitely many points on the unit circle ${{\mathbb {T}}}$; see the page 11.

Theorem 3.8

Let $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}})$ be symmetric in a sense $\theta (z)=\theta ^\#(z)$, and let $\Sigma _1=(A,B,C,D;{{\mathcal {X}}}_1,{{\mathcal {U}}};\kappa _1)$ be a simple conservative realization of $\theta$. Construct the Kreĭn space ${{\mathcal {X}}}$ and the self-adjoint realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}})$ by using the method given in the proof of Proposition 3.7. Then $\theta \in {\mathbf {U}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ if and only if ${{\mathcal {X}}}$ is a Pontryagin space with the negative index $\kappa _2.$

Proof

Let $*$ and ${[*]}$ refer, respectively, to the adjoint with respect to the inner product of ${{\mathcal {X}}}_1$ and ${{\mathcal {X}}}$.

$\Leftarrow$: Suppose that ${{\mathcal {X}}}$ is a Pontryagin space with the negative index $\kappa _2.$ Let J be the unitary similarity mapping used in Proposition 3.7 with the properties (3.20). Then, since

$$ \langle A^*x, y\rangle _{{{\mathcal {X}}}} = \langle J A^*x, y\rangle _{{{\mathcal {X}}}_1} =\langle x, A J y\rangle _{{{\mathcal {X}}}_1} = \langle x, J A^* y\rangle _{{{\mathcal {X}}}_1} =\langle x, A^* y\rangle _{{{\mathcal {X}}}} , $$

A and $A^*$ are both self-adjoint operators with respect to the inner product of the Pontryagin space ${{\mathcal {X}}}$, and it follows from [15, Corollary 3.15, pp. 106] that the non-real spectra of A and $A^*$ consist only of finitely many points. Then, $(I-\zeta A )^{-1}$ and $(I-{\overline{\zeta }} A^* )^{-1}$ exist for all but finitely many $\zeta \in {{\mathbb {T}}}.$ Since $\Sigma _1$ is conservative, the system operator T of $\Sigma$ is unitary, and therefore the defect spaces of T and $T^*$ in (3.13), are zero spaces. By using (3.15) from Lemma 3.3, it can be now deduced that

$$ I-\theta ^*(\zeta )\theta (\zeta ) =(1-\zeta {\overline{\zeta }} ) B^* (I-{\overline{\zeta }} A^*)^{-1} (I-{\zeta } A)^{-1} B =0 $$

for all but finitely many $\zeta \in {{\mathbb {T}}},$ which shows that $\theta =\theta ^\# \in {\mathbf {U}}_{\kappa _1}({{\mathcal {U}}}).$ Choose some fundamental decomposition of ${{\mathcal {U}}}$, and consider the Potapov–Ginzburg transformation $\theta _P$ as in (2.3). It can be assumed that $\theta _P$ is holomorphic at the origin, since if not, one only has to consider $\theta _P(\eta (z))$ as in Remark 2.2. Then by Proposition 2.1 and Theorem 2.8, $\theta _P$ is symmetric and the radial limit values of $\theta _P$ are unitary a.e. on ${{\mathbb {T}}}$. It follows then from [12, Theorems 9.4 and 10.2]; see also [31, Theorem 4.4], that all simple passive realizations of $\theta _P$ are conservative and minimal. Then it follows from [2, Theorems 2.1.3 and 4.3.3] that all simple conservative realizations of $\theta$ are minimal. Therefore $\Sigma _1$ is minimal, which implies that $\Sigma$ is also minimal, since the norms of spaces ${{\mathcal {X}}}_1$ and ${{\mathcal {X}}}$ are equivalent. Therefore, $\Sigma$ is a minimal self-adjoint system with a Pontryagin state space, and it follows from Proposition 3.1 that $\theta$ is a generalized Nevanlinna function whose negative index coincides with the index of the maximal negative subspace of the space of the form (3.10). But since $\Sigma$ is minimal, the space (3.10) is dense in ${{\mathcal {X}}}$, and by [16, Theorem 1.4 on p. 185], it contains a maximal uniformly negative subspace of ${{\mathcal {X}}}$. It follows that $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}).$

$\Rightarrow$: Let $\theta \in {\mathbf {U}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}).$ By using the Potapov–Ginzburg transformation similarly as above, it can be deduced that $\Sigma _1$ and $\Sigma$ are both minimal. By using a similar argument as in the proof Proposition 3.1, one deduces that the Nevanlinna kernel of $\theta$ can be represented as

$$\begin{aligned} N_{\theta }(w,z)= & {} C(I-zA)^{-1} (I-{\bar{w}}A)^{-1}B \nonumber \\= & {} B^{[*]}(I-zA^{[*]})^{-1} (I-{\bar{w}}A)^{-1}B. \end{aligned}$$

(3.23)

Then the matrix of the form

$$\begin{aligned} \left( \left\langle N_\theta (w_j,w_i)f_j,f_i \right\rangle _{{{\mathcal {U}}}}\right) _{i,j=1}^{n} = \left( \left\langle (I-\bar{w_j}A)^{-1}B f_j, (I-\bar{w_i}A)^{-1}B f_i \right\rangle _{{{\mathcal {X}}}}\right) _{i,j=1}^{n}, \end{aligned}$$

such that $f_i \in {{\mathcal {U}}}$, $w_i \in \Omega$, $i=1,\ldots ,n$, where $\Omega$ is a sufficiently small symmetric neighbourhood of the origin, is a Gram matrix. Since $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ the kernel $N_\theta$ has $\kappa _2$ negative squares. These facts combined with [2, Lemma 1.1.1’] imply that there exists a finite sequence $\{ (I-\overline{w_i}A)^{-1}Bf_i \}_{i=1}^n \subset {{\mathcal {X}}}$ of vectors such that the linear span of $\{ (I-\overline{w_i}A)^{-1}Bf_i \}_{i=1}^n$ contains a $\kappa _2$-dimensional anti-Hilbert subspace of ${{\mathcal {X}}}.$ Therefore, $\mathrm {ind}_-{{\mathcal {X}}}$ is at least $\kappa _2.$ Suppose that $\mathrm {ind}_-{{\mathcal {X}}}>\kappa _2.$ Then there exists a finite sequence $\{ x_i \}_{i=1}^{\kappa '} \subset {{\mathcal {X}}}$ of linearly independent negative vectors such that $\kappa '>\kappa _2$. Since $\Sigma$ is minimal, $\mathrm {span}\{\mathrm{ran}\,(I-zA)^{-1}B: z \in \Omega \}:={\mathfrak {S}},$ where $\Omega$ is a sufficiently small symmetric neighbourhood of the origin, is dense in ${{\mathcal {X}}}$. Then, each $x_i$ can be approximated as closely as desired by a vector $x_i'$ from ${\mathfrak {S}}$. By choosing a good enough approximation $x_i'$ for each $x_i$, one obtains $\kappa '$ negative linearly independent vectors $x_1',\ldots ,x_{\kappa '}' \in {\mathfrak {S}}$. That is, ${\mathfrak {S}}$ contains $k'$-dimensional anti-Hilbert subspace, and by using a similar argument as in the proof of Proposition 3.1, it can be shown that the kernel $N_{\theta }$ has at least $\kappa '>\kappa _2$ negative squares. This contradicts the assumption that $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$. Therefore, $\mathrm {ind}_-{{\mathcal {X}}}$ must be $\kappa _2$, and the proof is complete. $\square$

A simple conservative realization of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is unique up to unitary similarity, and therefore the results of Theorem 3.8 do not depend on the choice of $\Sigma$ and J. The concrete models for J can be obtained by using the canonical realizations of $\theta .$ If ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, for an ${\mathcal {L}}({{\mathcal {U}}},{{\mathcal {Y}}})$-valued function $\theta$ holomorphic on a neighbourhood $\Omega$ of the origin, the kernel (1.1) has $\kappa$ negative squares if and only if the related ${\mathcal {L}}({{\mathcal {Y}}}\oplus {{\mathcal {U}}})$-valued kernel

$$\begin{aligned} D_\theta (w,z)= \begin{pmatrix} K_\theta (w,z) &{} \dfrac{\theta (z)-\theta ({\overline{w}})}{z-{\overline{w}}} \\ \dfrac{\theta ^\#(z)-\theta ^\#({\overline{w}})}{z-{\overline{w}}} &{} K_{\theta ^\#}(w,z) \end{pmatrix}, \qquad w,z \in \Omega , \end{aligned}$$

(3.24)

has $\kappa$ negative squares; see [2, Theorem 2.5.2]. The Pontryagin space generated by the kernel (3.24), that is, the completion of the space

$$\begin{aligned} \mathrm {span}\left\{ D_\theta (w,z) {\tiny \begin{pmatrix} y \\ u \end{pmatrix}}: w \in \Omega ,\quad y \in {{\mathcal {Y}}}, \quad u \in {{\mathcal {U}}}\right\} , \end{aligned}$$

(3.25)

where $D_\theta (w,z) {\tiny \begin{pmatrix} y \\ u \end{pmatrix}}$ is treated as a function of z, is denoted by ${{\mathcal {D}}}(\theta )$. The function space ${{\mathcal {D}}}(\theta )$ is continuously contained in ${{\mathcal {H}}}(\theta ) \oplus {{\mathcal {H}}}(\theta ^\#)$, where ${{\mathcal {H}}}(\theta )$ is the Pontryagin space generated by the kernel (1.1). The spaces ${{\mathcal {H}}}(\theta )$ and ${{\mathcal {D}}}(\theta )$ can be chosen as state spaces of, respectively, an observable co-isometric realization and a simple conservative realization of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$. For $h \in {{\mathcal {H}}}(\theta )$, $\begin{pmatrix} h \\ k \end{pmatrix}$ $\in$ ${{\mathcal {D}}}(\theta )$ and $u \in {{\mathcal {U}}},$ define respectively

$$\begin{aligned} \left\{ \begin{aligned}&A_1: h(z) \mapsto \frac{h(z)-h(0)}{z}, \quad \; \;&B_1&: u \mapsto \frac{\theta (z)-\theta (0)}{z}u, \\&C_1: h(z) \mapsto h(0), \quad \; \;&D&: u \mapsto \theta (0)u, \end{aligned} \right. \end{aligned}$$

(3.26)

and

$$\begin{aligned} \left\{ \begin{aligned}&A_2: \begin{pmatrix} h(z) \\ k(z) \end{pmatrix} \mapsto \begin{pmatrix} \dfrac{h(z)-h(0)}{z} \\ zk(z)-\theta ^\#(z)h(0) \end{pmatrix},&B_2&: u \mapsto \begin{pmatrix} \dfrac{\theta (z)-\theta (0)}{z}u \\ \left( I_{{{\mathcal {U}}}}-\theta ^\#(z)\theta ^{\#*}(0)\right) u \end{pmatrix} , \\&C_2: \begin{pmatrix} h \\ k \end{pmatrix} \mapsto h(0),&D&: u \mapsto \theta (0)u. \end{aligned} \right. \nonumber \\ \end{aligned}$$

(3.27)

Then $\Sigma _1 = (A_1,B_1,C_1,D, {{\mathcal {H}}}(\theta ),{{\mathcal {U}}},{{\mathcal {Y}}}, \kappa )$ and $\Sigma _2 = (A_2,B_2,C_2,D, {{\mathcal {D}}}(\theta ),{{\mathcal {U}}},{{\mathcal {Y}}}, \kappa )$ are, respectively, an observable co-isometric realization of $\theta ,$ and a simple conservative realization of $\theta ;$ for the proof, see [2, Theorems 2.2.1 and 2.3.1]. These systems are called, respectively, the canonical co-isometric realization and the canonical unitary (or conservative) realization of $\theta$. Any observable co-isometric realization of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is unitarily similar with the system $\Sigma _1$, and any simple conservative realization is unitarily similar with $\Sigma _2.$

Suppose next the simple conservative realization of the symmetric function $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}})$ in Theorem 3.8 is chosen to be the canonical unitary realization. Then it can be derived from [3, Theorem 3.6] that the self-adjoint unitary similarity J is of the form $J={\widehat{J}}{\upharpoonright }_{{{\mathcal {D}}}(\theta )},$ where

$$\begin{aligned} {\widehat{J}}=\begin{pmatrix} 0 &{}\quad I \\ I &{}\quad 0 \end{pmatrix}: \begin{pmatrix} {{\mathcal {H}}}(\theta ) \\ {{\mathcal {H}}}(\theta ) \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {H}}}(\theta ) \\ {{\mathcal {H}}}(\theta ) \end{pmatrix}. \end{aligned}$$

(3.28)

In addition, if also $\theta \in {\mathbf {U}}_{\kappa _1}({{\mathcal {U}}}) ,$ it has been shown in the proof of Theorem 3.8 that all co-isometric observable or isometric controllable realizations of $\theta$ are minimal conservative. Therefore it can be assumed that $\Sigma _1$ in Theorem 3.8 is the canonical co-isometric realization. In that case, it can be derived from [3, Corollary 3.7] that J is the closure of a linear relation $\Lambda$ defined by

$$\begin{aligned} \Lambda \left( \sum _i K_{\theta }(w_i, z)f_i \right) = \sum _i N_{\theta }(w_i, z)f_i, \end{aligned}$$

where $K_{\theta }(w_i,z)f_i \in {{\mathcal {H}}}(\theta )$ and $N_{\theta }(w_i,z)f_i \in {{\mathcal {H}}}(\theta )$ are treated as a function of z.

Proposition 3.9

Let ${{\mathcal {U}}}$ be Pontryagin space and let $\theta$ be a symmetric ${{\mathcal {L}}}({{\mathcal {U}}})$-valued function holomorphic at the origin and meromorphic on ${{\mathbb {D}}}\cup {{\mathbb {C}}}\setminus {{\mathbb {R}}}$. Then the following statements are equivalent:

• (i) $\theta$ has a minimal conservative self-adjoint realization $\Sigma$ such that the state space of $\Sigma$ is a Hilbert space;

• (ii) $\theta \in {\mathbf {U}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$;

• (iii) $K_\theta (w,z)=N_\theta (w,z)$ and the kernels are nonnegative.

Proof

(i) $\Rightarrow$ (ii). Denote $\Sigma =(A,B,B^*,D;{{\mathcal {X}}},{{\mathcal {U}}};0)$. Since $\Sigma$ is a minimal conservative self-adjoint realization of $\theta$ such that ${{\mathcal {X}}}$ is a Hilbert space, it follows from Proposition 3.4 that $\theta \in {\mathbf {S}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$. Moreover, the main operator A is self-adjoint operator in the Hilbert space ${{\mathcal {X}}}$, and a similar argument as used in the proof of Theorem 3.8 can be used to show that the values of $\theta$ are unitary for every $\zeta \in {{\mathbb {D}}}\setminus \{-1,1\}$. Therefore $\theta \in {\mathbf {U}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$.

(ii) $\Rightarrow$ (iii). By definition of ${\mathbf {U}}({{\mathcal {U}}})$ and ${\mathbf {N}}({{\mathcal {U}}})$, the kernels $K_{\theta }$ and $N_{\theta }$ are nonnegative. If ${{\mathcal {U}}}$ is a Hilbert space, the other claim now follows by combining [4, Theorem 5.1] and [5, Proposition 3.1]. Therefore assume $\mathrm {ind}_- {{\mathcal {U}}}>0$. Fix some fundamental decomposition of ${{\mathcal {U}}}$, and consider the Potapov–Ginzburg transformation $\theta _P$ of $\theta$, defined by (2.3). Since $\theta$ is symmetric, the functions $\Phi$ and $\Psi$ defined by (2.5) coincide. It can be assumed that $\theta _{22}^{-1}(0)$ exists; otherwise, consider $\theta _P(\eta (z))$ as in Remark 2.2. Then $\theta _P \in {\mathbf {U}}({{\mathcal {U}}}') \cap {\mathbf {N}}({{\mathcal {U}}}')$ by Proposition 2.1(vii), and since ${{\mathcal {U}}}'$ is a Hilbert space, it now holds $K_{\theta _P}(w,z)=N_{\theta _P}(w,z)$. It follows then from (2.6) and (2.8) that also $K_{\theta }(w,z)=N_{\theta }(w,z)$.

(iii) $\Rightarrow$ (i) Since $K_{\theta }(w,z)$ is nonnegative, $\theta \in {\mathbf {S}}({{\mathcal {U}}})$, and the canonical unitary realization $\Sigma _2$ defined by the operators in (3.27) is simple conservative and the state space is a Hilbert space. Since $\theta =\theta ^\#$ and $K_{\theta }(w,z)=N_{\theta }(w,z)$, the kernel $D_{\theta }(w,z)$ in (3.24) reduces to

$$\begin{aligned} \begin{pmatrix} K_{\theta }(w,z) &{}\quad K_{\theta }(w,z) \\ K_{\theta }(w,z) &{}\quad K_{\theta }(w,z) \end{pmatrix} . \end{aligned}$$

Therefore, all the functions in the space (3.25) are of the form

$$ \begin{pmatrix} h_0(z) \\ h_0(z) \end{pmatrix}= \begin{pmatrix} {\sum }_{j=1}^{n}\alpha _j K_{\theta }(w_j,z)u_j \\ {\sum }_{j=1}^{n}\alpha _j K_{\theta }(w_j,z)u_j \end{pmatrix}, $$

such that $\alpha _j \in {{\mathbb {C}}}$, $u_j \in {{\mathcal {U}}}$ and $w_j \in \Omega$, where $\Omega$ is the domain of holomorphy of $\theta$. It follows that all the ${{\mathcal {L}}}({{\mathcal {U}}}\oplus {{\mathcal {U}}})$-valued functions in the completion ${{\mathcal {D}}}(\theta )$ of (3.25) are of the form ${\tiny \begin{pmatrix} h(z) \\ h(z) \end{pmatrix}}$. Then, the self-adjoint unitary similarity mapping $J={\widehat{J}}{\upharpoonright }_{{{\mathcal {D}}}(\theta )}$ between $\Sigma _2$ and $\Sigma _2^*$, where ${\widehat{J}}$ where is defined by (3.28), is identity. That is, $\Sigma _2$ is self-adjoint, and since it is simple, it is now minimal, and the proof is complete. $\square$

In Theorem 3.8, the condition that the space ${{\mathcal {X}}}$ induced by the mapping J is a Pontryagin space with the negative index $\kappa$ is equivalent to $\mathrm {ind}_- J=\kappa$, where $\mathrm {ind}_- J$ is with respect to the state space ${{\mathcal {X}}}_1$. By considering minimal passive realizations instead of simple conservative realizations, one can obtain a similar type of characterization when $\theta \in {\mathbf {S}}_{\kappa _1} ({{\mathcal {U}}})\cap {\mathbf {N}}_{\kappa _2} ({{\mathcal {U}}}).$

Denote $E_{{{\mathcal {X}}}}(x)=\langle x,x \rangle _{{\mathcal {X}}}$ for the vector x in an inner product space ${{\mathcal {X}}}.$ For $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative index, the realization $\Sigma$ of $\theta$ is called $\kappa$-admissible, if the negative index of the state space of $\Sigma$ is $\kappa$. A $\kappa$-admissible passive realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}},{{\mathcal {Y}}};\kappa )$ of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is called optimal if for any $\kappa$-admissible passive realization $\Sigma _0=(A_0,B_0,C_0,D;{{\mathcal {X}}}_0,{{\mathcal {U}}},{{\mathcal {Y}}};\kappa )$ of $\theta$ it holds

$$ E_{{{\mathcal {X}}}}\left( \sum _{n=0}^{N}{A}^n Bu_n \right) \le E_{{{\mathcal {X}}}_0}\left( \sum _{n=0}^{N}{A_0}^n B_0u_n \right) , $$

for any $N\in {{\mathbb {N}}}_0$ and $\{ u_n\}_{n=0}^N \subset {{\mathcal {U}}}.$ Moreover, an observable passive realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}},{{\mathcal {Y}}};\kappa )$ of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is called $^*$-optimal if for any observable $\kappa$-admissible passive realization $\Sigma _0=(A_0,B_0,C_0,D;{{\mathcal {X}}}_0,{{\mathcal {U}}},{{\mathcal {Y}}};\kappa )$ of $\theta$ it holds

$$ E_{{{\mathcal {X}}}}\left( \sum _{n=0}^{N}{A}^n Bu_n \right) \ge E_{{{\mathcal {X}}}_0}\left( \sum _{n=0}^{N}{A_0}^n B_0u_n \right) , $$

for any $N\in {{\mathbb {N}}}_0$ and $\{ u_n\}_{n=0}^N \subset {{\mathcal {U}}}.$ The requirement of the observability in the definition of $^*$-optimality is essential to avoid trivialities, see [9, Proposition 3.5 and example on page 144]. Moreover, the requirement that the considered realizations are $\kappa$-admissible is also essential, see [32, Example 3.1].

Let $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}})$ be symmetric. It follows from [34, Theorem 5.3] and [32, Theorem 3.5] that there exists a $^*$-optimal minimal passive realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa )$ of $\theta .$ Since $\theta =\theta ^\#,$ it follows from [34, Theorem 5.2] and [32, Theorem 3.5] that the dual system $\Sigma ^*=(A^*,C^*,B^*,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa )$ of $\Sigma$ is optimal minimal passive. Define

$$\begin{aligned} Z \left( \sum _{n=0}^{N}{A}^n B u_n\right) = \sum _{n=0}^{N}{A^*}^n C^*u_n \end{aligned}$$

for the vectors of the form $\sum _{n=0}^{N}{A}^n B u_n.$ Since $\Sigma$ and $\Sigma ^*$ are minimal and $\Sigma ^*$ is optimal, the linear relation Z is densely defined, contractive, and it has a dense range in ${{\mathcal {X}}}$. It follows from [2, Theorem 1.4.2] that the closure of Z, which is still denoted as Z, is an everywhere defined bounded contractive linear operator in ${{\mathcal {X}}}$. By proceeding as in the proof of [31, Theorem 2.5], one deduces that Z is injective, it has a dense range, and it holds

$$\begin{aligned} ZA=A^*Z, \quad C= B^*Z\quad \text {and} \quad ZB=C^*. \end{aligned}$$

(3.29)

That is, Z is an everywhere defined weak similarity. Moreover, it holds

$$ Z^* \left( \sum _{n=0}^{N}{A}^n Bu_n\right) = \sum _{n=0}^{N}{A^*}^nZ^* Bu_n = \sum _{n=0}^{N}{A^*}^nC^*u_n = Z \left( \sum _{n=0}^{M}{A}^n Bu_n\right) . $$

Since $\Sigma$ is minimal, it follows now that $Z: {{\mathcal {X}}}\rightarrow {{\mathcal {X}}}$ is self-adjoint. That is, Z is bounded injective self-adjoint operator. Moreover, an optimal ($^*$-optimal) minimal passive realization of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is unique up to unitary similarity [32, Theorem 3.5]. Therefore, the mapping Z is unique up to unitary equivalence, and the properties of Z in Theorem 3.10 below do not depend of the choice of a $^*$-optimal minimal passive realization of $\theta$.

Theorem 3.10

Let $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}})$ be symmetric and let $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa _1)$ be a $^*$-optimal minimal passive realization of $\theta .$ Then $\theta \in {\mathbf {S}}_{\kappa _1}({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$ if and only if $\mathrm {ind}_- Z =\kappa _2,$ where Z is the self-adjoint contraction in ${{\mathcal {X}}}$ with the properties (3.29).

Proof

Since Z is self-adjoint, bounded and injective with a dense range, it has a Bognár–Krámli factorization of the form $Z=V_ZV_Z^*,$ where $V_Z: {{\mathcal {D}}}_Z \rightarrow {{\mathcal {X}}},$ is bounded, ${{\mathcal {D}}}_Z$ is a Kreĭn space with $\mathrm {ind}_-{{\mathcal {D}}}_Z=\mathrm {ind}_-Z,$ and $V_Z$ and $V_Z^*$ have zero kernels and dense ranges; see [23, Theorem 2.1]. By using the realization $\Sigma$, the properties (3.29) of the self-adjoint mapping Z imply that $Z(I-zA)^{-1}=(I-zA^*)^{-1}Z$ whenever $(I-zA)^{-1}$ and $(I-zA^*)$ exist. Hence the Nevanlinna kernel of $\theta$ can be represented as

$$\begin{aligned} N_{\theta }(w,z)&= C(I-zA)^{-1} (I-{\bar{w}}A)^{-1}B = B^*Z(I-zA)^{-1} (I-{\bar{w}}A)^{-1}B\\&=B^*(I-zA^*)^{-1} Z (I-{\bar{w}}A)^{-1}B=B^*(I-zA^*)^{-1} V_ZV_Z^* (I-{\bar{w}}A)^{-1}B ; \end{aligned}$$

cf. (3.23) on page 20. This yields the identity

$$\begin{aligned}&\left( \left\langle N_\theta (w_j,w_i)f_j,f_i \right\rangle _{{{\mathcal {U}}}} \right) _{i,j=1}^{n}= \left( \left\langle Z(I-\bar{w_j}A)^{-1}B f_j, (I-\bar{w_i}A)^{-1}B f_i \right\rangle _{{{\mathcal {X}}}}\right) _{i,j=1}^{n} \nonumber \\&\quad = \left( \left\langle V_Z^*(I-\bar{w_j}A)^{-1}B f_j, V_Z^* (I-\bar{w_i}A)^{-1}B f_i \right\rangle _{{{\mathcal {D}}}_Z}\right) _{i,j=1}^{n} \end{aligned}$$

(3.30)

where $n \in {{\mathbb {N}}},$ $\{f_j\}_{j=1}^n \subset {{\mathcal {U}}},$ and $\{w_j\}_{j=1}^n \subset \Omega$ for some sufficiently small symmetric neighbourhood $\Omega$ of the origin, for the kernel $N_\theta$. Moreover, since $\Sigma$ is minimal, the space $\mathrm {span} \{\mathrm{ran}\,(I-zA)^{-1}B: z \in \Omega \} := {\mathfrak {S}}$ is dense in ${{\mathcal {X}}}.$ Let $y \in {{\mathcal {D}}}_Z$ such that $\langle y, V_Z^*x \rangle _{{{\mathcal {D}}}_Z} = 0$ for all $x \in {\mathfrak {S}} .$ Then, $\langle y, V_Z^*x \rangle _{{{\mathcal {D}}}_Z} = \langle V_Zy, x \rangle _{{{\mathcal {D}}}_Z}=0,$ which implies $V_Zy=0$ and then $y=0,$ since ${\mathfrak {S}}$ is a dense set and $V_Z$ has only the trivial kernel. That is, $V_Z^*{\mathfrak {S}}$ is a dense set in ${{\mathcal {D}}}_Z.$

$\Leftarrow$: Suppose that $\mathrm {ind}_- Z =\kappa _2.$ Then $\mathrm {ind}_- {{\mathcal {D}}}_Z =\kappa _2$, and it follows from the identity (3.30) that $N_\theta$ has at most $\kappa _2$ negative squares. Since $V_Z^* {\mathfrak {S}}$ is a dense set in ${{\mathcal {D}}}_Z,$ [2, Lemma 1.1.1] shows that there exists a finite sequence $\{ V_Z^*(I-\overline{w_i'}A)^{-1}Bf_i' \}_{i=1}^n \subset {{\mathcal {D}}}_Z$ of vectors such that the linear span of $\{ V_Z^*(I-\overline{w_i'}A)^{-1}Bf_i' \}_{i=1}^n$ contains a $\kappa _2$-dimensional anti-Hilbert subspace of ${{\mathcal {D}}}_Z.$ Then by the identity (3.30) the kernel $N_\theta$ has at least $\kappa _2$ negative squares. It has been showed that $N_\theta$ has exactly $\kappa _2$ negative squares, and therefore $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}).$

$\Rightarrow$: Assume $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$. Then the identity (3.30) shows that $\mathrm {ind}_- Z$ is at least $\kappa _2$. Since $V_Z^* {\mathfrak {S}}$ is a dense set, a similar argument as in the proof of Theorem 3.8 can be used to conclude that $\mathrm {ind}_- Z=\kappa _2.$ $\square$

4 Dilations and subclasses of generalized Schur–Nevanlinna functions

A dilation of the function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ is any function $\Theta$ holomorphic at the origin, which is of the form

$$\begin{aligned} \Theta (z)= \begin{pmatrix} \theta (z) &{}\quad \theta _2(z) \\ \theta _3(z) &{}\quad \theta _4(z) \end{pmatrix} \end{aligned}$$

(4.1)

and has values in ${{\mathcal {L}}}({{\mathcal {U}}}\oplus {{\mathcal {U}}}', {{\mathcal {Y}}}\oplus {{\mathcal {Y}}}')$, where ${{\mathcal {U}}}'$ and ${{\mathcal {Y}}}'$ are Hilbert spaces. In the case where ${{\mathcal {U}}}'$ or ${{\mathcal {Y}}}'$ is a zero space, the corresponding row or column in (4.1) will be left out. This definition is a straightforward generalization of the definition of a dilation of $\theta \in {{{\mathbf {S}}}( {{\mathcal {U}}},{{\mathcal {Y}}})}$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, as represented in [14]. A function $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ has a generalized bi-${\mathcal {J}}$-inner dilation, if there exists a dilation $\Theta$ of $\theta$ such that $\Theta \in {\mathbf {U}}_\kappa ({{\mathcal {U}}}\oplus {{\mathcal {U}}}', {{\mathcal {Y}}}\oplus {{\mathcal {Y}}}')$. The case when ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces and $\kappa =0$ corresponds to the ordinary bi-inner dilation. It is known from [14] that $\theta \in {{{\mathbf {S}}}( {{\mathcal {U}}},{{\mathcal {Y}}})}$, where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, has a bi-inner dilation if and only if there exists $\varphi \in {\mathbf {S}}({{\mathcal {U}}},{{\mathcal {Y}}}')$ and $\psi \in {\mathbf {S}}({{\mathcal {Y}}},{{\mathcal {U}}}')$, where such that $I_{{{\mathcal {U}}}}-\theta ^* (\zeta ) \theta (z)=\varphi ^*(\zeta ) \varphi (\zeta )$ and $I_{{{\mathcal {Y}}}}- \theta (\zeta ) \theta ^*(\zeta )= \psi (\zeta ) \psi ^*(\zeta )$ for a.e. $\zeta \in {{\mathbb {T}}}.$ Moreover, every $\theta \in {\mathbf {S}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$ has a bi-inner dilation $\Theta$, which can be chosen such that $\Theta \in {\mathbf {U}}\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right) \cap {\mathbf {N}}\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right)$ [5, pp. 4)].

As mentioned on page 12, any two minimal passive realizations of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ are only weakly similar in general. However, for some generalized Schur functions, any two minimal passive $\kappa$-admissible realizations are unitary similar. This happens, for an example, if the boundary values of $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ are (co-)isometric a.e. on ${{\mathbb {T}}}$, or when the generalized right or left defect function is identically zero; see [32, pp. 25]. Let ${\mathbf {S}}_\kappa ^{U}({{\mathcal {U}}},{{\mathcal {Y}}})$ be the class that consist of those functions $\theta \in {{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})}$ with the property that any two minimal passive $\kappa$-admissible realizations of $\theta$ are unitarily similar. A criterion for $\theta$ to be in ${\mathbf {S}}^{U}({{\mathcal {U}}},{{\mathcal {Y}}}),$ where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Hilbert spaces, were obtained by Arov and Nudelman in [10, 11]. This result was generalized for the class ${{{\mathbf {S}}}_{\kappa }( {{\mathcal {U}}},{{\mathcal {Y}}})},$ where ${{\mathcal {U}}}$ and ${{\mathcal {Y}}}$ are Pontryagin spaces with the same negative indices, by the author in [32, pp. 25].

Define ${\mathcal {R}}{\mathcal {S}}_{\kappa }({{\mathcal {U}}})$ to be the class of those functions in ${\mathbf {S}}_{\kappa }({{\mathcal {U}}}) \cap {\mathbf {N}}_{\kappa }({{\mathcal {U}}})$ which have $\kappa$-admissible passive self-adjoint realizations. If ${{\mathcal {U}}}$ is a Hilbert space and $\kappa =0,$ the class ${\mathcal {R}}{\mathcal {S}}_{0}({{\mathcal {U}}})$ coincides with ${\mathbf {S}}({{\mathcal {U}}}) \cap {\mathbf {N}}({{\mathcal {U}}})$ [5].

Theorem 4.1

Every $\theta \in {\mathbf {S}}^U_{\kappa _1}({{\mathcal {U}}})\cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$, where ${{\mathcal {U}}}$ is a Pontryagin space, has a generalized bi-${\mathcal {J}}$-inner dilation $\Theta \in {\mathbf {U}}_{\kappa _1}\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right)$, and every $\theta \in {\mathcal {R}}{\mathcal {S}}_{\kappa }({{\mathcal {U}}})$ has a generalized bi-${\mathcal {J}}$-inner dilation $\Theta \in {\mathbf {U}}_{\kappa }\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right) \cap {\mathbf {N}}_{\kappa }\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right)$.

Proof

Suppose $\theta \in {\mathbf {S}}^U_{\kappa _1}({{\mathcal {U}}})\cap {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}})$. Consider a $^*$-optimal minimal realization $\Sigma =(A,B,C,D;{{\mathcal {X}}},{{\mathcal {U}}};\kappa _1)$ of $\theta$ and the self-adjoint contraction Z with the properties (3.29). Since $\theta \in {\mathbf {S}}^U_{\kappa _1}$ and $\Sigma$ and $\Sigma ^*$ are both minimal passive $\kappa _1$-admissible realizations of $\theta ,$ the system $\Sigma$ is unitarily similar with $\Sigma ^*.$ It follows then easily that Z is actually unitary in ${{\mathcal {X}}}$, and therefore $Z=Z^*=Z^{-1}$ i.e. Then, since Z in boundedly invertible, the space ${{\mathcal {X}}}_Z,$ which coincide with ${{\mathcal {X}}}$ as a vector space but which is endowed with the inner product $\langle x,y \rangle _{{{\mathcal {X}}}_Z}=\langle Z x,y \rangle _{{{\mathcal {X}}}},$ is a Kreĭn space [15, 6.13 on page 40]. The same arguments as used in the proof of Proposition 3.7 shows that A and $A^*$ are self-adjoint with respect to the inner product of ${{\mathcal {X}}}_Z,$ and $\Sigma _Z:=(A,B,C,D; {{\mathcal {X}}}_Z,{{\mathcal {U}}})$ is a self-adjoint realization of $\theta .$ Since $\Sigma$ is minimal and $\theta \in {\mathbf {N}}_{\kappa _2}({{\mathcal {U}}}),$ similar arguments as in the proof of Theorem 3.8 can be used to conlude that $\Sigma _Z$ is minimal and ${{\mathcal {X}}}_Z$ is a Pontryagin space with the negative index $\kappa _2.$ Then, the non-real spectra of A and $A^*$ consist only of finitely many points, i,e., $(I-\zeta A)^{-1}$ and $(I-\zeta A^*)^{-1}$ are defined for all but finitely many $\zeta \in {{\mathbb {T}}}.$

Denote the system operator of $\Sigma$ as T. By Theorem 3.2, the system operator T has a Julia operator of the form

$$\begin{aligned} \begin{pmatrix} T &{}\quad D_{T^*} \\ D^*_{T} &{}\quad -L^* \end{pmatrix}: \begin{pmatrix} {{\mathcal {X}}}\oplus {{\mathcal {U}}}\\ {{\mathfrak {D}}}_{T^*} \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\oplus {{\mathcal {U}}}\\ {{\mathfrak {D}}}_{T} \end{pmatrix}, \end{aligned}$$

(4.2)

where ${{\mathfrak {D}}}_{T^*}$ and ${{\mathfrak {D}}}_{T}$ are Hilbert spaces, $D_{T^*}D_{T^*}^*=I-TT^*$ and $D_{T}D_{T}^*=I-T^*T$ such that $D_{T}$ and $D_{T^*}$ have zero kernels. Then, one can form the Julia embedding ${\widetilde{\Sigma }}$ of the system $\Sigma ;$ recall the embeddings introduced after Lemma 3.3. That is, the corresponding system operator ${T}_{{\widetilde{\Sigma }}}$ of the embedding ${\widetilde{\Sigma }}$ is a Julia operator of T,  and it is of the form

$$\begin{aligned} T_{{\widetilde{\Sigma }}} = \begin{pmatrix} A&\begin{pmatrix} B &{} D_{T_{,1}^*} \end{pmatrix} \\ \begin{pmatrix} C \\ D_{T_{,1}}^* \end{pmatrix} &{} \begin{pmatrix} D &{} D_{T_{,2}^*} \\ D_{T_{,2}}^* &{} -L^* \end{pmatrix} \end{pmatrix} : \begin{pmatrix} {{\mathcal {X}}}\\ \begin{pmatrix} {{\mathcal {U}}}\\ {{\mathfrak {D}}}_{T^*}\end{pmatrix} \end{pmatrix} \rightarrow \begin{pmatrix} {{\mathcal {X}}}\\ \begin{pmatrix} {{\mathcal {U}}}\\ {{\mathfrak {D}}}_{T}\end{pmatrix} \end{pmatrix}, \end{aligned}$$

where $D_{T^*} = {\tiny \begin{pmatrix} D_{T_{,1}^*}\\ D_{T_{,2}^*} \end{pmatrix}}$ and $D_{T} = {\tiny \begin{pmatrix} D_{T_{,1}}\\ D_{T_{,2}} \end{pmatrix}}$. The system ${\widetilde{\Sigma }}$ is conservative, and since $\Sigma$ is minimal, the inclusions $\mathrm {ran}\, B \subset \mathrm {ran} \begin{pmatrix} B&D_{T_{,1}^*} \end{pmatrix}$ and $\mathrm {ran}\, C^* \subset \mathrm {ran} \begin{pmatrix} C^*&D_{T_{,1}} \end{pmatrix}$ imply that ${\widetilde{\Sigma }}$ is also minimal. The transfer function of the Julia embedding is

$$\begin{aligned} \Theta (z)= \begin{pmatrix} D + zC(I-zA)^{-1}B &{} D_{T_{,2}^*}+zC(I-zA)^{-1} D_{T^*_{,1}} \\ D_{T_{,2}}^* +zD_{T_{,1}}^*(I-zA)^{-1}B &{}-L^* + zD_{T_{,1}}^*(I-zA)^{-1} D_{T^*_{,1}} \end{pmatrix}= \begin{pmatrix} \theta (z) &{} \psi (z)\\ \varphi (z) &{} \chi (z) \end{pmatrix}. \end{aligned}$$

The funtion $\Theta$ is a dilation of $\theta ,$ and since it is the transfer function of the minimal conservative system, it is a generalized Schur function with the index $\kappa _1$ [2, Theorem 2.1.2]. Since ${\widetilde{\Sigma }}$ is conservative, it follows from Lemma 3.3 that

$$\begin{aligned} \begin{aligned} I-\Theta (z)\Theta ^*(w)&= (1-z{\overline{w}}) \begin{pmatrix} C \\ D_{T_{,1}}^* \end{pmatrix} (I-z A)^{-1} (I-{\overline{w}} A^*)^{-1} \begin{pmatrix} C^*&D_{T_{,1}} \end{pmatrix} \\ I-\Theta ^*(w)\Theta (z)&=(1-z{\overline{w}}) \begin{pmatrix} B^* \\ D_{T_{,1}^*}^* \end{pmatrix} (I-{\overline{w}} A^*)^{-1} (I-zA)^{-1} \begin{pmatrix} B&D_{T_{,1}^*} \end{pmatrix}, \end{aligned} \end{aligned}$$

and therefore that $I-\Theta (\zeta )\Theta ^*(\zeta )=0$ and $I-\Theta ^*(\zeta )\Theta (\zeta )=0$ for all but finitely many $\zeta \in {{\mathbb {T}}}.$ That is, the radial limit values of $\Theta$ are unitary for all but finitely many $\zeta \in {{\mathbb {T}}}$, and therefore $\Theta$ is a unitary dilation of $\theta .$

Assume then that $\theta \in {\mathcal {R}}{\mathcal {S}}_\kappa ({{\mathcal {U}}})$, and let ${\widehat{\Sigma }}=({\widehat{A}},{\widehat{B}},{\widehat{C}},D; {\widehat{{{\mathcal {X}}}}}, {{\mathcal {U}}}; \kappa )$ be a $\kappa$-admissible passive self-adjoint realization of $\theta .$ Since ${\widehat{\Sigma }}$ is a $\kappa$-admissible and passive, the space ${\widehat{{{\mathcal {X}}}}}^s$ is a regular subspace with the negative index $\kappa ,$ and $({\widehat{{{\mathcal {X}}}}}^s)^\perp$ is a Hilbert space [32, Proposition 2.7]. This implies that the system operator $T_{{\widehat{\Sigma }}}$ can be represented as in (3.19). It then easily follows that the restriction $\Sigma =(A,B,C,D; {\widehat{{{\mathcal {X}}}}}^s, {{\mathcal {U}}}; \kappa )$ of ${\widehat{\Sigma }}$ to the simple subspace ${{\mathcal {X}}}^s$ is a minimal passive self-adjoint $\kappa$-admissible realization of $\theta .$ Denote the system operator of $\Sigma$ as T. By Theorem 3.2 there exists a Julia operator $U_T$ of T of the form (4.2) where ${{\mathfrak {D}}}_{T^*}$ and ${{\mathfrak {D}}}_{T}$ are Hilbert spaces. Since T is self-adjoint, it follows from [24, Theorem 5 and pp. 88] that $U_T$ can be chosen such that ${{\mathfrak {D}}}_{T^*}= {{\mathfrak {D}}}_{T}:={{\mathcal {U}}}'$ and $U_T\in {{\mathcal {L}}}\left( {{\mathcal {X}}}\oplus {{\mathcal {U}}}\oplus {{\mathcal {U}}}' \right)$ is self-adjoint; cf. (3.13). Now construct a Julia embedding ${\widetilde{\Sigma }}$ of $\Sigma$ similarly as above, by using $U_T$, and denote the transfer function of ${\widetilde{\Sigma }}$ as $\Theta .$ Then $\Theta$ is a dilation of $\theta$ and ${\widetilde{\Sigma }}$ is minimal self-adjoint conservative. Therefore $\Theta \in {\mathbf {S}}_{\kappa } \in \left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right)$, and by Proposition  3.1 also $\Theta \in {\mathbf {N}}_{\kappa }\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right)$. Since $A=A^*$ is a self-adjoint operator in a Pontryagin space with the negative index $\kappa ,$ a similar argument as above shows that the values of $\Theta$ are unitary for all but finitely many $\zeta \in {{\mathbb {T}}}.$ Therefore also $\Theta \in {\mathbf {U}}_{\kappa }\left( {{\mathcal {U}}}\oplus {{\mathcal {U}}}'\right) ,$ and the proof is complete. $\square$