Unitary boundary pairs for isometric operators in Pontryagin spaces and generalized coresolvents

An isometric operator V in a Pontryagin space H is called standard, if its domain and the range are nondegenerate subspaces in H. A description of coresolvents for standard isometric operators is known and basic underlying concepts that appear in the literature are unitary colligations and characteristic functions. In the present paper generalized coresolvents of non-standard Pontryagin space isometric operators are described. The methods used in this paper rely on a new general notion of boundary pairs introduced for isometric operators in a Pontryagin space setting. Even in the Hilbert space case this notion generalizes the earlier concept of boundary triples for isometric operators and offers an alternative approach to study operator valued Schur functions without any additional invertibility requirements appearing in the ordinary boundary triple approach.


Introduction
Extension theory for standard symmetric and isometric operators in Pontryagin spaces was first developed by I.S. Iokhvidov and M.G. Kreȋn in [30], generalized resolvents of such operators were described by M.G. Kreȋn and H. Langer in [37,38,39]. Following [28] we will use the notion standard for an isometric operator V in a Pontryagin space H, if its domain dom V and the range ran V are nondegenerate subspaces in H. In this case every unitary extension of V can be obtained in pretty much the same way as in the case of Hilbert space isometric operator. Similarly, the extension theory and the theory of generalized coresolvents of standard isometric operators in Kreȋn spaces was built by A. Dijksma, H. Langer and H. de Snoo in [28]. For a nonstandard isometric operator in a Pontryagin space, description of its regular (resp. nonregular) generalized coresolvents in Pontryagin spaces without growth (resp. with growth) of negative index was given by P. Sorjonen [46] (resp. by O. Nitz [43], [44]). However, the proof in [44] is not so convincing, as it becomes quite complicated and contains some gaps.
Another approach to the extension theory of symmetric operators in Hilbert spaces is based on the concept of abstract boundary value introduced by J. Calkin [13] and later formalized in the notion of boundary value space in [35], [33] (or ordinary boundary triple in [26]). In [25] with each boundary triple there was associated an analytic objectabstract Weyl function which allows to carry out spectral analysis of extensions of symmetric operators. In the case of a Hilbert space isometric operator (and more generally for a dual pair of operators) the notions of a boundary triple and a corresponding Weyl function were introduced in [41,42]. These notions, when generalized to the indefinite case in [5], proved to be an adequate language in the extension theory of nonstandard isometric operator V in a Pontryagin space, since they allowed to give full description of generalized coresolvents of V . However, the method proposed in [5] is restricted to the case of regular generalized coresolvents, which have minimal realizations in Pontryagin spaces H with the same negative index as H, and does not work for generalized coresolvents of V which have minimal realizations in Pontryagin spaces H with bigger negative indices.
This difficulty can be prevented by using an appropriate notion of boundary pairs, which extend the concept of ordinary boundary triples. In the case of symmetric operators in Hilbert spaces an extension of ordinary boundary triples, a so-called generalized boundary triple, was introduced and studied in [26]. This notion was further generalized in [19] to the notion of a unitary boundary pair (called therein as a boundary relation), which can be applied to study generalized resolvents of symmetric operators [20,21] and various general classes of boundary value problems for ordinary and partial differential operators, see [21,22,23,24]. In particular, in [19] it was shown that every Nevanlinna pair (or Nevanlinna family of holomorphic relations) can be realized as the Weyl family of some unitary boundary pair, and in [18,20] this notion was used to get a new proof of M.G. Kreȋn formula for generalized resolvents of symmetric operators via the coupling method developed therein. In [8] the notion of unitary boundary pair was introduced for symmetric operators in Pontryagin spaces and it was shown that every generalized Nevanlinna pair, allowing a finite negative index for the associated Nevanlinna kernel, can be realized as the Weyl family of such a unitary boundary pair.
In this paper a new notion of a unitary boundary pair with an associated Weyl function is introduced and studied in the setting of isometric operators in Pontryagin spaces. In particular, it is shown in Section 3 how a certain subclass of unitary boundary pairs is connected to unitary colligations (see [1]) and, moreover, that the Weyl functions associated with that subclass of unitary boundary pairs actually coincide with characteristic functions of the corresponding unitary colligations; see Theorems 3.10, 3.11. Furthermore, using some transformations results, being motivated by [1], it is also shown that every operator valued generalized Schur function (not necessarily holomorphic at the origin) can still be realized as the Weyl function of some unitary boundary pair for an isometric operator V in a Pontryagin space; see Theorems 3.15, 3.17. These two theorems show that the present notion of a unitary boundary pair for isometric operators in a Pontryagin (as well as in the classical Hilbert) space setting is a natural object to realize and study generalized (or standard) Schur functions as their Weyl functions. In particular, these new notions complement and extend the approach, which relies on characteristic functions of unitary colligations being associated with the special case stated in Theorem 3.11.
After these characteristic results on unitary boundary pairs and their Weyl function for isometric operators we study in Section 4 some spectral properties of proper extensions of V and find a formula for their canonical coresolvents, see Theorem 4.2, and then with these preparations prove an analog of M.G. Kreȋn formula for the generalized coresolvents of the isometric operator V . This latter problem is solved via the coupling method, where we consider a coupled unitary boundary pair as a direct sum of an ordinary boundary triple and a unitary boundary pair and then derive the formula for generalized coresolvents from the formula for canonical coresolvents associated with the coupled boundary pair.

Preliminaries
2.1. Indefinite inner product spaces. A linear space H endowed with an inner product [·, ·] H is called an inner product space, see [11], [4].
is a neutral subspace (the isotropic part of D) and D + and D − are closed (uniformly) positive and negative subspaces of (H, J); see e.g. [11,Theorems IX.2.5]. We will need the following slightly modified version of this statement.
Lemma 2.1. Every linear subspace T of a Pontryagin space (H, J) admits the following decomposition where T + is a positive subspace of (H, J), such that T + is a maximal positive subspace of T, and T 1 is a k-dimensional subspace of (H, J), where k = dim T/T + .
Proof. Let D be the closure of T in H and decompose D as in (2.2), , D − , and D + are closed neutral, negative, and positive subspaces of (H, J), respectively. Since T is a dense subset of D and the subspaces D 0 and D − are finite dimensional, T has a dense intersection with D + , see e.g. [32,Lemma 2.1]. Denote T + := T ∩ D + and let k = dim (D 0+ D − ). Since T = D one concludes that there exists a k-dimensional subspace T 1 ⊂ T \ D + . The closed subspace T 1 ⊂ T decomposes T and (2.4) together with a dimension argument leads to The equality T = T 1+ T + combined with (2.4) [2]. Often a linear operator T : H 1 → H 2 will be identified with its graph For a linear relation T from H 1 to H 2 the symbols dom T , ker T , ran T , and mul T stand for the domain, kernel, range, and multivalued part, respectively. The inverse T −1 is a relation from Denote by ρ(T ) the resolvent set of T , by σ(T ) the spectrum of T and by σ p (T ) (resp. σ c (T ), σ r (T )) the point (resp. continuous, residual) spectrum of T . The adjoint T [ * ] is the closed linear relation from H 2 to H 1 defined by (see [9]) If V is a single-valued closed isometric operator in a Pontryagin space H then the subspaces ran (V − λI) are closed for every λ ∈ D ∪ D e , see e.g. [40,Section 1.3] or Lemma 2.4 below, and each of the sets σ p (V ) ∩ D and σ p (V ) ∩ D e consist of at most κ = κ − (H) eigenvalues, see e.g. [31,p.49 Corollary 2]. Denote by N λ the defect subspace of V : As is known, see [31, Theorem 6.1], the numbers dim N λ take a constant value The numbers n ± (V ) are called the defect numbers of V .
In the case of Pontryagin spaces some further results on isometric and unitary relations can be established. For any isometric relation T between two Kreȋn spaces it is clear that ker T and mul T are neutral subspaces. Therefore, in a Pontryagin space ker T and mul T are necessarily finite dimensional. If T is closed then ker T and mul T are also closed. In Pontryagin spaces the following stronger result is true. Proof. The isometry of T means that T −1 ⊆ T [ * ] . Taking inverses one gets T = (T −1 ) −1 ⊆ T −[ * ] , i.e., T and T −1 are simultaneously isometric. Therefore, to prove the statement it suffices to prove that the range of T is a closed subspace in H 2 , since T is closed precisely when its inverse T −1 is closed and clearly dom T = ran T −1 . Now let dom T be decomposed as in Lemma 2.1, so that D + := T + is a maximal uniformly positive subspace of D := dom T . Next introduce the restriction of (the graph of) T by setting Then T + is closed and as a restriction of T it is also an isometric relation from (H 1 , J H1 ) to (H 2 , J H2 ). Moreover, dom T + = T + ⊆ D + is a uniformly positive subspace. This implies that for all {f, f ′ } ∈ T + and some δ > 0, , which shows that ker T + = {0}, so that (T + ) −1 is an isometric operator, and, moreover, . Therefore, the closed isometric operator (T + ) −1 is also bounded. Consequently, ran T + = dom (T + ) −1 is a closed subspace in H 2 . On the other hand, since dom T admits the decomposition (2.11), where T 1 is finite dimensional and mul T = mul T + (also finite dimensional), one concludes that ran T = T (T 1 ) + ran T + as a finite dimensional extension of the closed subspace ran T + is a closed subspace of H 2 . This completes the proof. Lemma 2.4 can be seen as an extension of [11,Theorem IX.3.1]. It is known e.g. from [11,Theorem IX.3.2] and [31,Theorems 6.2,6.3]) that if T is an isometric operator in a Pontryagin space such that ran T (resp. dom T ) is a nondegenerate subspace, then T (resp. T −1 ) is continuous. The next lemma contains main properties of isometric relations acting between two Pontryagin spaces.
Lemma 2.5. For an isometric relation T from the Pontryagin space (H 1 , J H1 ) to the Pontryagin space (H 2 , J H2 ) the following statements hold: , then ran T is a closed nondegenerate subspace of H 2 and, moreover, T and T −1 are continuous operators.
Thus T is a closable operator, which by Lemma 2.4 and the closed graph theorem implies that T and, therefore, also T is continuous. Similarly it is seen that if dom T is nondegenerate, and then T −1 and T −1 are continuous.
(ii) Since dom T = H 1 is nondegenerate, T −1 is a continuous operator by item (i). On the other hand dom T , as a dense subspace of H 1 , contains a negative subspace D − ⊂ dom T of dimension κ − (H 1 ); see [11,Theorem IX.1.4]. Then also ran T contains a negative subspace of the same dimension and hence κ − (H 2 ) ≥ κ − (H 1 ). Now assume that κ − (H 1 ) = κ − (H 2 ). Then ran T = ran T is necessarily a nondegenerate subspace of the Pontryagin space (H 2 , J H2 ), see e.g. [11,Lemma II.10.5], and (i) shows that T is a continuous operator.
(iii) This follows by applying (ii) to T −1 , which is also an isometric relation.
(iv) If T is unitary then the conditions ker T = mul T = {0} are equivalent to dom T = H 1 and ran T = H 2 ; see (2.9) in Proposition 2.2. Now the assertions follow from (ii) and (iii).

Operator colligations.
Let H be a Pontryagin space, L 1 and L 2 be Hilbert spaces. The set of bounded everywhere defined operators from L 1 to L 2 is denoted by B(L 1 , L 2 ), B(L 1 ) := B(L 1 , L 1 ). Let U be a bounded operator from H ⊕ L 1 to H ⊕ L 2 represented in the block form The quadruple (H, L 1 , L 2 , U ) is called a colligation, H is the state space, L 1 and L 2 are the incoming and the outgoing spaces, T is the main operator and U is called the connecting operator of the colligation. The colligation (H, ) is called adjoint to the colligation (H, L 1 , L 2 , U ); cf. [12,1]. Components of a unitary colligation satisfy the following identities Recall, see e.g. [1], that a B(L 1 , L 2 )-valued function Θ(λ) is said to belong to the generalized Schur class S 0 κ (L 1 , L 2 ) if it is holomorphic in a neighborhood Ω of 0 and the kernel (2.15) K Θ ω (λ) = I − Θ(λ)Θ(ω) * 1 − λω has κ negative squares in Ω × Ω, i.e. for any finite set of points ω 1 , . . . , ω n in Ω and vectors f 1 , . . . , f n in L 2 , the Hermitian matrix has at most κ negative eigenvalues, and for some choice of ω 1 , . . . , ω n in Ω and f 1 , . . . , f n in L 2 the matrix (2.16) has exactly κ negative eigenvalues.
As is known, see [1], the characteristic function of a closely connected unitary colligation belongs to the generalized Schur class S 0 κ (L 1 , L 2 ), where κ = κ − (H). Moreover, the converse is also true; see e.g. In what follows a B(L 1 , L 2 )-valued function Θ(·) holomorphic in some open subset Ω ⊂ D is said to belong to the generalized Schur class S κ (L 1 , L 2 ), if the kernel (2.15) has κ negative squares in Ω × Ω. In particular, we do not require that 0 ∈ Ω, which implies that characteristic functions of unitary colligations used in Theorem 2.6 are not sufficient to give a realization for all functions Θ(·) from the class S κ (L 1 , L 2 ) . In the next section we introduce the notions of a unitary boundary pair for an isometric operator and an associated Weyl function as a replacement for unitary colligations and their characteristic functions. These new notions allow to realize an arbitrary operator function from the class S κ (L 1 , L 2 ) as the Weyl function of a Pontryagin space isometric operator, corresponding to some unitary boundary pair.

Unitary boundary pairs for isometric operators
3.1. Unitary boundary pairs and the main transform. Let H be a Pontryagin space with the negative index κ and the fundamental symmetry J H and let L 1 and L 2 be Hilbert spaces. In this section we introduce the notion of a unitary boundary pair for an isometric operator V : here, and in what follows, V is assumed to be closed.
For this purpose equip the Hilbert spaces H 2 and L = L 1 × L 2 with the indefinite inner products by the formulas and (L, J L ) are Kreȋn spaces. In particular, for V ⊂ H 2 the linear set V [⊥] in the Kreȋn space (H 2 , J H 2 ) can be characterized as follows (1) V = ker Γ and for all { f , u}, { g, v} ∈ Γ the following identity holds Item (1) of Definition 3.1 means that Γ is an isometric linear relation from the Kreȋn space (H 2 , J H 2 ) to the Kreȋn space (L, J L ), while items (1) and (2) together mean that Γ is unitary.
Application of Proposition 2.2 to a unitary boundary pair leads to the following statement. Define the components Γ 1 and Γ 2 of Γ by In the case that Γ 1 and Γ 2 are single-valued, i.e. mul Γ 1 = mul Γ 2 = 0 and dom Γ = V −[ * ] and ran Γ = L 1 × L 2 the collection {L 1 × L 2 , Γ 1 , Γ 2 } is called an ordinary boundary triple for the isometric operator V . For a Hilbert space isometric operator the corresponding notion was introduced and studied in [41,42] as a boundary triple for the dual pair (V, V −1 ). For ordinary boundary triples an application of the closed graph theorem shows that the component mappings Γ 1 and Γ 2 are bounded. However, for a general unitary boundary pair (L, Γ) the mappings Γ 1 and Γ 2 need not be bounded or single-valued. With Γ 1 and Γ 2 one associates the extensions V 1 and V 2 of V by the equalities It follows from the identity (3.2) that V 1 is an expanding linear relation and V 2 is a contractive linear relation in the Pontryagin space H. Moreover, it is clear from In establishing some properties of unitary boundary pairs it is useful to connect the unitary relation Γ which acts between two Kreȋn spaces to another unitary relation that acts between two Pontryagin spaces, since unitary relations between Pontryagin spaces have simpler structure. For this purpose we introduce the following transform from the Kreȋn space It establishes a one-to-one correspondence between (closed) linear relations Γ from (H 2 , J H2 ) to (L, J L ) and (closed) linear relations U from H 2 to H 1 via The linear relation U will be called the main transform of Γ; cf. [19] for the case of symmetric operators. In the following lemma, which is an analog of [19, Proposition 2.10], some basic properties of the transform J are given.

Moreover, the transform J establishes a one-to-one correspondence between isometric (unitary, contractive, expanding) relations
Proof. It is straightforward to check that for all elements of the form the following identity is satisfied: In view of (3.1) this identity implies the equivalence which leads to identity (3.7). It follows from (3.7) that i.e., U is isometric (unitary) precisely when Γ is isometric (resp. unitary). The connection between contractive (expanding) relations Γ and U is clear from (3.8).
The next proposition contains the basic properties of Γ 1 , Γ 2 and V 1 , V 2 for a unitary boundary pair (L, Γ). (ii) ran Γ 1 = L 1 and ran Γ 2 = L 2 ; (iii) mul Γ j = P Lj (mul Γ), j = 1, 2, and the following equivalences hold: (v) the following equivalences hold: If one of the sets appearing in the above equivalences is trivial, then the main and letting n, m → ∞ one concludes that u 2,n − u 2,m L2 → 0. As a Cauchy sequence (u 2,n ) converges to some element u 2 in L 2 . This means that { f n , u n } → { f , u} and, since Γ is closed as a unitary relation, { f , u} ∈ Γ and thus { f , u 1 } ∈ Γ 1 . This proves that Γ 1 is closed. Similarly one proves that Γ 2 is closed.
(ii) First it is shown that ran Γ 1 is a closed subspace of L 1 . For this consider the main transform U = J (Γ). By Lemma 3.3 U is a unitary relation between the Pontryagin spaces H × L 2 and H × L 1 . Moreover, by Lemma 2.4 ran U is closed and Proposition 2.2 shows that mul U is the isotropic part of ran U. Therefore mul U is a closed finite dimensional subspace of H × L 1 and thus also the co-dimension k of ran U is finite (k ≤ κ − (H)). Let M be any k-dimensional subspace such that ran U+M = H × L 1 and let P 1 be the orthogonal projection from H × L 1 onto L 1 . Then L 1 = P 1 (ran U+M) = P 1 ran U + P 1 M, and here dim P 1 M ≤ k, which implies that P 1 ran U = ran Γ 1 is closed.
(iv) Since V j = ker Γ j and Γ j is closed by item (i) also V j is closed j = 1, 2. If f ∈ V 1 and g ∈ V 2 then it follows from (3.2) that [ f , g] H 2 = 0 and in view of (3.1) this means that the inclusions hold; these inclusions are clearly equivalent to each other.
(v) The definition in (3.6) shows that 2) implies that (u 1 , u 1 ) L1 = 0 and thus also u 1 = 0, i.e., mul U = {0}. The equivalence of ker U = {0} and ker V 1 = {0} can be seen in the same way. As to the last equivalence notice that Later it is shown that the inclusions in (iv) of Proposition 3.4 actually hold as equalities; see Theorem 3.15. In the special case that H is a Hilbert space and V is an isometry in H Proposition 3.4 can be specialized as follow. Proof. The fact that in the Hilbert space case V 2 and V −1 1 are contractive operators follows from (3.2) which with the choice f = g and u = v can be rewritten as In and the corresponding subset of V * : and an application of (3.2) shows that and define the vector valued function f j by Then the Gram matrix of the functions f j (λ) (j = 1, . . . , n) in the space L 2 (H) over the unit circle Proof. To determine the Gram matrix consider the inner product of the H-valued functions f j (λ) and f k (λ) with λ = e it , t ∈ [0, 2π], for j, k = 1, . . . , n.
(i) If λ j ∈ D then in view of the equalities differs in sign from the Gram matrix for functions f j (λ), j = 1, . . . , n.
The assumption λ ∈ σ p (V ) means that u 1 = 0. Now assume that λ 1 , . . . , λ κ+1 ∈ D e and for some linearly independent vectors Then from (3.9) one gets If λ j = λ k for some j = k then the vectors u j and u k are linearly independent by the assumptions λ j ∈ σ p (V ). On the other hand, if λ j = λ k , then the vector functions f j (λ) and f k (λ) defined in Lemma 3.6 are also linearly independent. Hence the matrix G in Lemma 3.6 is invertible. One concludes that the form is negative for linearly independent vectors f λj when λ j ∈ D e . This contradicts the assumption that the Pontryagin space H has negative index κ.
(ii) The second statement is proved analogously.
In the sequel the following two subsets of D and D e (cf. (2.10)) will often appear: It should be noted that for various realization results and for the study of proper extensions of the isometry V it is typically sufficient to assume that σ p (V ) = ∅; this is the case in particular when V is a simple isometric operator in H. In this case Proposition 3.7 shows that both of the sets σ p (V 2 ) ∩ D e and σ p (V 1 ) ∩ D contain at most κ points. Now consider the restrictions of (the graphs of) Γ 1 and Γ 2 to N λ (V * ), It follows from (3.10) that and the assumption σ p (V ) = ∅ guarantees in particular that the inverses determine single-valued operator functions, which will be denoted by the same symbols Let π 1 and π 2 be projections onto L 1 and L 2 in L, respectively.
Definition 3.8. The operator functions will be called the γ-fields of the unitary boundary pair (L, Γ).
The definition of the γ-fields of the unitary boundary pair (L, Γ) yields the following explicit formulas: Later it is shown that γ 1 (λ) and γ 2 (λ) are bounded everywhere defined operators and holomorphic in λ; see Theorem 3.15.
Definition 3.9. The family of linear relations defined by will be called the Weyl family of V corresponding to the unitary boundary pair (L, Γ), or, briefly, the Weyl family of the unitary boundary pair (L, Γ).
In Theorem 3.15 it will be shown that the formula (3.15) determines a singlevalued operator function, which is called the Weyl function of V corresponding to the boundary triple (L, Γ). If the mapping Γ is single-valued, then the Weyl function Θ(λ) can be defined by using the γ-fields

Unitary boundary pairs and unitary colligations.
In the present section we consider a unitary boundary pair whose main transform is a unitary colligation and write explicit formulas for all the objects connected with this unitary boundary pair in terms of the blocks of U .
and the linear relation V * takes the form (iv) The multivalued part of Γ has the representations (v) The linear relations V 1 and V 2 are given by and hence the sets D := ρ(V 1 ) ∩ D and D e := ρ(V 2 ) ∩ D e are nonempty, they coincide with the sets in (3.10) and are connected by (ii) Since the operator U : is also the graph of the operator U −1 in (3.17), and hence U has the following representations In view of (3.6) this yields the first formula in (3.18). The equality leads to the second representation of Γ in (3.18).
e . This completes the proof.
By Proposition 3.2 the closure of V * is V −[ * ] . Hence, by taking closures in (3.20) one arrives at the following representations for V −[ * ] :  .13), (3.14). Then the following statements hold: Proof. (i) Recall that N λ (V * ) consists of vectors f λ λf λ ∈ V * . Therefore, the vector Hence, for λ ∈ D = ρ(T −1 ) ∩ D one obtains Similarly, the vector (ii) By the first formula in (3.18) one gets and in view of (3.11) and (3.12) the γ-field γ 1 (λ) takes the form Similarly, by the second formula in (3.18) and hence (iii) It follows also from (3.28) that (iv) Notice that in view of (3.24) and (3.25) the defect subspaces N λ take the form  .
Proof To prove the remaining assertions let {f, f ′ } ∈ S. Then {f, f ′ − λf } ∈ (S − λI) and this is equivalent to where α ∈ D, 1−αλ = 0. This formula with λ ∈ C gives the equalities for ker (S −λ) and ran (S − λ). Analogously the choice λ = ∞ corresponds to µ (α) (∞) = −1/α and this yields the formulas for mul S and dom S in (i) and (ii). The statements (iii) and (iv) follow from (i) and (ii) when applying the definitions of the resolvent set ρ(S) and the spectral components σ j (S), j = s, c, r.
In the next lemma the transform M (α) ∈ B(H 2 ) is composed with a unitary boundary pair. Lemma 3.14. Let V be a closed isometric operator in a Pontryagin space H, let (L, Γ) be a unitary boundary pair for V , and let α ∈ C, |α| = 1. Then is also a closed isometric relation in H. Moreover, mul V (α) = {0} precisely when α −1 ∈ σ p (V ), and in this case: (i) The composition Proof. The statements concerning the linear relation V (α) are implied by Lemma 3.13.
The next two theorems contain a full characterization of the class of Weyl functions Θ(λ) of boundary pairs. In the first theorem it is shown that Θ(λ) belongs to the generalized Schur class S κ (L 1 , L 2 ).
There is an analog for the notion of transposed boundary triple (see [19]) for boundary pairs of isometric operators. In the present case this notion contains the second boundary triple associated with a dual pair {V, V −1 } as defined in [41] in the case of ordinary boundary triples for Hilbert space isometries. For this purpose the notion of transposed boundary pair (L 2 ⊕ L 1 , Γ ⊤ ) is introduced for boundary pairs of isometric operators and its basic properties are established in the next proposition.
(ii) The statement is clear from the definition of (L 2 ⊕ L 1 , Γ ⊤ ).
Remark 3.21. Proposition 3.20 shows that in a Pontryagin space H every unitary boundary pair (L, Γ) (as well as its transposed boundary pair) of an isometric operator V can be seen as an analog of so-called (B-)generalized boundary triple, since the component mappings Γ 1 and Γ 2 are surjective and the corresponding kernels V 1 = ker Γ 1 and V 2 = ker Γ 2 are closed extensions of V with nonempty resolvent sets; see [26,19] and [22,23,24] for some further developments.
If, in addition, the operator V is simple then the conditions (i)-(iv) are equivalent to Proof. (i) ⇔ (ii) This is item (ii) in Proposition 3.2.
If 0 ∈ ρ(S Θ λ (λ)) for some λ ∈ D ∪ D e , then the conditions (i)-(iv) hold. If one of the condition (i)-(iv) is satisfied then (L, Γ) is an ordinary boundary pair for V .
Finally, the fact that (L, Γ) is an ordinary boundary pair for V is clear from the properties in (ii).
The next example shows that the condition (v) in Proposition 3.23 cannot be replaced by a single condition ker S Θ λ (λ) = {0}. Example 3.25. Let H = C 4 with the skew-diagonal fundamental symmetry J = (δ j,5−k ) 4 j,k=1 , where δ j,k is the Kronecker delta and let us set e j = (δ j,k ) 4 k=1 , (j = 1, 2, 3, 4). Let V be an isometry in H which maps e 1 into e 2 . Then the defect subspaces of V are degenerate for all λ ∈ C and thus, the operator V is not standard.

The linear relation V −[ * ]
consists of vectors Therefore, the left part of the identity (3. 2) for f = g takes the form and can be rewritten in the diagonal form Hence a single-valued boundary triple (L 1 ⊕ L 2 , Γ 1 , Γ 2 ) can be chosen as follows (3.76) Then for |λ| < 1 one obtains from (3.75) and (3.76) where Λ = diag (1, λ, λ 2 ), Ω = diag (1, ω, ω 2 ). Notice that in this example det S Θ ω (λ) ≡ 0 for all λ, ω ∈ D\{0}, while ω∈D\{0} ker S Θ ω (λ) = {0}. In fact, for every pair and similarly with L = L 1 × L 2 it determines a unitary mapping from the Kreȋn It follows that the linear relation with the kernel ker Γ = A and the domain holds for all { f , u} ∈ Γ and due to [19,22] the unitarity of Γ means that (H 2 , Γ) is a unitary boundary pair for the symmetric operator A. This boundary pair becomes ordinary when the mapping Γ is surjective or, equivalently, when (H 2 , Γ) is an ordinary boundary pair for V .
Notice that part (b) of Proposition 3.27 contains the properties and generality that can be attained when applying (ordinary) boundary triples for isometric operators which have been introduced and studied in [41,42].
Remark 3.28. An analog of boundary triple in scattering form (3.2) is encountered in [17], where extension theory of multiplication operators in indefinite de Branges spaces was developed. The role of the Weyl function in that work is played by de Branges matrix.

Extension theory and generalized coresolvents
. The set of proper extensions of V was parametrized in [5] via an ordinary boundary triple. In the present section we consider extensions V of the isometric operator V , which are proper with respect to a given unitary boundary pair (L, Γ), i.e.
If 0 ∈ D then the formula (4.2) for λ = 0 takes the form Proof. (i) Using Definitions 3.8, 3.9 it is seen that for all u 2 ∈ L 2 , and some v 2 ∈ L 2 , λ ∈ D \ {0}. By applying the identity (3.2) to these elements one obtains or, equivalently, Since u 2 ∈ L 2 is arbitrary this implies the equality and , which together with (4.6) yields (4.4).

4.2.
Weyl function and spectrum of proper extensions of V . A unitary boundary pair (L, Γ) is a tool which allows to determine those extensions V of V that satisfy V V V * in the following way. Let Φ be a linear relation from L 1 to L 2 represented in the form where H is an auxiliary Hilbert space and Φ j are bounded linear operators Φ j : H → L j (j = 1, 2), such that The following theorem gives a description of the spectrum of V Φ and contains a Kreȋn type resolvent formula.
When (ii) is satisfied the resolvent of V Φ takes the form If λ ∈ D e then: When (iv) is satisfied the resolvent of V Φ takes the form (4.12) If, in addition, Π = (L, Γ) is an ordinary boundary triple for V then the implications (i)-(iv) become equivalences.
Proof. The proof is divided into steps.
The operator function (I H − zV Φ ) −1 is called the coresolvent of V Φ . Setting λ = 1/z in Theorem 4.2 one obtains the following statement for coresolvents of V Φ .
If z ∈ D e then: If, in addition, Π = (L, Γ) is an ordinary boundary triple for V then the implications (i)-(iv) become equivalences.
Remark 4.4. If (L 1 ⊕ L 2 , Γ 1 , Γ 2 ) is an ordinary boundary triple for V then every closed proper extension of V can be represented in the form (4.10) with Φ j ∈ B(H, L j ) (j = 1, 2) such that (4.9) holds due to [ .7), then the following equivalences hold: (1) V Φ is an isometric relation in H ⇐⇒ Φ is a graph of an isometric operator; (2) V Φ is a unitary relation in H ⇐⇒ Φ is a graph of a unitary operator; (3) V Φ is a contractive relation in H ⇐⇒ Φ is a graph of a contraction. The fact that the implications (i)-(iv) of Theorem 4.2 become equivalences for an ordinary boundary triple Π = (L, Γ) = (L, Γ 1 , Γ 2 ) was proved in [41] in the case when κ = 0, and in [5] in the case κ = 0.

Description of generalized coresolvents.
Definition 4.5 (see [39,38]). An operator-valued function where P H is the orthogonal projection from H onto H. Notice that in [38] the operator function K z in (4.29) is called generalized resolvent of V . The representation (4.29) of the generalized coresolvent of V is called minimal, if The generalized coresolvent K z is said to be k-regular, if k = κ − ( H[−]H) for a minimal representation (4.29).
Every generalized coresolvent K z of the isometric operator V admits a minimal representation (4.29) and every two minimal representations of K z are unitarily equivalent, see [14,Proposition 4.1] for the case of a symmetric operator.
Lemma 4.6. Let K z be a ( κ − κ)-regular generalized coresolvent. Then the kernel Proof. Let {z j } n j=1 be a set of points in ρ( V −1 ) ∩ D and let g j ∈ H, j = 1, 2, . . . , n. Denote Then it follows form the first equality in (4.31) that Since V is a unitary relation in H one obtains It follows from (4.30), (4.31), and (4.32) that n j,k=1 is dense in H[−]H and hence it contains a ( κ − κ)-dimensional negative subspace. Therefore, the form (4.33) has exactly κ − κ negative squares for an appropriate choice of z j , g j (j = 1, 2, . . . , n).
Remark 4.7. In the case of a standard isometric operator the statement of Lemma 4.6 was proved in [28].
Then for z ∈ D ∩ ρ( V −1 ) the formula establishes a one-to-one correspondence between the set of κ − κ-regular generalized coresolvents of V and the set of all operator-valued functions ε(·) ∈ S κ−κ (L 1 , L 2 ), such that For z ∈ D e ∩ ρ( V −1 ) the formula (4.34) takes the form Proof. The proof is divided into steps.
Then it follows from (4.55), (4.56) and (3.2) that with ξ j ∈ C, n j,k=1 a j,k ξ j ξ k := n j,k=1 the form in (4.57) is reduced to n j,k=1 a j,k ξ j ξ k = n j,k=1 [R z k (z j )g j , g k ] H ξ j ξ k .
Remark 4.9. 1) For a standard isometric operator in a Pontryagin (resp. Kreȋn) space similar formulas for generalized coresolvents were found in [39,38,40] (resp. [28]). For the case of a nonstandard isometric operator in a Pontryagin space see [44]. An elegant proof of the formula for generalized resolvents of a nonstandard Pontryagin space symmetric operator with deficiency index (1,1) given by H. de Snoo was presented in [34]. In [5] a description of regular generalized resolvents of a nonstandard Pontryagin space isometric operator was given by the method of boundary triples. For a Hilbert space isometric operator this method was developed earlier in [41] and applied to the proof of Kreȋn type resolvent formulas (4.34), (4.36).
2) The extension V ε(z) appearing in (4.52) is an analog of Shtraus extension, which was introduced in [47] for the case of a symmetric operator. In view of (4.53) the vector function f z = K z g can be treated as a solution of the following "abstract boundary value problem" with z-dependent boundary conditions 3) In abstract interpolation problem considered in [36] the crucial role was played by the Arov-Grossman formula for scattering matrices of unitary extensions of isometric operators, [3]. In [6] the formula for generalized coresolvents was applied to the description of scattering matrices of unitary extensions of Pontryagin space isometric operators which, in turn, was used in [7] for parametrization of solutions of an indefinite abstract interpolation problem, see also [15]. The present version of formula (4.34) will allow to consider k-regular indefinite interpolation problems with the growth of index κ.