Nonlinear flag manifolds as coadjoint orbits

A nonlinear flag is a finite sequence of nested closed submanifolds. We study the geometry of Fréchet manifolds of nonlinear flags, in this way generalizing the nonlinear Grassmannians. As an application, we describe a class of coadjoint orbits of the group of Hamiltonian diffeomorphisms that consist of nested symplectic submanifolds, i.e., symplectic nonlinear flags.


Introduction
Let M be a smooth manifold, and suppose S 1 , … , S r are closed smooth manifolds. A nonlinear flag of type S = (S 1 , … , S r ) in M is a sequence of nested embedded submanifolds N 1 ⊆ ⋯ ⊆ N r ⊆ M such that N i is diffeomorphic to S i for all i = 1, … , r . The space of all nonlinear flags of type S in M can be equipped with the structure of a Fréchet manifold in a natural way and will be denoted by Flag S (M) . The aim of this paper is to study the geometry of this space using the convenient calculus of Kriegl and Michor [18].
Nonlinear flag manifolds provide a natural generalization of nonlinear Grassmannians which correspond to the case r = 1 . Nonlinear Grassmannians (a.k.a. differentiable Chow manifolds) play an important role in computer vision [1,24] and continuum mechanics [25]. They have also been used to describe coadjoint orbits of diffeomorphism groups. Nonlinear Grassmannians of symplectic submanifolds have been identified with coadjoint orbits of the Hamiltonian group in [12]. Codimension two Grassmannians have been used to describe coadjoint orbits of the group of volume-preserving diffeomorphisms [12,16]. Let us also point out that every closed k-fold vector cross-product on a Riemannian manifold induces an almost Kähler structure on the nonlinear Grassmannians of (k − 1)-dimensional submanifolds [20].
In some applications, decorated nonlinear Grassmannians have been considered, that is, spaces of submanifolds equipped with additional data supported on the submanifold. Functional shapes (fshapes), for instance, may be described as signal functions supported on shapes [3][4][5]. Weighted nonlinear Grassmannians of isotropic submanifolds have been used to describe coadjoint orbits of the Hamiltonian group [9,19,28]. Recently, weighted nonlinear Grassmannians of isotropic submanifolds have been identified with coadjoint orbits of the contact group [13]. Decorated codimension one Grassmannians may be used to describe coadjoint orbits of the group of volume-preserving diffeomorphisms [10]. The nonlinear flag manifolds considered in this paper may be regarded as yet another class of decorated Grassmannians.
Some nonlinear flag manifolds have already appeared in the literature too. Landmarkconstrained planar curves, for instance, have been used in a statistical elastic shape analysis framework in [26]. Landmark-constrained surfaces in the context of shape analysis are being discussed in [17,Chapter 6]. An attempt to use the nonlinear flag manifold of surfaces in ℝ 3 decorated with curves as shape space can be found in [6]. Manifolds of weighted nonlinear flags are the object of study in [14]. We hope that the foundational material on nonlinear flag manifolds provided in this paper will prove helpful in future research.
As a first application, we will use nonlinear flag manifolds to describe certain coadjoint orbits of the Hamiltonian group. To be more explicit, suppose M is a closed symplectic manifold and let Flag The remaining part of this paper is organized as follows. Section 2 contains a rigorous study of the Fréchet manifold Flag S (M) and related principal bundles. In Sect. 3, we discuss the oriented analogue, that is, the Fréchet manifold of all oriented nonlinear flags, a finite covering of Flag S (M) . In Sect. 4, we study the action of the Hamiltonian group on the open subset of symplectic flags and provide a proof of Theorem 4.5 mentioned before.

3
In Proposition 2.3, we will show that Flag S (M) may be considered as a smooth submanifold in the product of nonlinear Grassmannians, Gr S 1 (M) × ⋯ × Gr S r (M) . Recall that for a closed manifold S, the Grassmannian Gr S (M) , i.e., the space of all submanifolds in M which are diffeomorphic to S, is a smooth Fréchet manifold whose tangent space at N ∈ Gr S (M) can be canonically identified as T N Gr S (M) = (TM| N ∕TN) . The Grassmannian is the base of a (locally trivial) smooth principal bundle with structure group Diff(S) ; see [2,22,23] and [18,Theorem 44.1]. Recall that the space of embeddings, Emb S (M) , is a smooth Fréchet manifold whose tangent space at ∈ Emb S (M) can be canonically identified as T Emb S (M) = ( * TM) . Moreover, the group of all diffeomorphisms, Diff(S) , is a Fréchet Lie group with Lie algebra (S) , the Lie algebra of vector fields. We will show that the space of nonlinear frames, i.e., the space of all parametrized flags, is the total space of a smooth principal bundle over Flag S (M) with structure group Diff(S 1 ) × ⋯ × Diff(S r ) which generalizes the fundamental frame bundle over Gr S (M) in (1).
In Proposition 2.10, we will exhibit a reduction of the structure group that permits to regard (unions of connected components of) Flag S (M) as the base of a principal bundle with total space Emb S r (M) and structure group Diff(S r ;Σ) , the group of diffeomorphisms preserving a certain flag Σ in S r .
In Proposition 2.6, we will show that the manifold Flag S (M) is diffeomorphic to a twisted product of two flag manifolds of shorter lengths. Iterating this observation, one is led to a description of Flag S (M) as a twisted product of nonlinear Grassmannians, cf. Remark 2.7.
In Proposition 2.9, we will describe (unions of connected components of) Flag S (M) as a homogeneous space of Diff c (M) , the group of compactly supported diffeomorphisms. Recall that the group Diff c (M) is a smooth Lie group with Lie algebra c (M) ; see [22] and [18,Theorem 43.1].
Evidently, the aforementioned statements on nonlinear flag manifolds can be considered as generalizations of well known facts about diffeomorphism groups, spaces of embeddings and nonlinear Grassmannians. Since the proofs we will provide rely crucially on these classical results (and little else), we start by summarizing them in Lemma 2.1.

Background on nonlinear Grassmannians
Throughout this paper, we will use the convenient calculus of Kriegl and Michor [18] to describe smooth structures on spaces of manifolds and maps. Within this framework (infinite dimensional), smooth manifolds are modeled on convenient vector spaces, a class of locally convex vector spaces satisfying a weak completeness assumption. A map is smooth if and only if it maps smooth curves to smooth curves. The natural domains for locally defined smooth functions are subsets which are open with respect to the c ∞ -topology, the final topology with respect to all smooth curves. For Fréchet manifolds, the c ∞ -topology coincides with the Fréchet topology; see [18,Theorem 4.11(1)].
A submanifold will be called splitting submanifold if the corresponding (closed) linear subspace in a submanifold chart admits a complement, cf. [18,Definition 27.11]. A subgroup H in a Lie group G will be called a splitting Lie subgroup if it is a splitting submanifold of G. In this case, H is a Lie group with the induced structure.
Let ∶ TM → M be a smooth map such that TM → M × M , X ↦ ( (X), (X)) , is a tubular neighborhood of the diagonal, where ∶ TM → M denotes the tangent bundle projection. In particular, we assume (0 x ) = x for all x ∈ M . If U is a sufficiently C 1 small zero neighborhood in c (TM) , then is a standard chart for the smooth structure on Diff c (M) centered at the identity; see [18,Theorem 43.1]. We may choose such that X ∈ TN ⇔ ( (X), (X)) ∈ N × N . Thus, in the aforementioned chart, the sequence of subgroups corresponds to the sequence of linear inclusions Since both linear inclusions admit complements, we see that Diff c (M; ) is a splitting Lie subgroup of Diff c (M;N) and the latter is a splitting Lie subgroup of Diff c (M). Let is a standard chart for the smooth structure on Emb S (M) centered at ; see [18,Theorem 42.1]. By construction, Note that the first assertion in Lemma 2.1(c) may be considered as a strengthening of the classical isotopy extension theorem[15, Theorem 1.3 in Chapter 8].
We will also use the following simple fact.

The fundamental frame bundle
Suppose S 1 , … , S r are closed smooth manifolds and put S ∶= (S 1 , … , S r ) . Let

Proposition 2.3
In this situation, the following hold true: Proof It is well known that Emb S i (M) → Gr S i (M) is a smooth principal fiber bundle with structure group Diff(S i ) ; see Lemma 2.1(a). Hence, the product of these maps, is a smooth principal fiber bundle with structure group ∏ r i=1 Diff(S i ) . Clearly, Fr S (M) is the preimage of Flag S (M) under the map (4). Therefore, it suffices to show (a).
We will prove (a) by induction on r. Suppose N r ∈ Gr S r (M) . Since the Diff c (M) action on Gr S r (M) admits local smooth sections (see Lemma 2.1(b)), there exists an open We obtain a diffeomorphism Clearly, this diffeomorphism maps the part of Flag S (M) contained in is a splitting smooth submanifold of Gr S i (M) according to Remark 2.2. Combining these two statements, we conclude that Flag S 1 ,…,S r−1 (N r ) × U is a splitting smooth submanifold of . It is straightforward to track the tangent spaces through this inductive proof and establish the description in (2). ◻

Remark 2.4
Note that the principal Diff(S ) bundle (3) is the restriction of the principal bundle in (4) along the inclusion The only freedom for the i , i < r , is in their

A tower of Grassmannians
Suppose, for a moment, that S = (S 1 , S 2 ) consist of just two model manifolds. Then, is the associated bundle to the principal bundle Emb S 2 (M) → Gr S 2 (M) for the natural Diff(S 2 ) action on Gr S 1 (S 2 ) . To see this, we first observe that the projection . Indeed, the canonical identification is a diffeomorphism, cf. the proof of Proposition 2.6(a). Via this identification, the natural right action of  where each arrow is labeled with its typical fiber or structure group, respectively.
Proof Clearly, the map in (7) is bijective with inverse, Smoothness of the inverse follows from the fact that this is the restriction of a smooth map given by the same formula and from Proposition 2.3(b). To check smoothness of the map in (7), we fix Then, the map in (7) may be expressed in the form provided +1 ∈ U and +1 (S +1 ) ⊆ V . Note that the same formula provides a smooth extension, mapping an open subset in . Hence, using Proposition 2.3(b), we conclude that (7) is smooth. This proves (a).
Using Proposition 2.3(c) and [18,Section 37.12], one readily checks that (7) induces a diffeomorphism as indicated in (9), whence (b). The statements in (c) are now obvious. ◻ Remark 2.7 Iterating Proposition 2.6(a), we obtain a canonical diffeomorphism: Iterating Proposition 2.6(b), we see that the nonlinear flag manifold Flag S (M) may be regarded as a twisted product of the nonlinear Grassmannians Gr S 1 (S 2 ), … , Gr S r−1 (S r ) and Gr S r (M).

3
The forgetful map (8) becomes (N 1 , … , N r ) ↦ N r . This allows to interpret nonlinear flags as nonlinear Grassmannians decorated with an extra structure: The flag (N 1 , … , N r ) can be seen as a submanifold N r of M decorated with a nonlinear flag

Nonlinear flag manifolds as homogeneous spaces
Clearly for all N � r ∈ U . Using Proposition 2.3(a) and Remark 2.2, we see that . Hence, we obtain a smooth map The statement in (c) is an immediate consequence of (a) and (b). ◻

A reduction of the structure group
Consider a sequence of embeddings (11) (20), mapping N to ( 1 , … , r ).
The statements in (e) are now obvious. ◻

Remark 2.11
As in Proposition 2.6, we split the sequence S into S � = (S 1 , … , S ) and S �� = (S +1 , … , S r ) , with � ∶= ( 1 , … , −1 ) and �� ∶= ( +1 , … , r−1 ) . Moreover, we consider the flags Σ ∶= (Σ 1 , … , Σ r−1 ) and Σ �� ∶= (Σ +1 , … , Σ r−1 ) in S r with Σ i as in (18). As in the proof of Proposition 2.9(b), one can show that the canonical homomorphism is the embedding of a splitting Lie subgroup; see also Proposition 2. where each arrow is labeled with its typical fiber or structure group, respectively.  f (x)) . All these facts appear to be well-known folklore. More general results for flag manifolds will be formulated and proved below; see Proposition 2.12. Tautological bundles will be used in Sect. 3 to describe the transgression of differential forms. In [7], they are used for the transgression of differential characters to nonlinear Grassmannians.

Tautological bundles
Over the manifold Flag S (M) of nonlinear flags, we have a nested sequence of tautological bundles with typical fibers S 1 , … , S r . The proof we will present below uses the description of the nonlinear flag manifold as a homogeneous space in Proposition 2.9.

Proposition 2.12 (Tautological bundles) For 1 ≤ i ≤ r , consider
Then, the following hold true:  The last two subsections are dedicated to the transgression of differential forms. We use integration along the fiber of tautological bundles to get differential forms on oriented nonlinear Grassmannians, as well as on manifolds of oriented nonlinear flags, from differential forms on M. All this follows readily from Lemma 2.1.

Example 3.1 The double coverings Gr or
and Gr or S 1 (S 2 ) → Gr S 1 (S 2 ) are nontrivial, while Gr or S 1 (ℝ 2 ) → Gr S 1 (ℝ 2 ) and Gr or S 1 (S 1 × S 1 ) → Gr S 1 (S 1 × S 1 ) are trivial double coverings. Indeed, if S 1 ≅ N ⊆ ℝ 2 is an embedded circle, then every diffeomorphism in Diff c (ℝ 2 ;N) ∩ Diff c (ℝ 2 ) • restricts to an orientation-preserving diffeomorphism on either connected component of the complement, ℝ 2 ⧵ N , and, thus, preserves the (induced boundary) orientation on N too. The same argument works for contractible circles in the torus, for the complement of such a circle consists of two nondiffeomorphic connected components. If S 1 ≅ N ⊆ S 1 × S 1 is not contractible, then the inclusion induces an injective homomorphism in first homology, H 1 (N) → H 1 (S 1 × S 1 ) . As every diffeomorphism in Diff(S 1 × S 1 ;N) ∩ Diff(S 1 × S 1 ) • induces the identity on H 1 (S 1 × S 1 ) , its restriction to N preserves the fundamental class of N and, thus, the corresponding orientation also.

Oriented nonlinear flags
Let us denote the space of all oriented nonlinear flags of type S by It follows from Proposition 2.3(a) that this is a splitting smooth submanifold in ∏ r i=1 Gr or Gr or

Transgression to nonlinear Grassmannians
We first recall the natural transgression of differential forms on M to differential forms on the nonlinear Grassmannian Gr or S (M) of oriented submanifolds [12,Section 2]. Each ∈ Ω dim(S)+ (M) induces ̃∈ Ω (Gr or S (M)) by Here, X denotes the infinitesimal action of X ∈ (M) on Gr or S (M). Let S be endowed with an orientation o S . Using the fiber integral for the trivial S-bundle

Transgression to manifolds of nonlinear flags
It works similarly for the transgression of differential forms to the manifold Flag or S (M) of oriented nonlinear flags. We start with a collection of differential forms on M: The transgression to Flag or S (M) can be defined with the help of the transgression (28)

3 4 Coadjoint orbits of symplectic nonlinear flags
As an application of the results presented above, we will now discuss how certain coadjoint orbits of the Hamiltonian group Ham(M) of a closed symplectic manifold can be parametrized by nonlinear flag manifolds, cf. Theorem 4.5. This generalizes [12, Theorem 3] about symplectic nonlinear Grassmannians (recalled in the first subsection below). We consider the manifold

Symplectic nonlinear Grassmannians
Let M be a closed manifold endowed with a symplectic form , and let S be a closed 2k-dimensional manifold. The symplectic nonlinear Grassmannian Gr symp S (M) of symplectic submanifolds of (M, ) of type S, introduced and studied in [12], is an open subset of the nonlinear Grassmannian Gr S (M) . Restricting the fundamental frame bundle in (1)  It has the same expression as in (28), but no orientation is needed now, since the symplectic submanifolds are naturally oriented by their induced Liouville volume forms. It also has similar functorial properties to the tilde calculus on oriented nonlinear Grassmannians (29).
Again, there is a way to obtain the transgressed form ̃ with a tautological bundle. Let  for all r ∈ (TM| N r ∕TN r ) ≅ (TN r ) that satisfy r | N r−1 = 0 . Let be a fiberwise complex structure on TN r tamed by , and let be a nonnegative smooth function on M with zero set −1 (0) = N r−1 . For r = r , we have ( r , r ) = ( r , r ) ≥ 0 . Hence, ( r , r ) = 0 in view of (37). Using positivity, we deduce that r vanishes on N r ⧵ N r−1 . By continuity, we obtain r = 0 on all of N r . By repeating this procedure, we successively obtain that all components i of must vanish; hence, Ω is nondegenerate. be related to a dual pair [14]: They are obtained again by performing symplectic reduction on one leg of the dual pair.
implies the second part. ◻