Control on the Manifolds of Mappings with a View to the Deep Learning

Deep learning of the artificial neural networks (ANN) can be treated as a particular class of interpolation problems. The goal is to find a neural network whose input-output map approximates well the desired map on a finite or an infinite training set. Our idea consists of taking as an approximant the input-output map, which arises from a nonlinear continuous-time control system. In the limit such control system can be seen as a network with a continuum of layers, each one labelled by the time variable. The values of the controls at each instant of time are the parameters of the layer.


Introduction and Problem Setting
The name deep learning stands for a set of the methods and the tools which study the problems of classification such as image recognition and speech recognition.These methods involve multilayered artificial neural networks (ANN) and one of the key moments is the training of the networks on a set of classified objects.For a simple mathematical model of the multilayered ANN and of the process of its training we refer to [10].
The functioning of the ANN results from a composition of the actions of separate neurons.Each neuron realizes an activation function σ : R → R with parameters.There are plenty of choices for the activation function, that is normally nonlinear monotone sigmoid-like function.
The vector functions can be assembled from the scalar activation functions: σ : R m → R m : σ (x 1 , . . ., x m ) = (σ (x 1 ), . . ., σ (x m )). ( One can assemble neurons in a multi-layer network in such a way that the outputs of the neurons from a previous layer serve as the inputs for the successive level. One can introduce parameters into the activation functions via a substitution of their variables.For example a linear change of the argument in (1) results in σ (Kx + B), where The output of an ANN realizes the composition of the functions, each one of which corresponds to a layer: = σ K [M] σ K [M−1] . . .σ (K [1] x + B [1] ) . . .+ B [M−1] + B [M] .
To set the classification problem we consider a finite set of objects, which are represented by the vectors x i ∈ R d , i ∈ I. Let X = {x i |i ∈ I}.There is a R s -valued classifying function c : R d → R s , defined on X, which attributes to each object x i its "class" c(x i ) ∈ R s .
The objective of the training of an ANN amounts to the adjustment of the values of the parameters K [1] , . . ., K [M] , B [1] , . . ., B [M] in order to achieve the best approximation of the classifying function c(x) by the output map (2).More specifically one seeks to minimize the value of the loss function, which measures the discrepancy between the input-output map of the system and the classification function.For example the least square loss function has form C K [M] , . . ., K [1] , B [M] , . . ., B [1] = N i=1 c(x i ) − F (x i ) 2 2 → min K [j ] ,B [j ]   . ( Minimization of (3) results in a problem of nonlinear programming, which even for a "medium" number of layers can turn rather complex for classical approaches.
In this contribution we base on a continuous-time dynamic or residual network model for deep learning, with a continuum of layers, labelled by the time variable.The parameters involved at each layer are the values of the controls at the respective instant of time.The analogue of the composition (2) is the end-point of the trajectory or the output of the continuous-time control system in their dependence on control.
As in the model, we referred to above, in the control-theoretic setting one seeks for the values of the parameters (the controls), which provide the best approximation of the classifying function by the output of the control system.Precise formulations and the description of the model can be found in Section 3.
The setting allows for the application of analytic methods of dynamic optimization such as dynamic programming, Bellman's optimality principle and Pontryagin's maximum principle together with the corresponding numerical algorithms.This approach to the deep learning has been initiated in the last years by a number of scholars; see for example [8,11,12] and references therein.The readers must be warned that we consider a very restricted issue of possible application of the methods of ensemble controllability and ensemble optimal control to the problems of deep learning.Therefore, we only cite the references related to this concrete topic, leaving aside not only a huge amount of literature on deep learning, but also on application of the methods of deep learning to the problems of optimal control.
In the contribution we concentrate on finding the classes of control systems, which are able to guarantee approximation of the classifying functions at each rate.It amounts to studying the problems of ensemble controllability of the control systems and the action of the flows, generated by the control systems, on the manifold of mappings.We formulate the sufficient criteria (Theorem 5.1, Corollary 5.2) of ensemble controllability and provide examples of the nonlinear control systems which demonstrate approximate controllability property in the group of diffeomorphisms of R n (Theorem 6.3), of a torus T n (Theorem 6.6) and of the 2-dimensional sphere S (Theorems 6.7 and 6.10).

Neural Networks Modelled by Control Systems
It is an easy task to reformulate optimization problem ( 2)-( 3) as an optimal control problem for a discrete-time controlled dynamic system.If one sets the variables z 1 , z 2 , . . ., z M , which satisfy the relations then the map, defined by (2), coincides with the "end-point map" F (x, K [1] , . . .K [M] , B [1] , . . ., B [M] ) = z M . ( Alternatively one can introduce the intermediate variables y j and define the dynamics getting again formula (5) for the map (2).Denote z i j (respectively y i j , z i j ) the points of the trajectories of (4) (respectively ( 6)), which start with the initial data z i 1 = x i ∈ X, i ∈ I.Then, the problem of the best least square approximation (3) takes the form Ĉ K [2] , . . ., K [M] , B [2] , . . ., B [M] Problems ( 4), (7), respectively (6), (7) are Mayer problems of optimal control for the control systems with discrete time with free end-point.There are quite few numerical algorithms developed for this class of problems, but we do not treat them in this contribution, making emphasis instead on the continuous-time control systems. 1he way to the representation of the input-output map ( 5)-( 6) as an output of a continuous-time control system is rather straightforward.Let us consider a system, which for the sake of the computational simplicity we choose control-linear: For the purpose of our illustration we choose smooth vector field f 0 (z) to be nonlinear, and the vector fields f 1 (z), . . ., f r (z), to form a basis of the space of the affine vector fields in R m .Require the diffeomorphism e f 0 (z) to coincide with σ (z), so that the map σ (z) is generated by control system (8), driven by the constant control u(t) = (1, 0, . . ., 0) on a unit time interval.Each affine diffeomorphism Kx +B, with det K > 0, can be represented as a composition of the diffeomorphisms e a (z) , where a(z) are affine vector fields in R m .Hence, such diffeomorphisms are generated by the control system (8), driven by the piecewise-constant controls.
Therefore the composition (2) or, the same the output map (5) of the discrete-time system (6) can be represented as the end-point map of the continuous-time system (8), driven by a piecewise-constant control.

Ensemble Optimal Control Model
We consider a training set X = {x 1 , . . ., x N } ⊂ M, consisting of N points of a connected Riemannian manifold M. In what follows M will be a submanifold of R d .
We set an optimal control model for the training process of an ANN, which involves a control system in R d , which We introduce the terminology of the ensembles of points.A finite ensemble of points of a smooth manifold M is an N -ple γ = (x 1 , . . ., x N ) ∈ M N , whose components x j ∈ M are pairwise distinct: i = j ⇒ x i = x j .Thus if N ⊂ M N stands for the set of N -ples (x 1 , . . ., x N ) ∈ M N with (at least) two coinciding components, then the space E N (M) of the ensembles of N points of M is the complement of N : E N (M) = M N \ N = M (N) .Note that whenever dim M > 1, M (N) is an open connected subset and a submanifold of M N .
Introduce a classifying map c : X → C, where C is a connected Riemannian manifold.
Our goal is to approximate the map c by an action of the flow P t , generated by the control system (9) which is driven by a control u(t) = (u 1 (t), . . ., u r (t)).The flow P t acts on an ensemble (x 1 , . . ., x N ) as where z k (t) are the points of the trajectories of the Cauchy problems We introduce an output map p : M → C, which is a submersion in the cases we consider.We fix T > 0 and seek to minimize under constraints (10)- (11).
The infimum of ( 12) is either positive or null.The distinction is related to the presence or the lack of controllability of system (9) in the space of finite ensembles of points.The problems of controllability have been addressed in [4], where we arranged examples of the systems, which are controllable in the space of finite ensembles of points.We proved that for arbitrary N generic r-ples of vector fields f 1 (z), . . ., f r (z) ∈ Vect R d manifest this property.
Note that even for ensemble controllable systems, the greater is N , more complex are the controls u 1 (t), . . ., u r (t), which are needed to achieve controllability.
For this reason we opt for a tradeoff between the rate or quality of the approximation (minimization of (12)) and the complexity of the needed control, introducing the loss functional Problem ( 10)-( 11)-( 13) is Bolza optimal control problem with free end-point.In what regards study of the optimal control problem we limit ourselves to the formulation (in the following subsection) of the first-order optimality condition for the problem.In the rest of the contribution we concentrate on the problems of ensemble controllability.

Equations of Pontryagin Maximum Principle for Ensemble Optimal Control Problem (10)-(11)-(13)
We introduce the pre-Hamiltonian for (10), ( 13) where The adjoint equations of the corresponding pre-Hamiltonian system are The end-point conditions for the adjoint variables are According to the Pontryagin's Maximum Principle if ũ(t), z(t) are the optimal control and the corresponding optimal trajectory of the problem, then there must exist β ≥ 0 and an adjoint covector ψ(t), which satisfy the (15) and ( 16) and such that By the maximality condition we get ∂H ∂u i | ( ũ(t),z(t)) = 0, i = 1, . . .r, which in the normal (β > 0) case implies: Substituting expressions (17) into pre-Hamiltonian ( 14) we obtain the maximized (with respect to u) Hamiltonian As we already said minimization of the Bolza functional (13) on the set of trajectories ( 10)-( 11) is an interesting example of a sequence of sub-Riemannian Bolza problems depending on the cardinality N of the ensemble (the training set).The complexity of optimal controls for these problems is known to grow with the cardinality.It is rather unrealistic to expect a possibility to integrate Hamiltonian equations of the Maximum Principle ( 10)-( 11)-( 14)-( 15)-( 16)-( 17) for a nonlinear control system (10), which is able to solve the interpolation or ensemble controllability problem for each N .In such cases the use of the direct numeric methods for the solution of the variational problem ( 10)-( 11)-( 13) looks more realistic, and also for the numeric methods Hamiltonian ( 14), (17) plays important role.We plan to advance in this direction in our future research.In what follows we study interpolation, or ensemble controllability problems, or more generally approximate controllability in the spaces of mappings.

Finite Ensemble Controllability via Lie Algebraic Methods
We approach ensemble controllability from the viewpoint of geometric control theory, in the spirit of what has been done in our previous publication [4].See also preprint [12] where the Lie algebraic methods are applied to a different class of systems in the context of deep learning.
We start with basic definitions.
Definition 4.1 (finite ensemble controllability) System (9) has the property of finite ensemble controllability if for each N = 1, 2, . .., for each T > 0 and for any two N -ples ) there exists a control u(t) = (u 1 (t), . . ., u r (t)) which steers the corresponding system (10) from x α to x ω in time T .Remark 4.1 If system (10) can steer the point x α to x ω in time T > 0 by means of a control u(t), t ∈ [0, T ], then it can do the same in any time T > 0 by means of the control For a smooth vector field X ∈ VectM consider its N -fold -the vector field on M (N) , defined as X N (x 1 , . . ., x N ) = (X(x 1 ), . . ., X(x N )).System (10) can be given form γ = X N (γ ), γ = (x 1 , . . ., x N ) ∈ M (N) .
For X, Y ∈ Vect M, and N ≥ 1 we define the Lie bracket of the N -folds The same holds for the iterated Lie brackets.We denote Lie{f 1 , . . ., f r } the Lie algebra generated by the vector fields f 1 , . . ., f r , and Lie{f N 1 , . . ., f N r } the Lie algebra generated by their N -folds.For the vector fields f 1 , . . .f r on M, their N -folds f N 1 , . . ., f N r are called bracket generating on M (N) , if the evaluations of the iterated Lie brackets of f N 1 , . . ., f N r at each γ ∈ M (N) , span the tangent space T γ M (N) = N j =1 T x j M. Evidently for N > 1 the bracket generating property for f N 1 , . . ., f N r on M (N) is strictly stronger, than the same property for f 1 , . . ., f r on M.
Rashevsky-Chow theorem [2] implies Proposition 4.2 If dim M > 1 and ∀N ≥ 1 the N -folds f N 1 , . . ., f N s are bracket generating on M (N) , then system (9) has the property of finite ensemble controllability on M.
In [4] we proved that the latter property holds for each N and a generic r-ple f 1 , . . ., f r of vector fields.In the present context it is more convenient to check a stronger property, which implies the bracket generating property for any N .
Let us introduce the standard notation for the seminorms in the space of smooth vector fields on a manifold M: for a compact K ⊂ M and r ≥ 0 In the formulations of controllability results we invoke the following assumptions for the vector fields f 1 , . . ., f r ∈ Vect(M), which define control system (9).
Assumption 1 (boundedness) The vector fields f j (x), j = 1, . . ., r, are C ∞ -smooth and bounded on M together with their covariant derivatives of each order.Assumption 2 (Lie algebra approximating property) A system of smooth vector fields f 1 , . . ., f r ∈ Vect(M) demonstrates the Lie algebra approximating property, if ∃m ≥ 1 such that for each C m -smooth vector field Y ∈ Vect(M) and each compact K ⊂ M there holds: We show that this property suffices to guarantee finite ensemble controllability.
Theorem 4.3 (Lie algebra approximating property and finite ensemble controllability) If dim M > 1 and the vector fields f 1 , . . ., f r meet Assumptions 1 and 2, then ∀N ≥ 1 system (10) is controllable in the space E N (M) of ensembles of N points.
Proof Fix N .Choose an ensemble γ = (x 1 , . . . ,x N ) ∈ M (N) .We prove that the N -folds f N 1 , . . ., f N r are bracket generating at γ .Pick m for which the Lie algebra approximating property holds.Consider the space Vect m (M) of C m -smooth vector fields on M and define for each γ ∈ M (N) the evaluation map This linear map is obviously surjective and continuous with respect to C 0 -metric in Vect m (M).By virtue of Assumption 2 the image E γ (Lie{f 1 , . . ., f r }) is a dense linear subspace of T γ M (N) and hence must coincide with it.
Remark 4.2 Below we provide formulations for specific cases in which dim M = 1.

Lie Algebra Strong Approximating Property: Controllability in the Diffeomorphism Groups and the Manifolds of Mappings
In the previous section we dealt with finite ensembles of points.In this section we show that if a stronger approximating property holds for the Lie algebra Lie{f 1 , . . ., f r }, associated to control system (9), then approximate controllability of system (9) holds in the group Diff c 0 of diffeomorphisms on M and on the manifolds of smooth mappings of M.
In our proofs we make occasional use of few notations of chronological calculus for the flows generated by the time-dependent vector fields [1].In particular for a vector field X t (x), which is smooth in x and locally integrable in t we denote by −→ exp t t 0 X s ds the flow P t , generated by the time-dependent differential equation ẋ = X t (x), P t 0 = I .If X t is time independent: X t (x) ≡ X(x), then the flow is denoted by P t = e (t−t 0 )X .A brief presentation of the chronological calculus can be found in [2].
The following definition has been used in [4].Put for > 0 and a compact K ⊂ M: Assumption 3 (Lie algebra strong approximating property) A system of smooth vector fields f 1 , . . ., f r ∈ Vect(M) possesses Lie algebra strong approximating property, if ∃m ≥ 1, such that for each C m -smooth vector field Y ∈ Vect(M) and each compact K ⊂ M ∃ > 0 for which: Denote by Diff c 0 the connected component of the identity of the group of the compactly supported diffeomorphisms of M. Theorem 5.1 (C 0 -approximate controllability in the group of diffeomorphisms) Let P ∈ Diff c 0 (M).Let C ∞ -smooth vector fields f j (x), j = 1, . . ., r, meet Assumptions 1 and 3. Then for each K ⊂ M and each ε > 0 there exists a control u(t) = (u 1 (t), . . ., u r (t)), t ∈ [0, T ], such that for the corresponding flow generated by system (9), the diffeomorphism P T ε-approximates P in C 0 on K: Proof Join the identity I with P by a curve t → Pt (x), t ∈ [0, T ] in Diff c 0 (M).Without loss of generality we may assume that (t, x) → Pt (x) is C 1 -smooth.The curve t → Pt (x) can be represented as a flow Pt = −→ exp t 0 Y τ dτ , generated by a non autonomous vector field Y t , which is continuous in t; one can take Y t (x) = (P t ) −1 * dP t dt (x).Denote by K t , t ∈ [0, T ] the images of a compact set K under the flow Pt .As far as for each t ∈ [0, T ] condition (18 holds for the vector fields Y t and control system (9), then one can apply Theorem 4.3 of [4] to the vector field Y t , the diffeotopy K t , t ∈ [0, T ] and system (9).According to this Theorem for each ε > 0 there exists a control u(t) = (u 1 (t), . . ., u r (t)), t ∈ [0, T ] such that for the flow (19) The approximation result, we have just proved, can be extended from diffeomorphisms of M to a broader class of continuous maps ϕ : M → C.
One of possible constructions can be realized on the manifold M × C. Consider the projection p : M × C → C and a diffeomorphic immersion ı : M → M × C. We opt for ı(x) = (x, ν), ∀x ∈ M, where ν is a selected point of C. Let the metric d on M × C be defined by Let ϕ : M → C be a continuous mapping which is approximately C 1 -smoothly homotopic to the constant mapping ϕ 0 (x) = ν.This means that in any C 0 -neighborhood of ϕ there are C 1 -smooth functions φ, which are contractible to the constant function by C 1 -smooth homotopies φt (x), t ∈ [0, 1]: Without loss of generality we can limit ourselves to the case in which ϕ = φ is C 1smooth and C 1 -smoothly homotopic to the constant function.Consider the graphs of the mappings ϕ t (x) : Γ t = {(x, ϕ t (x)), x ∈ M} ⊂ M × C. For each t the sets Γ t are diffeomorphic to Γ 0 and to M. The flow Pt , generated on the manifold M×C by the vector field ∂ϕ t (x) ∂t ∂ ∂c , defines the diffeotopy of the graphs: Let control system (9), defined now on M × C, possess the Lie algebra strong approximating property.By the previous theorem for each compact K ⊂ M and each ε > 0 there exists a control u(•) = (u 1 (•), . . ., u r (•)), such that for the flow and we conclude with the corollary.
Corollary 5.2 Let control system (9), defined on M × C, meet Assumptions 1 and 3. Then the system is C 0 -approximately controllable on the manifold of mappings: for each continuous mapping ϕ : M → C, which is approximately smoothly homotopic to a constant, each ε > 0 and each compact K ⊂ M there exists u(t) = (u 1 (t), . . ., u r (t)), t ∈ [0, T ], such that for the corresponding flow (20) on M × C there holds ϕ(x) − p • P T • ı(x) 0,K < ε.
Recent publication [6] contains an interesting example of five polynomial vector fields X 1 , . . ., X 5 in R n , which generate the Lie algebra of all the polynomial vector fields in R n and as a consequence guarantee in the terminology of [6] the "universal interpolation property".As far as polynomial vector fields C m -approximate each C m -smooth vector field X 1 , . . ., X 5 possess Lie algebra strong approximating property, i.e meet Assumption 3 (and hence Assumption 2).Then by Theorem 4.3, the control system ẋ = X 1 u 1 (t) + • • • + X 5 u 5 (t) possesses the property of ensemble controllability for finite ensembles of points (equivalent to the "universal interpolation property"), and moreover by Theorem 5.1 possesses the property of C 0 approximate controllability in the group of diffeomorphisms.
An interesting algebraic question consists of finding a minimal collection of the polynomial vector fields in R n , which would meet Assumption 3 or more particularly generate a Lie algebra of all the polynomial vector fields.From our treatment of the case of a 2D sphere, accomplished in the next section, one gets an idea that the representation theory for GL(n) can be instrumental for a construction of such minimal collection.

Ensemble Controllable Systems on Euclidean Spaces R d , tori T d and the 2-dimensional Sphere S
In this section we consider several manifolds, such as Euclidean spaces R d , d-dimensional tori T d and 2-dimensional sphere S. We provide examples of control systems on the manifolds, which possess controllability properties for finite ensembles and properties of approximate controllability in the group of diffeomorphisms of the manifolds.
For the sake of brevity along the Section we will call system ensemble controllable if the conclusions of Theorems 4.3 and 5.1 hold for it.
The key point of the proofs is the verification of the Lie algebra strong approximating condition.Such a verification regards two moments.First we have to establish kind of "Lie rank condition" -the approximability of the vector fields by the vector fields from Lie{f 1 , . . ., f r }.The second issue is the regularity of these approximations, including boundedness of the derivatives of the approximants.

Ensemble Controllable System in R d
Consider control-linear system in R d : where and We call it GH system as far as Gaussian density function e −γ (z) and Hermite polynomials play important role in its study.We consider the action of system (23) onto an ensemble of points (x 1 , . . ., x N ) ∈ R d N .To establish the property of ensemble controllability we verify the Lie algebra strong approximation condition for GH system.
As one knows where Any function from L 2 (R d ) can be expanded into a L 2 -convergent series in Hermite polynomials.To verify the Lie algebra strong approximating condition one has to prove that for each sufficiently smooth vector field Y (X) = d j =1 Y j (z) ∂ ∂z j with compact support in R d , there exists > 0 such that for each j = 1, . . ., d and each ε > 0 one can find a linear combination X j of the functions (24) for which Pick its component Y j (x) and consider the orthogonal expansion of the function Y j (z)e γ (z) in Hermite polynomials: |m|≤n c m H m be a partial sum of this expansion.Lie algebra strong approximating condition is implied by the following Lemma.
∂z i and hence are bounded by a constant independent of n.
Proof of the lemma can be found in the Appendix.By virtue of Theorems 4.3 and 5.1 and Proposition 6.1 there holds and only if they are equally ordered:
Proof Boundedness is obvious.
We prove that the Lie algebra Lie{f 0 , f 1 , f 2 }, generated by vector fields (27), contains the vector fields sin kϕ ∂ ∂ϕ , Consider a vector field Y (ϕ) ∂ ∂ϕ on T 1 together with its Fourier expansion By the aforesaid partial sums of the series belong to Lie{f 0 , f 1 , f 2 }.For Y (ϕ) being C 2smooth the partial sums S n (ϕ) of the Fourier series converge uniformly to Y (ϕ), as n → ∞.
The derivatives S n (ϕ) converge uniformly to Y (ϕ) and hence are equibounded, wherefrom the Lie algebra strong approximating condition follows.
To extend the construction to the d-dimensional torus we introduce the coordinates ϕ 1 , . . ., ϕ d in T d and define the vector fields Consider the control-linear system Lemma 6. 5 The Lie algebra L T d generated by vector fields (28) contains all the monomial vector fields of the form where I ∪ I c = {1, . . ., d}, I ∩ I c = ∅.
Proof By the previous lemma the monomial vector fields cos k i ϕ i ∂ ∂ϕ i , sin k i ϕ i ∂ ∂ϕ i , i = 1, . . ., d, belong to the Lie algebra.So do the vector fields and by induction in l we conclude that all the monomial vector fields cos lϕ j ∂ ∂ϕ i , sin lϕ j ∂ ∂ϕ i , i, j = 1, . . . ,d, belong to L T d .We define the degree of a monomial vector field (30) as the cardinality of the set {i ∈ {1, . . ., d}|k i = 0} and proceed by induction in the degree.Each monomial vector field of degree s + 1 is either M(ϕ) cos k α ϕ α ∂ ∂ϕ j or M(ϕ) sin k α ϕ α ∂ ∂ϕ j , where M(ϕ) has degree s and does not depend on ϕ α .
In the first case if M(ϕ) does not depend on ϕ j , and hence If M(ϕ) depends on ϕ j , then α = j and one can easily find a monomial M 1 (ϕ) of degree In this way we conclude the step of induction and the proof.
The Lie algebra strong approximating property for (29) follows from the lemma by classical approximation results for multivariate trigonometric polynomials.
In what regards the formulation of criteria of finite ensemble controllability there is some peculiarity in the case of T 1 .Note that for a given orientation of T 1 any ensemble of N points on T 1 is ordered up to cyclic permutation.Two ensembles are equally ordered if the sequences of their indices are the same up to a cyclic permutation.Theorem 6. 6 Control system (26) and (29) have the following ensemble controllability properties: i) for d > 1 system (29) is ensemble controllable on T d ; ii) for M = T 1 system (26) is C 0 -approximately controllable in Diff 0 (T 1 ); iii) two finite ensembles on T 1 can be steered one into another by means of control system (26) in time T > 0, if and only if they are equally ordered.

Ensemble Controllability on the 2-dimensional Sphere
We construct examples of control systems on the 2-dimensional sphere S ⊂ R 3 , which demonstrate the property of ensemble controllability.Both examples are related to the study in [3] of the controllability of the Navier-Stokes equation on S.
We consider a Riemannian structure on S, induced by the Euclidean structure of R 3 ⊃ S. If f : S → R is the restriction onto S of a smooth function F : R 3 → R, then the spherical gradient is the projection of the gradient ∇F onto the tangent bundle T S to S.Here Ex stands for the Euler vector field in R 3 : Ex = 3 i=1 x i ∂ i .In general if X is a smooth vector field in R 3 , then the projection onto T S of the restriction of X to S is pr S X(x) = X(x) − X(x), x E(x), x ∈ S, which will be a smooth vector field on S.
Consider standard symplectic structure σ x (•, •) on S ⊂ R 3 defined by the area form.For x ∈ S, ξ, η ∈ T x S one has σ x (ξ, η) = x, ξ, η , where the latter trilinear form is the mixed product in R 3 .
We introduce the spherical divergence div S pr S X(x) of pr S X(x) with respect to the area form σ .To this end we consider the interior product of the vector field pr S X(x) with the differential 2-form σ ; it is the 1-form defined by where × stands for the cross product in R 3 .The exterior derivative of the 1-form is the 2form ψ(x)σ , whose coefficient ψ(x) coincides with the spherical divergence div S pr S X(x).
To compute the exterior derivative we apply Stokes theorem to the integral of the 1-form (31) along a closed curve on S and conclude that it equals to the flow of the curl of the vector field x × X(x) through the spherical area circumvented by the curve.Hence In particular div S pr S X(x) = div X, if X is tangent to S.
Once we have defined spherical divergence div S and spherical gradient ∇ S , then spherical Laplacian of a function f on S is defined as: Consider the homogeneous harmonic polynomials on R 3 \ 0 and take their restrictions onto S; those are called spherical harmonics.We call them linear, quadratic, cubic, of nth degree etc., if they are restrictions of the homogeneous polynomials of the corresponding degree.Spherical harmonics are the eigenfunctions of the spherical Laplacian.
Restriction of any smooth function ϕ in R 3 onto S gives rise to the Hamiltonian vector field − → ϕ on S, which is defined by the relation: We provide an example of Hamiltonian control system which has the property of approximate controllability in the group of the area-preserving diffeomorphisms on S. Theorem 6.7 Given three independent linear harmonics 1 , x), (l 2 , x), (l 3 , x), a quadratic harmonic q(x), a cubic harmonic c(x) and the corresponding Hamiltonian vector fields is controllable in the space of finite ensembles on S and approximately controllable in the group SDiff 0 (S) of the area-preserving diffeomorphisms of S.
Proof The following statement has been proved in [3,Theorem 10.4].

Proposition 6.8
The Lie algebra generated by the Hamiltonian vector fields (32) contains all the symplectic vector fields − → h , which correspond to harmonic homogeneous polynomials (spherical harmonics) h(x), and therefore is dense in the space of all the divergence-free vector fields.
Spherical harmonics form a complete system in L 2 (S).To prove the Lie algebra strong approximating condition we consider the expansions of functions on S in Laplace series with respect to spherical harmonics.We apply a result by M.Ganesh, I.G.Graham & J.Sivaloganathan [9, Theorem 3.5] on the best approximation by Laplace series of smooth functions on the spheres S m together with their derivatives up to some order.Lemma 6.9 Let C(S) be the space of continuous functions on the sphere and P n be the space of spherical polynomials of degree ≤ n.For each n ≥ 1 there exist continuous linear operator T n : C(S) → P n and for every l ≥ 0 a constant b l such that for all k = 0, . . ., l; f ∈ C l (S) (This result builds on the previous work by D.L.Ragozin and D.J.Newman & H.S. Shapiro; see references in [9]).
Let Y (x) be a C 2 -smooth divergence-free (Hamiltonian) vector field on S and Υ the corresponding C 3 -smooth Hamiltonian.By Lemma 6.9 for some constant b 2 > 0. This implies that T n Υ and its first and second derivatives DT n Υ, D 2 T n Υ converge uniformly to Υ, DΥ, D 2 Υ correspondingly as n → ∞.This means that the Hamiltonian vector fields − − → T n Υ converge uniformly to Y , and their derivatives D − − → T n Υ converge uniformly to DY as n → ∞.Hence the derivatives D − − → T n Υ are equibounded, and the vector fields − − → T n Υ are equilipschitzian.According to Proposition 6.8 the vector fields − − → T n Υ belong to the Lie algebra generated by the vector fields (32) and hence the Lie algebra strong approximating condition holds for control system (33).degree 3 belong to L. The proof can be completed by induction in the degree of harmonics with Corollary 6.16 applied at each induction step.

Proposition 6 . 1
Vector fields (22) meet Assumptions 1 and 3. Proof Direct computation of the iterated Lie brackets of vector fields (22) gives ad m j 1