Design and performance analysis of deterministic learning of sampled-data nonlinear systems

In this paper, we extend the deterministic learning theory to sampled-data nonlinear systems. Based on the Euler approximate model, the adaptive neural network identifier with a normalized learning algorithm is proposed. It is proven that by properly setting the sampling period, the overall system can be guaranteed to be stable and partial neural network weights can exponentially converge to their optimal values under the satisfaction of the partial persistent excitation (PE) condition. Consequently, locally accurate learning of the nonlinear dynamics can be achieved, and the knowledge can be represented by using constant-weight neural networks. Furthermore, we present a performance analysis for the learning algorithm by developing explicit bounds on the learning rate and accuracy. Several factors that influence learning, including the PE level, the learning gain, and the sampling period, are investigated. Simulation studies are included to demonstrate the effectiveness of the approach.


Introduction
Neural network (NN), due to its approximation properties as well as the inherent adaptation features, has been widely used as a black-box approximation tool for identification and control of a broad category of complex nonlinear systems [1][2][3][4].Different from the the area of NN-based control where NNs are only functioning in achieving accurate state estimation to pursue the control goal, NN-based system identification further requires the NN weights to converge to their correct values so as to capture the best approximation of the underlying system dynamics [2].To achieve this objective, the persistent excitation (PE) condition is normally required to be satisfied [5][6][7].Nevertheless, the PE condition is very difficult to be characterized and usually cannot be verified a priori, especially for identification of nonlinear systems [6].During the past decade, continuous efforts have been seen on seeking the conditions for fulfilling the satisfaction of the PE condition in system identification using NNs [6,8,9].However, the proposed conditions usually turn out to be too restrictive for practical implementations.
To overcome the difficulties concerning the PE condition in NN identification, Ref. [10] proposed a deterministic learning mechanism for identification of a class of nonlinear systems using radial basis function networks (RBFNs).It was shown that a regressor vector consisted of radial basis functions can satisfy the PE condition.Specifically, for RBFNs constructed on a regular lattice, any recurrent trajectory that stays within the regular lattice can lead to the satisfaction of a partial PE condition.This partial PE condition leads to exponential stability of the identification error system.Consequently, accurate NN identification of the nonlinear dynamics (including closed-loop dynamics) is achieved within a local region along the recurrent trajectory generated by the identified nonlinear system.This approach is referred to as "deterministic learning" since it is developed not by using statistical principles (e.g.[11]), but by utilizing deterministic algorithms from adaptive control (e.g.[12,13]).The deterministic learning approach provides an effective solution to the problem of learning in dynamic environments, and is useful in many applications (see [14][15][16]).
For NN-based system identification, while most of the development is done in continuous time due to the simplicity of deriving adaptation schemes, discrete time NNs are more convenient for real applications, and some significant results are available for the system identification in discrete-time (e.g., [1,4,17,18]).For the sake of dealing with vast amount of data, Ref. [19] has extended the deterministic learning theory to modeling and control of discrete-time nonlinear systems.A framework of data-based modeling, rapid dynamical pattern recognition, and pattern-based intelligent control has been proposed for a class of discrete-time nonlinear systems.However, considering that most of the discrete-time data are collected from a sample device connecting to the physical plant (i.e., the continuous-time nonlinear system) in practice, it is desired to take sampling into account during the design process and further analyze its influence upon the learning performance of NNs.
In this paper, we investigate deterministic learning of sampled-data nonlinear systems.Firstly, based on the deterministic learning theory and the methodology of adaptive identification of discrete-time systems [4,20], we consider the Euler approximation model of the continuous-time system as a method for taking the sampling parameters into account and propose a normalized learning algorithm for the RBFNs.It is proven by introducing a state transformation and using Lyapunov stability method that the satisfaction of the PE condition (which is guaranteed by the periodic or recurrent sequence [21]) leads to the exponential stability of the identification error system.Consequently, the internal nonlinear dynamics of the sampled-data system can be accurately learned along the state sequence, and the knowledge can be represented by using a constant-weight neural network.The learning algorithm proposed in this paper is more usable for practical applications than that in [19] for discrete-time systems, and the Lyapunov-based proving method is much more compact and concise than the UCO-based method (i.e., a method based on the uniform complete observability technique) in [19,22].Furthermore, it is subsequently shown that this algorithm also facilitates the analysis of learning performance, which is important for both theoretical and practical reasons.Secondly, we present a performance analysis for the learning algorithm by developing explicit bounds on the learning rate and accuracy.Several factors that influence learning, such as the level of PE, the learning gain, and the sampling period, are investigated.
The rest of the paper is organized as follows: Preliminary results and problem formulation are contained in Section 2. Section 3 proposes the approach of deterministic learning of sampled-data systems via Euler approximations.The analysis of learning performance is presented in Section 4. Numerical simulations to illustrate the results are given in Section 5. Section 6 concludes the paper.

Preliminaries
The RBF networks can be described by [23], where Z ∈ Ω Z ⊂ R q is the input vector, W = [w 1 , . . ., w N ] T ∈ R N is the weight vector, N is the NN node number, and with s i (•) being a radial basis function, and ξ i (i = 1, . . ., N) being distinct points in state space.The Gaussian function ] is one of the most commonly used radial basis functions, where ξ i = [ξ i1 , ξ i2 , . . ., ξ iq ] T is the center of the receptive field and η i is the width of the receptive field.The Gaussian function belongs to the class of RBFs in the sense that s i ( Z − ξ i ) → 0 as Z → ∞.
It has been shown in [24] that for any continuous function f (Z) : Ω Z → R where Ω Z ⊂ R q is a compact set, and for the NN approximator, where the node number N is sufficiently large, there exists an ideal constant weight vector W * , such that for each * > 0, f (Z) = W * T S(Z) + , ∀Z ∈ Ω Z , where | | < * is the approximation error.Moreover, for any bounded trajectory Z(t) within the compact set Ω Z , f (Z) can be approximated by using a limited number of neurons located in a local region along the trajectory: , where subscript (•) ζ stands for the regions close to the trajectory  [6,8,9], it is shown in [21] that for a localized RBF network W T S(Z) whose centers are placed on a regular lattice, almost any recurrent trajectory Z(t) can lead to the satisfaction of the PE condition of the regressor subvector S ζ (Z).
A broadened version of the definition of persistency of excitation, an one that encompasses the discrete and continuous cases, is provided by [8].Definition 1.Let μ be a positive, -finite Borel measure on [0, ∞).We will say that a continuous, vector-valued function w : [0, ∞) → R N is persistently exciting if there exist positive constants δ, α 1 , and α 2 such that holds for every t 0 0 and every constant column The following definition of PE in discrete form provided by [25] is utilized throughout this paper.
Definition 2. A sequence x(t) ∈ R n is said to be persistently exciting (in N steps), if there exists uniformly in t 0 .

Problem formulation
Consider a general nonlinear dynamical system in the following form: where X = [x 1 , . . ., x n ] T ∈ R n is the state of the system, which is measurable, p is a system parameter vector, F (X; p) = [f 1 (X; p), . . ., f n (X; p)] T is a smooth but unknown nonlinear vector field.Assume that the state X remains uniformly bounded, i.e., X(t) ∈ Ω ⊂ R n , ∀t t 0 , where Ω is a compact set.Moreover, the system trajectory starting from x 0 , denoted by ϕ * ζ (X 0 ), is a recurrent trajectory 1) .Assume that system (3) is between a sampler and a zero-order hold.To achieve the locally accurate identification/learning of the unknown nonlinear dynamics F (X; p) by measuring the temporal sequences sampled from the recurrent continuous-time trajectory ϕ * ζ (X 0 ), we consider the Euler approximation for the sampled-data presentation of (3), which can be expressed as The sampling period T s is assumed to be a design parameter.Note that the Euler approximation model ( 4) is parameterized by the sampling period T s .More importantly, the unknown internal dynamics 1) Roughly, a recurrent trajectory is characterized as follows: given ν > 0, there exists a finite T (ν) > 0 such that the trajectory returns to the ν-neighborhood of any point on the trajectory within a time not greater than T (ν).(Please refer to [26] for a rigorous definition of recurrent trajectory.)F (X; p), which is the learning target, is preserved.Furthermore, the discrete-time state X(k) of ( 4) is obtained by sampling from X(t) of (3) at each sampling moment.The following assumption is given regarding the data sequence X(k).Assumption 1.The state X(k) remains uniformly bounded, i.e., X(k) ∈ Ω ⊂ R n , ∀k 0, where Ω is a compact set.Moreover, the state sequence starting from X 0 , denoted by ϕ ζ (X 0 ), is a recurrent sequence.
In this paper, our objectives are to design an adaptive NN identifier based on the Euler approximate model ( 4) to learn the unknown internal dynamics F (X; p) of the continuous-time system (3), thereby looking into several factors that influence learning, including the PE level, the sampling period T s , and the system design parameters.

Adaptive NN identifier design
To show the basic idea, we consider one subsystem of system (4) and neglect the subscript as ( The NN identifier used to learn f (X(k); p) of the given system is described by the difference equation where 0 < |a| < 1 is a design parameter.
To adjust the NN weights Ŵ , the adaptive law with normalization is designed as where 0 < α < 2 is the learning gain for design, P = P T > 0, and λ max (P ) denotes the largest eigenvalue of the matrix P .
Remark 1.Note that the learning algorithm (7) avoids constructing an a priori output which is typically employed in classical adaptive identification and control algorithms, e.g., [22,27].Furthermore, it will be shown in the following that this new learning algorithm directly facilitates the exponential stability analysis of the overall system as well as the analysis of learning performance.Define the following error vectors: where e is the state tracking error, Ŵ is the estimate of the ideal weight W * .It follows from ( 6) and ( 5) that e(k) satisfies where = f (X; p) − W * T S(X) is the NN approximation error.The parameter error equation can be written as The state error equation ( 9) and the parameter error equation (10) together constitute the error model of the overall system, which can be considered as a discrete perturbed LTV system.

Exponential convergence under the PE condition
Consider the identification error system consisting of ( 9) and ( 10).An assumption regarding S(X(k)) is given as follows.
Assumption 2. There exists a constant S M > 0 such that for all k 0, the following bound is satisfied Remark 2.
In [8], inequality (11) has been proved to be satisfied for RBF networks where the RBF is with Gaussian functions.Assumption 2 is given for the convenience of presentation.
The following theorem presents the learning ability of NN identifier ( 6) with learning law (7).
Theorem 1.Consider the close-loop learning system consisting of sampled-data nonlinear system (4), NN identifier ( 6) and learning law (7) with Assumptions 1 and 2. For almost any recurrent sequence ϕ ζ (X 0 ) starting from an initial condition X(0) = X 0 ∈ Ω, and with initial values Ŵ (0) = 0.If the sampling period T s satisfies 0 < T s < 2 α , we have: (i) the state estimation error e(k) (defined as ( 9)) converges exponentially to a small neighborhood of zero; (ii) for the neurons near the sequence ϕ ζ (X 0 ), the neural weight estimate Ŵζ converges exponentially to a small neighborhood of its optimal value W * ζ , and a locally-accurate approximation for the unknown internal dynamic f (X; p) of system (3) along the sequence ϕ ζ (X 0 ) is obtained by Ŵ T S(ϕ ζ ), as well as by W T S(ϕ ζ ), where with {k a , . . ., k b } representing a time segment after the transient process.
Proof.For detailed proof, please refer to Appendix A.
Remark 3. The former half of proof above follows along similar lines as the proof of [28].However, under the satisfaction of the PE condition, the latter half in establishing exponential stability appears to be much simpler than its counterpart in [28], since it is required in [28] that the identified plant should be completely controllable for linear systems or the form of the nonlinearity should be known for nonlinear systems and the input signals should be sufficiently general.Moreover, our proof is considerably more compact and concise than those which rely on showing the equivalence of PE and uniform complete observability (UCO) and the invariance of the UCO under output injection.For the sake of comparison see, for instance, Ref. [13] for continuous-time systems and Refs.[19,22,29] for discrete-time systems.
Remark 4. Note that 0 < T s < 2 α is a prerequisite for Theorem 1.Nevertheless, since α is designed to satisfy 0 < α < 2, the most rigorous requirement imposed on the sampling period T s is 0 < T s 1, which surely can be guaranteed in practical implementations.

Parameter convergence
We further investigates the convergence properties of the framework presented above, particularly the exponential convergence rate and accuracy of the NN weights.The following theorem provides a basis for the performance analysis of deterministic learning of sampled-data systems.
Theorem 2. Consider the close-loop learning system consisting of nonlinear dynamical system (4), NN identifier (6) and learning law (7) with Assumptions 1 and 2. For the neurons near the sequence ϕ ζ (X 0 ), the neural weight estimation error Wζ converges exponentially to the residual set with the convergence rate as where , and Proof.For detailed proof, please refer to Appendix B.

Performance analysis
Theorem 2 provides the explicit expressions of the convergence rate and residual bound of the NN weights, based on which, the performance of deterministic learning of sampled-data system, including the learning speed and learning accuracy 2) , can be revealed.In this subsection, we mainly focus on three parameters, i.e., the PE level α 1 , the learning gain α, and the sampling period T s .The PE level α 1 is related to NN input data, i.e., the recurrent sequence ϕ ζ (X 0 ), while α and T s are subject to control.

PE level α 1
Based on Theorem 2, we have the following corollary summarizing the relationship between the PE level and learning performance.
(ii) We now consider b in ( 13): 2) The learning accuracy defined in this paper is indicated by either the convergence bound b or the practical approximation errors by using the RBF networks, e.g. 1 of (A19) and ζ 1 of (A20).Intuitively, the learning accuracy is inverse proportional to b as well as 1 and ζ 1 .Calculating ∂b ∂α1 yields which implies that b is inversely proportional to α 1 .We further examine the manner of this monotone relationship between b and α 1 .Calculating the second derivative of b with respect to α 1 , we have The second part is proven by denoting α * 1 = 8 9Cpe2 and α * 1 = 1 Cpe2 .(iii) The proof of the third part is similar with that of Corollary 3 in [30].From the fact that ∀α = −∞, it can be concluded that by Corollary 3 of [30], there exist constants 0 ) is guaranteed.On the other hand, since Wζ b, and from (A19), as in [21], we have Since the neural weights Ŵζ will only be slightly updated, both Ŵζ and Ŵ T ζ Sζ(ϕ ζ ) will remain very small [21].Thus, the practical approximation error 1 in (A20) satisfies From (A21) and according to [21], we have a similar result for the practical approximation error by using W T S(ϕ ζ ) as 2 = O( * ), which ends the proof.18) and ( 20) indicate that b decreases from ∞ to g * as the PE level α 1 increases from 0 to α * 1 .It decreases sharply in the right neighborhood of zero and the left neighborhood of α * 1 , and decreases relatively slowly in the neighborhood of α * 1 .Note that this decreasing behavior for deterministic learning of discrete-time systems is different from that for continuous-time systems as discussed in [30].Remark 6.It is worth noting from ( 15) and ( 17) that a small α 1 may lead to a small λ and a large b.Actually, such phenomenon that a low PE level may bring about a reduction or loss in accurate convergence and robustness of learning has been deeply investigated in [30][31][32].
(i) There exists an optimum value α op such that the learning speed λ is maximized with respect to α, i.e., furthermore, the learning speed is proportional to α for α ∈ (0, α op ), and inversely proportional to α for α ∈ (α op , 2).
there exists an optimum value T sop such that the learning speed λ is maximized with respect to T s , i.e., where Ts is independent of T s , and the learning speed is proportional to T s for T s ∈ (0, T sop ), and inversely proportional to T s for T s ∈ (T sop , 2 α ).
1, the learning speed is proportional to T s for T s ∈ (0, 2 α ).Proof.(i) From ( 14), we have Evaluating the partial derivative of λ with respect to α yields Setting ∂λ ∂α = 0 and solving, the value of α * can be obtained Furthermore, since 0 < α < 2, and 2[T 0 (T 0 − 1)L 4 + T 0 L 2 ] > 0, it can be concluded that α * < 2, the optimum value of α is α op = α * , and from (26), ∂λ ∂α > 0 for α ∈ (0, α * ) and ∂λ ∂α < 0 for α ∈ (α * , 2).(ii) The proof of the second part follows the line of the first part.From ( 14), we have Setting ∂λ ∂Ts = 0 and solving, the value of T * s can be obtained Eq. ( 30), together with 0 < T s < 2 α , shows that if 2T 0 (T 0 − 1)C 2T 0 (T 0 − 1)( ) 2 (from Theorem 2) is typically larger than one.Thereby, the optimum value of T s can be obtained as which is a rather small number compared to 2 α .The relationship between the learning speed represented by the parameter convergence rate λ, and the design parameters α and T s can be expressed explicitly by Corollary 2, while the case for the learning accuracy indicated by b is not straightforward.Nevertheless, from the above discussion on α op and T sop , we see that their values are rather small in practice.Here we would like to examine the manner in which b depends on α and T s , respectively in the corresponding dominant intervals [α op , 2) and [T sop , 2 α ): • Consider the parameter α, we rewrite b from (13) to made the expression of b with respect to α explicit. where Evaluating the derivative of b with respect to α, we have By Corollary 2, for all α ∈ [α op , 2), the polynomial in ( 32) satisfies (2C α4 + C α3 )α 2 + 4α − 4 > 0, and in turn, ∀α ∈ [α op , 2), ∂b ∂α > 0. This, together with Corollary 2, indicates that both the learning speed and the learning accuracy are inversely proportional to α, for α ∈ [α op , 2).
• For the sampling period T s , rewrite b in (13) where C Ts1 = g * Ts is independent of T s .Calculating its derivative to T s , we have It implies that degradation of the learning performance, including learning speed and accuracy, follows the increase of T s for T s ∈ [T sop , 2 α ).

Simulation studies
In this section, the simulation result on a sampled-data model of the Rossler system [33] via Euler approximations is presented to show the design and performance of the NN identifier.The continuoustime model of Rossler system is where x = [x 1 , x 2 , x 3 ] T ∈ R 3 is the state vector available from measurement, p = [p 1 , p 2 , p 3 ] T is a constant vector of system parameters, the system dynamics ) are assumed mostly unknown to the identifier.Its Euler approximate model can be written in the form of system (4) as where T s is the sampling period set at 0.01.To show the basic idea of the methodology, we assume that the smooth function f 3 (x; p) is unknown, and an identifier will be designed to learn the nonlinear dynamic f 3 (x; p) along the recurrent trajectories generated by (36).According to ( 6) and ( 7), the NN identifier with adaptive learning law can be obtained as where e 3 (k) = x3 (k) − x 3 (k).The RBF network Ŵ T 3 S 3 (x) is constructed, containing 441 nodes with the centers ξ i evenly placed on [−7, 7]×[−7, 7], and the widths η i = 0.7, i = 1, 2, . . ., N. The design parameters are a = 0.5, α = 0.5, and P = I.The initial conditions of the overall system are [x 1 (0), x 2 (0), We fix the system parameter as p = [0.2,0.2, 2.5] T to generate a periodic sequence as seen in Figure 1(a).Figure 1(b) displays the NN input sequence, i.e., state sequences of x 1 (k) and x 3 (k).The parameter convergence is shown in Figure 2(a).Note that only part of neural weights converge, while many other neural weights remain to be zero or to be small values.The NN approximations of f 3 (x; p) in Figure 2(b) are used to show the learning accuracy.It is seen that good NN approximation of the unknown

Conclusion
In the paper, the deterministic learning theory has been extended to sampled-data nonlinear system.Firstly, based on the Euler approximate model, the adaptive NN identifier with a normalized learning algorithm for RBFNs has been designed.It has been shown that by properly setting the sampling period, the overall system can be made stable and partial NN weights can exponentially converge to their optimal values.It is shown that the normalized learning algorithm not only avoids the usage of a posteriori signal that is frequently seen from the previous work, but also facilitates the stability analysis of the overall system and the analysis of learning performance.Secondly, a performance analysis for the learning algorithm has been presented by developing explicit bounds on the learning rate and accuracy.The effects of factors including the level of PE, the learning gain, and the sampling period, upon learning performance have been revealed, which is of great importance in providing guidance for practical design.
Future research of this topic lies in the following directions.One is to further propose a method of rapid recognition for sampled-data nonlinear systems, and the other is to investigate the deterministic learning of closed-loop system dynamics during NN control of sampled-data systems and the patternbased intelligent control.

Appendix A Proof of Theorem 1
Proof.The identification error system ( 9) and ( 10) can be written in a perturbed LTV form as follows 1+λmax(P )S T (X(k−1))S(X(k−1)) (A1) We introduce the following state transformation: Then, the perturbed LTV system (A1) is reformulated as It is clear from (A2) that the stability of (A3) in the {z(k), W (k)} space implies the stability of (A1) in the {e(k), W (k)} space.By using the localization property of RBF networks, the perturbed LTV system (A3) can be expressed along the recurrent sequence ϕ ζ (X0) as (•) ζ and (•)ζ stand for terms related to the regions close to and away from the sequence ϕ ζ (X0), respectively.S ζ (X) is a subvector of S(X), Ŵζ is the corresponding weight subvector, and is the approximation error along the sequence ϕ ζ (X0).
Based on the properties of RBF networks (see Subsection 2.1), as has been proven in [21], almost any periodic or recurrent sequence ϕ ζ (X0) ensures PE of the regressor subvector S ζ (X).Then, under Assumption 1, S ζ (X) in (A4) satisfies the PE condition in the sense of (2), i.e., there exist constants α1 > 0, T0 n ζ > 0 such that where n ζ denotes the dimension of S ζ (X).
Under the PE condition, we consider firstly the nominal part of the perturbed LTV system (A4), i.e., Choosing the Lyapunov function candidate as where β > 0 will be defined latter.Evaluate its difference along the trajectory of (A7), we get Rewrite (A9) in the matrix form, we have Then, with Assumption 2, it is clear that the matrix M (k) in (A10) is positive definite for all k if and only if 2αTs − The existence of a positive scalar β can be guaranteed by choosing the sampling period Ts such that As a result, M (k) > 0 leads to Therefore, z(k) → 0, and S T ζ (X(k−1)) Wζ (k) → 0 exponentially fast as k → ∞.Thus, we conclude that e(k) → 0 exponentially fast as k → ∞ (see (A2)).e(k) → 0 implies that Ŵ (k) tends to a constant vector (see (7)), and Wζ (k) in turn tends to a constant vector which is denoted as Wc.We now prove Wc = 0 under the satisfaction of PE condition (A6).
Since S T ζ (X) Wc = 0 (see (A14)), we have where The PE condition of φ(k) (B4) leads us to consider We now bound each term on the right side of (B6) from above, and derive an inequality involving Wζ (0) and Wζ (T0).Consider the first term on the right side of (B6).From (B3), we have Then, consider the second term on the right side of (B6).Based on (B3) and (B5), by using Schwarz's inequality, we have Yuan C Z, et al.Sci China Inf Sci March 2014 Vol.57 032201:7

f 3
(x; p) is achieved along the periodic sequence.
Figure 3 illustrates the knowledge representation by using W T S(ϕ ζ ).It is shown that after the NN training phase, the unknown nonlinear dynamic f 3 (x; p) can be locally-accurate represented along ϕ ζ (X 0 ) by the neural network with constant neural weights.