RNN

Position-force control is challenging for redundant manipulators, especially for the ones considering both joint physical limitations and model uncertainties. In this chapter, we considered adaptive motion-force control of redundant manipulators with uncertainties of the interaction model and physical parameters. The whole control problem is formulated as a QP equation with a set of equality and inequality constraints, where based on admittance control strategy, the desired motion-force task is combined with the kinematic property of redundant manipulators, corresponding to an equality constraint in the formed QP equation. Moreover, the uncertainties of both system model and physical parameters are also considered, together with the complicated joint physical structure constraints, formulating as a set of inequality constraints. Then an adaptive recurrent neural network is designed to solve the QP problem online. This control scheme generalizes recurrent neural network based kinematic control of manipulators to that of position-force control, which opens a new avenue to shift position-force control of manipulators from pure control perspective to cross design with both convergence and optimality consideration. Numerical results on a 7-DOF manipulator LBR iiwa and comparisons with existing methods show the validity of the proposed control method.


Introduction
A manipulator is called redundant if its DOFs are greater than those required to complete a task. The redundant DOFs enable the robot to maintain the position and direction of the end actuator to complete a given task and adjust its joint configuration to complete a secondary task. Take advantage of this feature, typical manipulator systems such as collaborative robots, space robotic arms, dexterous hands [1,2] are all designed as redundant ones.
In Chaps. 1 and 2, we mainly focus on kinematic problems, in which we assume the end-effector of a manipulator could move freely in cartesian space. In fact, in industrial applications, the interaction between robot and external environment must be considered, for example, in tasks such as grinding, human-robot interaction, not only the high-precision motion control to a given trajectory should be guaranteed, but also the contact force exerted by the external environment should be guaranteed.
There are several approaches to achieve force control for robot manipulators. By introducing series elastic actuators as flexible units, force control can be realized by adjusting the compliance of joint angles. In [3], in order to overcome the discontinuous friction and complexity problem of traditional back-stepping based methods, a modified command-filter is introduced, and then an adaptive back-stepping controller is designed. Experimental results show the effectiveness of the proposed method. Other control schemes realize force in cartesian space. The most widely used method is called impedance control [4], where the robot and environment are regarded as impedance and admittance, respectively. The interaction model (which is also called impedance model) can be a spring-mass-damper system, spring-damper system, etc. Besides, a series of hybrid position-force controllers are designed in [5,6], which consist of two independent loops, namely position loop, and force loop. By designing control schemes separately, the final control efforts can be formulated as the sum of the output of the independent loops. Similar research can be found in [7][8][9].
In industrial applications, the accurate value of the impedance model can hardly be obtained. for example, the stiffness parameter may be sensitive to environmental factors such as temperature, humidity, etc. Besides, the uncertainties in physical parameters would also affect control performance. In order to due with the uncertainties in the interaction model, in [10], an adaptive impedance controller is designed, in which a neural network is used to learn the nonlinear dynamics of the interaction part. In [11], by considering the influence of unknown dynamics of the external environment, and a radial basis function based controller is proposed, in which an objective function is used to regulate the torque and an adaptive admittance technique is used to minimize path tracking errors. In [12], a human-like-learning based controller is designed for interaction with environmental uncertainties. It is proved that the controller is capable of handling unstable situations such as tool switching, and can finally achieve an expected stability margin. Besides, contact force sensors are not required. Using the approximation ability of artificial neural networks, some intelligent controllers are reported in [13][14][15][16][17]. As to physical uncertainties, in [18], a fixed point controller is proposed based on robust adaptive control theory, the controller also ensures the bounded-ness control torque. Cheng et al. propose a unit quaternion based controller based on neural networks [19], which shows good performance in eliminating singularities, and semi-global stability is proved by theoretical results. In [20], a Jacobian adaptation method based on zeroing dynamics is proposed, in which the Jacobian matrix is updated according to the information of desired and actual accelerations. Other feasible adaptive strategies are reported in [21][22][23][24], in which the Jacobian is estimated by updating physical parameters online. As to physical constraints, in [42], an adaptive neural network control scheme is designed for systems with non-symmetric input dead-zone, as well as output constraints and model uncertainties. The output constraints are guaranteed by the barrier Lyapunov function. In [43], a boundary adaptive robust controller is established for flexible rise systems, in which an auxiliary system is introduced to suppress vibrational offset, and an estima-tor is constructed to observe to upper-bound of disturbances. The controller achieves the global convergence of control errors. Although the above-mentioned controllers could handle uncertainties in the interaction model or physical parameters, few studies have considered both uncertainties at the same time. More importantly, those controllers rarely consider the secondary task, let alone the redundancy resolution problem. Besides, the boundary of joint states is ignored, which is essential in protecting the robot.
In order to accomplish the secondary task in the reliable physical range, a kinematic control method of redundant manipulator based on QP is proposed [25][26][27][28]. The objective function is based on the secondary task, and the constraints describe the basic properties and physical constraints of the system [29]. Because of the high efficiency of parallel computing, the recurrent neural network is often used to solve the redundancy decomposition problem based on QP. In recent years, research shows that RNN based controller has good performance in motion control of redundant manipulator [30]. In [31], in order to achieve task space tracking, the joint velocity command is designed to ensure the boundary of joint angle, velocity, and acceleration. In the paper [32], by maximizing the indirectness of its time derivative, an operational optimization scheme is proposed. Numerical experiments show that the average increase in this method is 40%. In [33], different levels of redundancy resolution are discussed. Recently, RNN based methods have been extended to control examples of flexible robots, multi-robot systems, and methods such as [34][35][36][37][38][39][40]. However, as far as we know, there is no existing dynamic neural network (including RNN and DNN) protocol for the force control of redundant manipulators. It is necessary to consider not only the trajectory tracking problem of free-motion direction, but also the precise control of the contact force, especially for the system with model uncertainty. In addition, from the literature review, one of the research directions of a dynamic neural network is to extend the protocol of redundant manipulator of motion control task to those aspects that need precise control of tracking ability and contact force.
Based on the above observation results, we propose the first RNN based redundant manipulator position force controller, which considers the uncertainty of the interaction model and physical parameters. In this paper, the ideal case of the known model parameters is discussed, and then an adaptive admittance control scheme based on RNN is established. It ensures the boundary of joint angle and velocity. The effectiveness of the proposed controller is verified by the theoretical derivation and numerical results of LBR iiwa. Before concluding this chapter, the main contributions compared to the existing work are as follows • As far as we know, this is the first time to focus on the motion-force control of redundant manipulators with model uncertainties based on the framework of RNNs, comparing to researches on kinematic control, the research is an important extension in the theoretical framework of dynamic neural networks. • Different from traditional methods, an optimization-based controller is proposed, while ensuring the stability of the system, physical limitations are also guaranteed. Which is very helpful to improve the security of the system.
• The controller proposed in this chapter is able to learn uncertain model parameters online, which has better robustness and adaptability to uncertain working conditions. • The proposed method is pseudo-inversion-free, which can save computing costs effectively.

Problem Formulation
When a robot is controlled to perform a given operational task, the forwardkinematics of a serial manipulator is formulated as with θ ∈ R n being the generalize variable of the robot, and x(t) ∈ R m being the description of end-effector s coordinate in task space. Without loss of generality, in this chapter, we assume that all joint are rotational joints. Therefore, θ represents the vector of joint angles. In the velocity level, the Jacobian matrix where a k ∈ R l is a vector of physical parameters. In terms with (3.2), an important property which will be used in the controller design is given as below with Y (θ,θ) ∈ R m×l being the kinematic regressor matrix. The physical parameters are very essential in describing the robot's kinematic model, for example, as the most common physical parameters, the length of robot links affects the DH parameters directly, which are fundamental in the controller design. In this chapter, the physical parameters refer to the length of robot links. Figure 3.1 shows the interaction between the robot and environment, the contact force between the robot and workpiece is required to be precisely controlled. When the fixed contact surface is known, according to the idea of admittance control, the interaction model can be described as a spring-damping system as where K p ∈ R 3×3 and K d ∈ R 3×3 are the corresponding damping and stiffness coefficients, x d is the desired trajectory. If K p and K d are known, the desired contact

Remark 3.1
In this chapter, we only consider the contact force on the vertical direction of the contact surface, and friction force is ignored, therefore F is aligned with the normal direction of the surface. When the surface is priorly known, by defining a rotational matrix R between the tool coordinate system and based coordinate system, K p and K d can be formulated as respectively. Then K p and K d can be described as single parameters.
In practical applications, the real values of system parameters such as a k , K p and K d are usually unavailable. In terms of a k , due to machining and installation error, the length of robot s links may be different from the nominal value, and the robot may hold uncertain tools, which would lead to uncertainties in a k . As to K p and K d , the real values are even more difficult to obtain. K d and K p are related to the material and structure of the workpiece, furthermore, those parameters would change in different environmental conditions. Therefore, it is a challenging issue to achieve precise force control in the presence of parameter uncertainties.
For a redundant manipulator, the redundant DOFs can enhance the flexibility of the robot, and this property can be used to achieve a secondary task. In industrial applications, by minimizing the norm of joint speed, the kinematic energy can be optimized. Therefore, in this chapter, the objective function is selected as In order to save energy consumption in the control process, a smaller value of H (θ ) is preferred.

Remark 3.2
The objective function H (θ) is a typical function to describe the secondary task in redundant resolution problems, as reported in [21,25]. In actual implementations, this function can be defined according to the designer's preferences or actual requirements. In this chapter, we propose a generalized RNN based force control strategy with simultaneous optimization ability. Based on the proposed control strategy, similar controllers can be easily designed by defining different objective functions.

Control Objective and QP Problem Formulation
Before pointing out the control objective, it is noteworthy that the robot must satisfy certain constraints. For example, due to the physical structure, every joint angle θ i must not exceed its limitations i.e., the lower bound θ − i and upper bound θ + i . Furthermore, limited by actual performance of actuators, joint speedθ is also restricted, i.e.,θ − ≤θ ≤θ + .
When the actual parameters of interaction model are unknown, the control objective of this chapter is to design a force controller with adaptation ability, i.e., to realize accurate force control along the predefined contact surface, in the sense that F → F d , at the same time, physical constraints of joint angles and velocities must be ensured. According to (3.2), (3.5) and (3.6), the control objective can be described in the view of optimization as

Remark 3.3
So far, we have arrived at a generalized description of admittance control for redundant manipulators in the QP problem. Apparently, there exist parameter uncertainties in J (θ, a k ), k p and k d as formulated in (3.7b) and (3.7c). In the next chapter, we will solve the problem (3.7) with the aid of RNNs.

Main Results
In this chapter, an recurrent neural network based adaptive admittance controller is proposed to solve (3.7). An ideal situation where real values of system model are perfectly known is firstly considered, which lays the foundation of the later discussion. Then an adaptive RNN is proposed to achieve force control in the presence of model uncertainties. We also show the stability of the control method.

Nominal Design
In order to explain the proposed adaptive control scheme more clearly, an ideal case in which all parameters are perfectly known is firstly discussed. It can be regarded as a special case of uncertain parameter ones. In this case, both K d and K p are available, then the real value ofẋ r is available according to (3.5).
Let ω =θ and define a Lagrange function as with λ being the Lagrange multiplier. Similar to [25], a RNN with provable convergence can be designed as where ε is a positive constant and P ω (•) is a projection operator to set Ω as P ω (x) = argmin y∈Ω ||y − x||, and the set Ω = {ω ∈ R n |ω i min ≤ ω i ≤ ω i max } is a convex set describing the modified speed constraints based on escape velocity method [29], and ω min = max{α(θ min − θ),θ min }, u max = min{α(θ max − θ),θ max }, α > 0. The stability of system can be readily proved, which is similar in [25], is omitted here.

Adaptive Control Method Based on RNN
Based on the previous description, in this subchapter, by learning the uncertain parameters online, an adaptive RNN is established to solve the force control problem with gravity torque optimization under model uncertainties, the stability of the system is also proved.

Adaptive RNN Design
In order to handle the uncertain interaction parameters K p and K d , letK p andK d be their estimates. Although K p and K d are unknown, they are considered to be constant. Then the estimated reference velocityx r can be derived by replacing K p , K d withK p andK d respectively according to (3.5) Then we can rewrite (3.9) as F d =Ŵ T η. However, due to the uncertainties in K d and K p , in the actual process, the resulting contact force F usingx r directly is F = W T η, it is noteworthy that the contact force F can be measured by force/torque sensors.
As to the uncertain a k , the alternative Jacobian matrix J (θ,â k ) is used by substituting a k with its estimateâ k , then J (θ, a k ) in equality constraint (3.7b) is replaced by J (θ,â k ). Therefore, the force control problem with joint speed optimization considering model uncertainties can be formulated as To solve (3.10), by defining a Lagrange function as L = ω T ω + λ(J (θ,â k )ω − x r ), the adaptive RNN is designed as where ε, Γ 1 and Γ 2 are positive gains.

Remark 3.4
In this chapter, we consider the case where m = 6, n = 7 (where m is the dimension of the cartesian space, and n is the number of joint angles). Since only the contact force on the vertical direction of the surface is considered, the dimension of K d and K P are all 1(the contact surface if known). As illustrated in Fig. 3.2, the proposed adaptive RNN has a typical one-layer architecture, and the total number of neurons is n + l + m + 2. On the other hand, unlike the adaptive RNNs based kinematic control strategies in [21,22], the proposed controller can achieve both precise control of both position and contact force.

Stability Analysis
So far, a theorem about the convergence of the force control problem using proposed adaptive RNN in presence of model uncertainties can be summarised as below Theorem 3.1 Consider the force control problem for a category of redundant manipulators described in (3.1)-(3.4) with model uncertainties, the state variable ω of the proposed adaptive RNN will converge the optimal solution of (3.7), i.e., the force control error will converge to 0, and the norm of joint speed will be optimized simultaneously.
Proof The proof consists of three steps. Firstly, we will prove thatŴ andâ k could learn the model parameters online, and then the stability in the inner-loop is also analyzed.
Step 1. Define the estimate error of concatenated form of W asW =Ŵ − W , and e f = F − F d be the error between the contact force and the desired signal. From (3.9), e f can be formulated as e f =Ŵ T η − W T η =W T η. Consider the Lyapunov function as V 1 = tr(W TW )/2, which tr(•) is the trace of a matrix. Calculating the time derivative of V 1 yieldṡ From (3.12) and (3.10) and using LaSalle s invariance principle [41], we have e T f e f = 0, as t → ∞. In other words, the state variableŴ ensures the convergence of force error e f by modifying the end-effector s reference speedx r according to (3.5).
Step 2. Define the estimate error of a k asã k =â k − a k , and let V 2 =ã T kã k /2. It is notable that during the control process, a k can be regarded as constant, then we havė a k =ȧ k . Actually, using the property described in Eq. (

Algorithm 3 The proposed adaptive RNN based force controller
Input: Nominal values of interaction model K n p , K n d and physical parameters a n k . Physical range of joint angles and joint velocities θ i max , θ i min ,θ i max ,θ i min . The desired trajectory x d ,ẋ d and the desired contact force F d . Positive control gains α, Γ 1 , Γ 2 , ε. Sensor readings of contact force F and movement of the end-effector x,ẋ d . Task duration T . Output: To achieve position-force control in presence of model uncetainties 1: Initialize λ(0),â k (0) ← a k ,Ŵ (0) ← [K n p ; K n p ] 2: Repeat 3: x,ẋ, θ,θ, F ← Sensor readings 4: Obtain the Jacobian matrix J (θ,â k ) and kinematic regressor matrix Y (θ,θ) 5: Calculate the modified reference trajectoryx r by Equation ( descriptions ofθ and a k , respectively, the task-space velocityẋ has two equivalent descriptions, namely J (θ, a k )θ and Y (θ,θ)a k . As a result, the estimated valuex also has two similar descriptions, depending on the estimated value of kinematic parameterâ k . Therefore, the updating law Eq.
Step 3. Then we will prove the stability of inner-loop system. According to (3.11), the dynamics of ω and λ can be reformulated as with I being the identity matrix. Then it can be readily obtained that ∇ F + (∇ F) T is positive semi-definite. According to the definition in [32], F is a monotone function of ξ . From the description of (3.11) and (3.15), PΩ can be formulated as PΩ = [P Ω ; P R ], where P R ∈ R m is a projection operator of λ to set R, with the upper and lower bounds being ±∞. Therefore, PΩ is a projection operator to closed setΩ. Based on Lemma 1 in [32], the adaptive RNN (3.11) is stable, and the will be ultimately equivalent to the solution of (3.7). This completes the Proof.

Illustrative Examples
In this chapter, numerical results on a 7-DOF robot manipulator LBR iiwa are carried out. The physical structure and D-H parameters of iiwa are shown in Fig. 3.3. All we all know, up to 6 DOFs (3 DOFs of position and another 3 DOFs of orientation) are required to fulfill a given task in engineering applications, therefore, the iiwa is a typical redundant manipulator in the force control when considering both the position and orientation of the end-effector. As to the contact force, the contact surface is selected as a plane in the workspace, as shown in Fig. 3.3a. The end-effector is controlled to offer a desired contact force on the contact surface while tracking a given path on it. In the control process, the orientation of the end-effector is wished to keep constant. This chapter mainly consists of three parts, firstly, a comparative simulation between the proposed controller and pseudo-inverse of Jacobian matrix(PJMI) based method is firstly discussed, and then the effectiveness of the proposed adaptive controller is checked via more cases. In addition, more discussion about the superiority of the proposed method is carried out to enlighten the contribution of this chapter.

Simulation Setup
In this chapter, the initial value of joint angles is set as θ 0 = [0, π/3, 0, π/3, 0, π/3, 0] T rad, and the corresponding coordinate of the end-effector is noted as P0. The initial value of joint velocity is set asθ 0 = [0, 0, 0, 0, 0, 0, 0] T rad/s. The contact surface is defined as a horizontal plane with z = 0.094m, and the physical parameters of the interaction model are set as K p = 5000, K d = 20, respectively. The limitations of

Comparative Simulation Between PJMI Methods
Firstly, a comparative simulation between the proposed control strategy and traditional Jacobian-inverse based methods is carried out to show the superiority of the RNN based controller. The robot is expected to provide a contact force of 20N at a fixed point P1 = [0.2, 0.6, 0.094] T m, without considering the orientation control of the end-effector. In traditional PJMI based methods, the joint commands are obtained by directly calculating the inverse of the Jacobian matrix online, and only the special solution is considered. Simulation results are shown in Fig. 3.4. Both controllers can guarantee the convergence of positional and force errors. Using the same control gain in the outer loop, although the controller based on PJMI achieves a faster convergence of control errors, its output is big at the beginning of the simulation (with the Euclidean norm of joint velocity being about 20 rad/s), moreover, as shown in Fig. 3.4c, the joint angle θ 6 exceeds its upper bound during 0.2-1 s. In contrast, using the RNN based controller, both joint angles and velocities are ensured not to exceed their limits. It is worth noting that at about t = 0.6s, θ 6 reaches its upper limit (Fig. 3.4e), correspondingly, the joints move a relatively big range, as shown in Fig. 3.4f, as a result, θ 6 stops increasing and then converges to a group of certain values via self-motion.

Force Control Along Predefined Trajectories with Model Uncertainties
In this subchapter, we will carry out a group of experimental tests to further verify the validity of RNN based admittance controller (3.11). In terms of the interaction parameters, we assume the nominal values of K p and K d is 4500 and 15, respectively. As to the kinematic parameters, the nominal value of a k is set to bê  The position error when the simulation begins is about 0.2 m, accordingly, the proposed RNN based controller generates a large output, which ensures the quick convergence of both motion and force errors. The stabilization time is about 0.5 s. At t = 5s, the target point is switched to P2, leading to an instantaneous change of position error. Using the adaptive admittance controller (3.11), the robot adjusts its joint configurations quickly and then slows down with the convergence of errors. It is remarkable that the second joint reaches its maximum value, and during the whole process, the joint velocities are guaranteed not to exceed the predefined limits. The estimated values ofK p andK d are shown in Fig. 3.5f, although the exact values of K d and K p are unknown, by updatingK p andK p online according to (3.11), precise control of control is achieved. The difference between the task-space speedẋ and its estimate value Yâ k is shown in Fig. 3.5g, correspondingly,D 1 ,D 2 ,D 3 andD 4 converge to a group of constant value.

(2) Force Control Along A Circular Path
In this example, the end-effector is controlled to offer constant contact force F d = 20N while tracking a circular trajectory on the contact surface, this trajectory is defined as x d = [−0.1 + 0.1cos(0.5t), −0.6 − 0.1sin(0.5t), 0.094] T m, and the orientation is required to remain the same as the initial state. Numerical results are shown in Figs. 3.7 and 3.8. As shown in Fig. 3.7a, the robot tracks the desired path successfully, and both position and orientation errors converge to zero in less than 1 s, the expected contact force is also obtained. Because of the periodicity of the desired commands, the robot s joint angles and angular velocities change periodically, at the same time, boundedness of θ andθ is also guaranteed. On the other hand, the smooth change of θ andθ shows that the proposed controller is very stable. Based on the adaptive strategy (3.11d), the system shows great robustness against uncertain system parameters.

(3) Force Control Along A Rhodonea Path
In this example, we consider the case where the robot provides a time-varying contact force while tracking a Rhodonea path. The desired contact force is set to be F d = 20 + 5sin(0.2t)N, and the Rhodonea path is defined as  Numerical results are shown in Figs. 3.9 and 3.10. Figure 3.9a, b show the positional and orientational errors the end-effector respect to the desired path, respectively. In the steady-state, accurate motion control is realized using the proposed controller, and the operation force between the end-effector and contact surface is shown in Fig. 3.9c. At the beginning stage, the joint speed is high, which enables the end-effector to move toward the desired path rapidly. As the end-effector approached the expected path, the robot moves at a low speed periodically and smoothly, correspondingly, the joint angles changes at the same frequency. The Euclidean norm of Yâ k −ẋ is illustrated in Fig. 3.9g, the proposed RNN based control strategy could calculate control command ω online with subject to model uncertainties. The estimated model parameters are given in Fig. 3.10f, h.

Comparison
To further illustrate the contribution of the proposed force control strategy, we provide a comparison between the proposed method and the existing related methods, as shown in the Table 3.1. In [11], an adaptive admittance control scheme based on neural network approximation capability is proposed and the admittance adaptive tracking error is optimized. However, no physical constraints are considered. In [10], although the established impedance controller can guarantee input saturation, the controller still needs to calculate the pseudo-inverse of the jacobian. In [25,32], a pseudo-inverse-free controller based on RNN is designed to realize the task space tracking of redundant robots, and its convergence is proved. Convex optimization and non-convex optimization are also obtained. Considering physical uncertainties,   [10], only the input saturation is considered b In [21,22], the authors only consider the uncertainties of physical parameters, while the contact force is ignored two different adaptive strategies are proposed in [21,22]. This is the first RNN-based force controller in this chapter. On the other hand, this control scheme is suitable for the case of model uncertainty, so it is no longer necessary to calculate the pseudoinverse of the Jacobian matrix and has great application potential in force control.

Summary
In this chapter, we propose an adaptive admittance control method for redundant robots based on a recursive neural network. The convergence of the adaptive RNN is proved by the theoretical derivation of the Lyapunov technique, and the effectiveness of the control strategy is verified by numerical simulation on the 7-DOF robot  iiwa. Compared with the existing control methods, the controller not only has better performance in dealing with physical constraints but also has better performance in eliminating pseudo-inversion calculation. Finally, it is worth noting that this is the first time that an RNN-based approach has been extended to force control for the redundant manipulator, especially for model uncertainty manipulators. The research of this subject is of great significance to grinding robot, assembly robot and industrial application. Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.