Deep RNN Based Obstacle Avoidance Control for Redundant Manipulators

In this chapter, we consider the obstacle avoidance problem of redundant robot manipulators with physical constraints compliance, where static and dynamic obstacles are investigated. Both the robot and obstacles are abstracted as two critical point sets, respectively, relying on the general class-K functions, the obstacle avoidance problem is formulated into an inequality in speed level. The minimal-velocity-norm (MVN) is regarded as the cost function, converting the kinematic control problem of redundant manipulators considering obstacle avoidance into a constraint-quadratic-programmingproblem,inwhichthejointanglesandjointveloc-ityconstraintsarebuiltinvelocitylevelinformofinequality.Tosolveit,anovel deeprecurrentneuralnetworkbasedcontrollerisproposed.Theoreticalanalysesand thecorrespondingsimulativeexperimentsaregivensuccessively,showingthatthe proposedneuralcontrollerdoesnotonlyavoidcollisionwithobstacles,butalsotrack thedesiredtrajectorycorrectly.


Introduction
With development of intelligent manufacturing and automation, the research on robot manipulators is obtaining increasing attention from a large number of scholars, numerous fruits have been reported on painting, welding and assembly [1,2] and so on. With the popularization of robots, higher requirements such as flexibility and execution ability are imposed on robots, especially working in the complicated environment [3]. Consequently, more and more scholars cast light on redundant robots which show better flexibility, responsiveness [4,5].
Stem from the consideration of human-machine collaboration, robots are no longer arranged in a separate area [6][7][8], which makes the obstacle avoidance for robots become an important part of kinematic control of the robot manipulators. There has reported many obstacle avoidance methods applicable to robot manipulators. A modified RRT based method, namely Smoothly RRT, was proposed in [9]. This paper established a maximum curvature constraint to obtain a smooth curve when avoiding obstacles. Compared to the traditional RRT based method, the proposed method achieves faster convergence. In [10], Hsu investigated the probabilistic foundations 4

.1 Introduction
Motivated by the above observations, in this chapter, a RNN-based obstacle avoidance strategy was proposed for redundant robot manipulators. Both the robot and obstacles are abstracted as two critical point sets, respectively, relying on the class-K functions, the obstacle avoidance problem is formulated into an inequality in speed level. The minimal velocity-norm (MVN) is regarded as the cost function, converting the kinematic control problem of redundant manipulators considering obstacle avoidance into a constraint-quadratic-programming problem, in which the joint angles and joint velocity constraints are built in velocity level in form of inequality. To solve it, a novel deep recurrent neural network based controller is proposed. Theoretical analyses and the corresponding simulative experiments are given successively, showing that the proposed neural controller does not only avoid collision with obstacles, but also track the desired trajectory correctly. The main contributions of this chapter are summarized as below: • A deep RNN-based controller is proposed. Four objectives are achieved at the same time, i.e, the desired path tracking, obstacle avoidance, physical constraints compliance considering joint angles and velocity constraints and optimality of cost functions. • Relying on a class-K function, a novel obstacle avoidance strategy is given, where robots and obstacles are abstracted into a set of critical point sets, the distance between them can be described as the point-to-point distance. • Numerical results show that the effectiveness of the designed RNN controller, i.e, the static and dynamic obstacles can be avoided while tracking the desired motion trajectory.

Basic Description
When a redundant robot is controlled to track a particular trajectory in the cartesian space, the positional description of the end-effector can be formulated as where x ∈ R m and θ ∈ R n are the end-effector s positional vector and joint angles, respectively. In the velocity level, the kinematic mapping betweenẋ andθ can be described asẋ = J (θ )θ, (4.2) where J (θ ) ∈ R m×n is the Jacobian matrix from the end-effector to joint space. In engineering applications, obstacles are inevitable in the workspace of a robot manipulator. For example, robot manipulators usually work in a limited workspace restricted by fences, which are used to isolate robots from humans or other robots. This problem could be even more acute in tasks which require collaboration of multiple robots. Let C 1 be the set of all the points on a robot body, and C 2 be the set of all the points on the obstacles, then the purpose of obstacle avoidance of a robot manipulator is to ensure C 1 ∪ C 2 = ∅ at all times. By introducing d as a safety distance between the robot and obstacles, the obstacle avoidance is reformulated as gives a basic description of obstacle avoidance problem in form of inequalities. However, there are too many elements in sets C 1 and C 2 , the vast majority of which are actually unnecessary. Therefore, by uniformly selecting points of representative significance from C 1 and C 2 , and increasing d properly, Eq. (4.3) can be approximately described as below

Reformulation of Inequality in Speed Level
In order to guarantee the inequality (4.4), by defining D = |O j A i | − d, an inequality is rebuilt in speed level as d(|O j A i |)/dt ≥ −sgn(D)g(|D|), (4.5) in which g(•) belongs to class-K . Remarkable that the velocities of critical points A i can be described by joint velocities where J ai ∈ R m×n is the Jacobian matrix from the critical point A i to joint space. If O j is prior known, the left-side of Eq. (4.5) can be reformulated as where Therefore, the collision between critical point A i and object O j can be obtained by ensuring the following inequality Based on the inequality description (4.8), the collision between robot and obstacle can be avoided by ensuring ] T ∈ R ab×n is the concatenated form of J oi considering all pairs between A i and O j , B = [B 11 , · · · , B 1b , · · · , B a1 , · · · , B ab ] T ∈ R ab is the vector of upper-bounds, in which B i j = sgn(D)g(|D|) − − −−− → |O j A i | TȮ j . Remark 4.1 According to Eq. 4.5 and the definition of class-K functions, the original escape velocity based obstacle avoidance methods in [34,35] can be regarded as a special case of 4.5, in which G(|D|) is selected as G(|D|) = k|D|. Therefore, in this chapter, the proposed obstacle avoidance strategy is more general than traditional methods.

QP Type Problem Description
As to redundant manipulators, in order to take full advantage of the redundant DOFs, the robot can be designed to fulfill a secondary task when tracking a desired trajectory. In this chapter, the secondary task is set to minimize joint velocity while avoiding obstacles. In real implementations, both joint angles and velocities are limited because of physical limitations such as mechanical constraints and actuator saturation. Because of the fact that rank(J ) < n, there might be infinity solutions to achieve kinematic control. In this chapter, we aim to design a kinematic controller which is capable of avoiding obstacles while tracking a pre-defined trajectory in the Cartesian space. For safety, the robot is wished to move at a low speed, in addition, lower energy consumption is guaranteed. By defining an objective function scaling joint velocity asθ Tθ /2, the tracking control of a redundant manipulator while avoiding obstacles can be described as It is remarkable that the constraints Eq. (4.10b)-(4.10e) and the objective function (4.10a) to be optimized are built in different levels, which is very difficult to solve directly. Therefore, we will transform the original QP problem (4.10) in the velocity level. In order to realize precise tracking control to the desired trajectory x d , by introducing a negative feedback in the outer-loop, the equality constraint can be ensured by letting the end-effector moves at a velocity ofẋ =ẋ d + k(x d − x). In terms with (4.10c), according to the escape velocity method, it can be obtained by limiting joint speed as Combing the kinematic property (4.2), the reformulated QP problem is It is noteworthy that both the formula (4.11a) and (4.11d) are nonlinear. The QP problem cannot be solved directly by traditional methods. Using the parallel computing and learning ability, a deep RNN will be established later.

Deep RNN Based Solver Design
In this chapter, a deep RNN is proposed to solve the obstacle avoidance problem (4.11) online. To ensure the constraints (4.11b), (4.11c), and (4.11d), a group of state variables are introduced in the deep RNN. The stability is also proved by Lyapunov theory.

Deep RNN Design
Firstly, by defining a group of state variables λ 1 ∈ R m , λ 2 ∈ R ab , a Lagrange function is selected as λ 1 and λ 2 are the dual variables corresponding to the constraints (4.11b) and (4.11d). According to Karush-Kuhn-Tucker conditions, the optimal solution of the optimization problem (4.12) can be equivalently formulated aṡ Notable that Equation (4.13c) can be simply described as where (•) + is a projection operation to the non-negative space, in the sense that y + = max(y, 0). Although the solution of (4.13) is exact the optimal solution of the constrainedoptimization problem (4.11), it is still a challenging issue to solve (4.13) online since its inherent nonlinearity. In this chapter, in order to solve (4.13), a deep recurrent neural network is designed as with ε is a positive constant scaling the convergence of (4.15).

Remark 4.2
As to the established deep RNN (4.15), the first dynamic equation is also the output dynamics, which combines the dynamics of state variables λ 1 and λ 2 , as well as the physical limitations including joint angles and velocities. State variable λ 1 is used to ensure the equality constraint (4.11b), as shown in (4.15b), λ 1 is updated according to the difference between reference speedẋ d + k(x d − x) and actually speed J (θ )θ . Similarly, λ 2 is used to ensure the inequality constraint (4.11d), which will be further discussed later. It is remarkable that ε plays an important role in the convergence of the deep RNN. The smaller ε, the faster the deep RNN converges.

Remark 4.
3 By introducing the model information such as J , J o , etc., the established deep RNN consists of three class of nodes, namelyθ , λ 1 and λ 2 , and the total number of nodes is n + m + ab. Comparing to traditional neural networks in [19], the complexity of neural networks is greatly reduced.

Stability Analysis
In this part, we offer stability analysis of the obstacle avoidance method based on deep RNN based. First of all, some basic Lemmas are given as below.

Lemma 4.1 A dynamic neural network is said to converge to the equilibrium point if it satisfies
where κ > 0 and ρ > 0 are constant parameters, and P S = argmin y∈S ||y − x|| is a projection operator to closed set S.
About the stability of the closed-loop system, we offer the following theorem.
Theorem 4.1 Given the obstacle avoidance problem for a redundant manipulator in kinematic control tasks, the proposed deep recurrent neural network is stable and will globally converge to the optimal solution of (4.10).
Proof The stability analysis consists of two parts: firstly, we will show that the equilibrium of the deep RNN is exactly the optimal solution of the control objective described in (4.11), which prove that the output of deep RNN will realize obstacle avoidance while tracking a given trajectory, and then we will prove that the deep recurrent neural network is stable in sense of Lyapunov. Part I. As to the deep recurrent neural network (4.15), let (θ * , λ * 1 , λ * 2 ) be the equilibrium of the deep RNN, then (θ * , λ * 1 , λ * 2 ) satisfies withθ * be the output of deep RNN. By comparing Equation (4.18) and (4.13), we can readily obtain that they are identical, i.e., the equilibrium point satisfies the KKT condition of (4.10). Then we will show that the equilibrium point (output of the proposed deep RNN) is capable of dealing with kinematic tracking as well as obstacle avoidance problem. Define a Lyapunov function V about the tracking error e = x d − x as V = e T e/2, by differentiating V with respect to time and combining (4.11b), we havė (4.19) in which the dynamic equation (4.18b) is also used. It can readily obtained that the tracking error would eventually converge to zero. It is notable that the dynamic equation (4.18c) satisfies According to the property of projection operator (•) + , y − (y) + ≤ 0 holds for any y, then we have J oθ * − B ≤ 0, together with (4.5), the inequality (4.5) is satisfied. Notable that (4.5) can be rewritten aṡ D ≥ −sgn(D)g(|D|). (4.21) As to (4.21), we first consider the situation when equality holds. Since g(|D|) is a function belonging to class K, it can be easily obtained that D = 0 is the only equilibrium ofḊ = −sgn(D)g(|D|). Define a Lyapunov function as V 2 (t, D) = D 2 /2, and select two functions as α 1 (|D|) = α 2 (|D|) = D 2 /2. It is obvious that α 1 (|D|) = α 2 (|D|) belongs to class-K, and the following inequality will always hold Taking the time derivative of V 2 (t, D), we have According to Lemma 4.2, the equilibrium x = 0 is uniformly asymptotically stable. Then we arrive at the conclusion that if the equality d(|O j A i |)/dt = −sgn(D)g (|D|) holds, |D| = 0 will be guaranteed, i.e., |O j A i | − d for all i = 1 · · · a, = 1 · · · b. Based on comparison principle, we can readily obtain that |O j A i | ≥ d when d(|O j A i |)/dt ≥ −sgn(D)g(|D|).
Part II. Then we will show the stability of the deep RNN (4.15). Let ξ = [θ T , λ T 1 , λ T 2 ] T be the a concatenated vector of state variables of the proposed deep RNN, then (4.15) can be rewritten as (4.24) where P S (•) is a projection operator to a set S, and According to the definition of monotone function, we can readily obtain that F(ξ ) is monotone. From the description of (4.24), the projection operator P S can be formulated as P S = [P Ω ; P R ; P ], in which P Ω is defined in (4.13), P R can be regarded as a projection operator of λ 1 to R, with the upper and lower bounds being ±∞, and P = (•) + is a special projection operator to closed set R ab + . Therefore, P S is a projection operator to closed set [Ω; R m ; R ab + ]. Based on Lemma 4.1, the proposed neural network (4.15) is stable and will globally converge to the optimal solution of (4.10). The proof is completed.

Numerical Results
In this chapter, the proposed deep RNN based controller is applied on a planar 4-DOF robot. Firstly, a basic case where the obstacle is described as a single point is discussed, and then the controller is expanded to multiple obstacles and dynamic ones. Comparisons with existing methods are also listed to indicate the superiority of the deep RNN based scheme.

Simulation Setup
The physical structure of the 4-link planar robot to be simulated is shown in Fig. 4.2, in which the critical points of the robot are also marked. As shown in Fig. 4.2, critical points A 2 , A 4 , A 6 are selected at the joint centers, and A 1 , A 3 , A 5 , A 7 are selected at the center of robot links. It is notable that A i and the Jacobian matrix J oi are essential in the proposed control scheme. Based on the above description of A i , the D-H parameters of A 1 is a 1 = 0.15, a 2 = a 3 = 0, α 1 = α 2 = α 3 = 0,

Single Obstacle Avoidance
In this simulation, the obstacle is assumed to be centered at [−0.1, 0.2] T m, the desired path is set as x d = [0.4 + 0.1cos(0.5t), 0.4 + 0.1sin(0.5t)] T m, and the initial joint angles are set to be θ 0 = [π/2, −π/3, −π/4, 0] T rad. The class-K function is selected as G(|D|) = K 1 |D| with K 1 = 200. In order to show the effectiveness of the proposed obstacle avoidance method, contrast simulations with and without inequality constraint (4.10e) are conducted. Simulation results are shown in Fig. 4.3. When ignoring the obstacle, the end-effector trajectories and the corresponding incremental configurations are shown in Fig. 4.3a, although the robot achieves task space tracking to x d , obviously the first link of the robot would collide with the obstacle. After introducing obstacle avoidance scheme, the robot moves other joints rather than the first joint, and then avoids the obstacle effectively (Fig. 4.3b). Simultaneously, the tracking errors when tracking the given circle are shown in Fig. 4.3c. From the initial state, the end-effector moves towards the circle quickly and smoothly, after that, the tracking error in stable state keeps less than 1 × 10 −4 m, showing that the robot could achieve kinematic control as well as obstacle avoidance tasks. To show more details of the proposed deep RNN based method, some important process data is given. As the obstacle is close to the first joint, critical points A 1 and A 2 are more likely to collide with the obstacle, therefore, as a result, the distances between the obstacle O 1 and A 1 , A 2 are shown in Fig. 4.3d, from t = 2s to t = 6.5s, ||A 1 O 1 || remains at the minimum value d = 0.1, that is to say, using the proposed obstacle Numerical results of single obstacle avoidance. a is the motion trajectories when ignoring obstacle avoidance scheme, b is the motion trajectories when considering obstacle avoidance scheme, c is the profile of tracking errors when considering obstacle avoidance scheme, d is the profile of distances between critical points and obstacle, e is the profile of joint velocities, f is the profile of joint angles avoidance method, the robot maintains minimum distance from the obstacle. The profile of joint velocities are shown in Fig. 4.3e, at the beginning of simulation, the robot moves at maximum speed, which leads to the fast convergence of tracking errors. The curve of joint angles change over time is shown in Fig. 4.3f.

Discussion on Class-K Functions
In this part, we will discuss the influence of different class-K functions in the avoidance scheme (4.5). Four functions are selected as G 1 (|D|) = K |D| 2 , G 2 (|D|) = K |D|, G 3 (|D|) = K tanh(5|D|), G 4 (|D|) = K tanh(10|D|), Fig. 4.4a shows the com- Minimun distance(m)  Fig. 4.4b. When selecting the same positive gain K , the minimum distance is about 0.08 m, which shows the robot can avoid colliding with the obstacle using the avoidance scheme (4.5). The close-up graph of the tracking error is also shown, it is remarkable that the minimum distance deceases, as the gradient of the class-K function increases near zero. Therefore, one conclusion can be drawn that the function can be more similar with sign function, to achieve better obstacle avoidance.

Multiple Obstacles Avoidance
In When λ 2 is set to 0, as shown in Fig. 4.5a, the inequality constraint (4.11d) will not work, in other words, only kinematic tracking problem is considered rather than obstacle avoidance, in this case, the robot would collide with the obstacles. After introducing online training of λ 2 , the simulation results are given in Fig. 4.5b-h. The tracking errors are shown in Fig. 4 Numerical results of multiple obstacle avoidance. a is the motion trajectories when ignoring obstacle avoidance scheme. b is the motion trajectories when considering obstacle avoidance scheme. c is the profile of tracking errors when considering obstacle avoidance scheme. d is the profile of distances between critical points and obstacles. e is the profile of joint velocities. f is the profile of λ 2 . g is the profile of joint angles. h is the profile of λ 1 from Fig. 4.5d and g, when the distance between A 3 and O 1 is close to 0.1m, the corresponding dual variable λ 2 becomes positive, making the inequality constraint (4.11d) hold, and the boundary between the robot and obstacle is thus guaranteed. After t = 18s, ||A 3 O 1 || becomes greater, then λ 2 converges to aero. Notable that although λ 1 and λ 2 do not converge to certain values, the dynamic change of λ 1 and λ 2 ensures the regulation of the proposed deep RNN.

Enveloping Shape Obstacles
In this part, we consider obstacles of general significance.  Fig. 4.6b, in which a local amplification diagram is also given, showing that the critical points A 3 is capable of avoiding O 2 and O 3 . It is worth noting that by selecting proper point group and safety distance, the obstacle can be described by the envelope shape effectively. Figure 4.6c, h also give important process data of the system under the proposed controller, including tracking errors, joint angles, angular velocities, and state variables of deep RNNs. We can observe that the physical constraints as well as kinematic control task are realized using the controller.

Comparisons
To illustrate the priority of the proposed scheme, a group of comparisons are carried out. As shown in Table 4.1, all the controllers in [12,16,34,35] achieve the avoidance of obstacles. Comparing to APF method in [12,16] and JP based method in [12,16], the proposed controller can realize a secondary task, at the same time, we present a more general formulation of the obstacle avoidance strategy, which is helpful to gain a deeper understanding of the mechanism for avoidance of obstacles. Moreover, in this chapter, both dynamic trajectories and obstacles are considered. The comparisons above also highlight the main contributions of this paper. Numerical results of enveloping shape obstacles. a is the motion trajectories when ignoring obstacle avoidance scheme. b is the motion trajectories when considering obstacle avoidance scheme. c is the profile of tracking errors when considering obstacle avoidance scheme. d is the profile of distances between critical points and obstacles. e is the profile of joint velocities. f is the profile of joint angles. g is the profile of λ 2 . h is the profile of λ 1  [34,35] and [16], dynamic obstacles are not considered ** Regular escape velocity method is used, which is only a special case of 4.5

Summary
In this chapter, a novel obstacle avoidance strategy is proposed based on a deep recurrent neural network. The robots and obstacles are presented by sets of critical points, then the distance between the robot and obstacle can be approximately described as point-to-points distances. By understanding the nature escape velocity methods, a more general description of obstacle avoidance strategy is proposed. Using minimum-velocity-norm (MVN) scheme, the obstacle avoidance together with path tracking problem is formulated as a QP problem, in which physical limits are also considered. By introducing model information, a deep RNN with simple structure is established to solve the QP problem online. Simulation results show that the proposed method can realize the avoidance of static and dynamic obstacles.