Deep RNN Based Obstacle Avoidance Control for Redundant Manipulators

Abstract

In this chapter, we consider the obstacle avoidance problem of redundant robot manipulators with physical constraints compliance, where static and dynamic obstacles are investigated. Both the robot and obstacles are abstracted as two critical point sets, respectively, relying on the general class-K functions, the obstacle avoidance problem is formulated into an inequality in speed level. The minimal-velocity-norm (MVN) is regarded as the cost function, converting the kinematic control problem of redundant manipulators considering obstacle avoidance into a constraint-quadratic-programming problem, in which the joint angles and joint velocity constraints are built in velocity level in form of inequality. To solve it, a novel deep recurrent neural network based controller is proposed. Theoretical analyses and the corresponding simulative experiments are given successively, showing that the proposed neural controller does not only avoid collision with obstacles, but also track the desired trajectory correctly.

4.1 Introduction

With development of intelligent manufacturing and automation, the research on robot manipulators is obtaining increasing attention from a large number of scholars, numerous fruits have been reported on painting, welding and assembly [1, 2] and so on. With the popularization of robots, higher requirements such as flexibility and execution ability are imposed on robots, especially working in the complicated environment [3]. Consequently, more and more scholars cast light on redundant robots which show better flexibility, responsiveness [4, 5].

Stem from the consideration of human-machine collaboration, robots are no longer arranged in a separate area [6,7,8], which makes the obstacle avoidance for robots become an important part of kinematic control of the robot manipulators. There has reported many obstacle avoidance methods applicable to robot manipulators. A modified RRT based method, namely Smoothly RRT, was proposed in [9]. This paper established a maximum curvature constraint to obtain a smooth curve when avoiding obstacles. Compared to the traditional RRT based method, the proposed method achieves faster convergence. In [10], Hsu investigated the probabilistic foundations of PRM based methods, obtaining a conclusion that the visibility properties has a heavier impact on the probability, and the convergence would be faster if extract partial knowledge could be introduced. However, due to the heavy computational burdens, those methods can be hardly used online.

Apart from stochastic sampling based algorithms mentioned above, the artificial potential field method is also a potential method for obstacle avoidance, and have been found their application in [11,12,13,14,15]. Taking advantage of redundant DOFs, obstacles can be avoided by the self-motion in the null space. Using pseudo-inverse of Jacobian matrix, the solution can be built as the sum of a minimum-norm particular solution and homogeneous solutions [16,17,18].

With parallelism and easier to implement in hardware, neural networks have been a powerful tool in robot control. Artificial intelligence algorithms based on neural networks provide a new view for robotic control, these methods are very promising due to neural networks’ excellent learning ability [19]. For example, in [20], a neural network based learning scheme was proposed to handle functional uncertainties. In [21], a bio-mimetic hybrid controller was designed, where the control strategy consist of an RBF neural network based feed-forward predictive machine and a feedback servo machine based on a proportional-derivative controller. In [22], a fuzzy logic controller is proposed for long-term navigation of quad-rotor UAV systems with input uncertainties. Experiment results show that the controller can achieve better control performance when compared to their singleton counterparts. In [23], an online learning mechanism is built for visual tracking systems. The controller uses both positive and negative sample importance as input, and it is shown that the proposed weighted multiple instance learning scheme achieves wonderful tracking performance in challenging environments. The system model of robot manipulators is highly nonlinear, however, if the prior information of the model is known in advance, the neural network can be optimized. This is to say, on one hand, the number of nodes in neural networks can be reduced. In addition, the excellent learning efficiency is maintained simultaneously [24]. Therefore, to achieve the real-time control of robot manipulators, a series of dynamic neural network are proposed, such as [25,26,27]. For kinematic control of redundant manipulators, such a time-varying problem will be transformed into a quadratic programming from perspective of optimization, where nonlinear mapping from joint space to cartesian space is abstracted as a linear equation. Dynamic neural networks can be used to solve the quadratic-programming problem online, therefore, the kinematic control of manipulators is achieved when the formulated linear equation is ensured. More importantly, these methods can also handle inequality constraints considering joint physical constraints, and model uncertainties [28,29,30,31,32]. There are few works on obstacle avoidance using dynamic neural network. In [33], the obstacle avoidance problem is considered as an equality constraint, however the parameters of the escape velocity is not easy to get. In [34], the distance between the robot and obstacles are formulated as a group of distances from critical points and robot links. On this basis, an improved method is proposed by Guo et. al. in [35], which can suppress undesirable discontinuity in the original solutions.

Motivated by the above observations, in this chapter, a RNN-based obstacle avoidance strategy was proposed for redundant robot manipulators. Both the robot and obstacles are abstracted as two critical point sets, respectively, relying on the class-K functions, the obstacle avoidance problem is formulated into an inequality in speed level. The minimal velocity-norm (MVN) is regarded as the cost function, converting the kinematic control problem of redundant manipulators considering obstacle avoidance into a constraint-quadratic-programming problem, in which the joint angles and joint velocity constraints are built in velocity level in form of inequality. To solve it, a novel deep recurrent neural network based controller is proposed. Theoretical analyses and the corresponding simulative experiments are given successively, showing that the proposed neural controller does not only avoid collision with obstacles, but also track the desired trajectory correctly. The main contributions of this chapter are summarized as below:

• A deep RNN-based controller is proposed. Four objectives are achieved at the same time, i.e, the desired path tracking, obstacle avoidance, physical constraints compliance considering joint angles and velocity constraints and optimality of cost functions.

• Relying on a class-K function, a novel obstacle avoidance strategy is given, where robots and obstacles are abstracted into a set of critical point sets, the distance between them can be described as the point-to-point distance.

• Numerical results show that the effectiveness of the designed RNN controller, i.e, the static and dynamic obstacles can be avoided while tracking the desired motion trajectory.

4.2 Problem Formulation

4.2.1 Basic Description

When a redundant robot is controlled to track a particular trajectory in the cartesian space, the positional description of the end-effector can be formulated as

$$\begin{aligned} x=f(\theta ), \end{aligned}$$

(4.1)

where $x\in \mathbb {R}^m$ and $\theta \in \mathbb {R}^n$ are the end-effector$'$s positional vector and joint angles, respectively. In the velocity level, the kinematic mapping between $\dot{x}$ and $\dot{\theta }$ can be described as

$$\begin{aligned} \dot{x}=J(\theta )\dot{\theta }, \end{aligned}$$

(4.2)

where $J(\theta )\in \mathbb {R}^{m\times n}$ is the Jacobian matrix from the end-effector to joint space.

In engineering applications, obstacles are inevitable in the workspace of a robot manipulator. For example, robot manipulators usually work in a limited workspace restricted by fences, which are used to isolate robots from humans or other robots. This problem could be even more acute in tasks which require collaboration of multiple robots. Let $C_1$ be the set of all the points on a robot body, and $C_2$ be the set of all the points on the obstacles, then the purpose of obstacle avoidance of a robot manipulator is to ensure $C_1\cup C_2=\emptyset$ at all times. By introducing d as a safety distance between the robot and obstacles, the obstacle avoidance is reformulated as

$$\begin{aligned} |O_jA_i|\ge d,\quad \quad \forall A_i\in C_1, \forall O_i\in C_2. \end{aligned}$$

(4.3)

where $|O_jA_i|=\sqrt{(A_i-O_j)^{\text{ T }}(A_i-O_j)}$ is the Euclidean norm of the vector $A_iO_j$.

Equation (4.3) gives a basic description of obstacle avoidance problem in form of inequalities. However, there are too many elements in sets $C_1$ and $C_2$, the vast majority of which are actually unnecessary. Therefore, by uniformly selecting points of representative significance from $C_1$ and $C_2$, and increasing d properly, Eq. (4.3) can be approximately described as below

$$\begin{aligned} |O_jA_i|\ge d, \end{aligned}$$

(4.4)

with $A_i,i=1,\dots ,a$ and $O_j,j=1,\dots ,b$ being the representative points of the robot and obstacles, respectively. The schematic diagram of Eq. (4.4) in shown in Fig. 4.1.

Fig. 4.1

The basic idea of obstacle avoidance in this chapter

4.2.2 Reformulation of Inequality in Speed Level

In order to guarantee the inequality (4.4), by defining $D=|O_jA_i|-d$, an inequality is rebuilt in speed level as

$$\begin{aligned} \text{ d }(|O_j A_i|)/\text{ d }t\ge -\text {sgn}(D)g(|D|), \end{aligned}$$

(4.5)

in which $g(\bullet )$ belongs to class-K. Remarkable that the velocities of critical points $A_i$ can be described by joint velocities

$$\begin{aligned} \dot{A}_i=J_{ai}(\theta )\dot{\theta }, \end{aligned}$$

(4.6)

where $J_{ai}\in \mathbb {R}^{m\times n}$ is the Jacobian matrix from the critical point $A_i$ to joint space. If $O_j$ is prior known, the left-side of Eq. (4.5) can be reformulated as

$$\begin{aligned} \dfrac{\text{ d }}{\text{ d }t}(|O_j A_i|)=&\dfrac{\text{ d }}{\text{ d }t}(\sqrt{(A_i-O_j)^{\text{ T }}(A_i-O_j)})\nonumber \\ =&\dfrac{1}{|O_j A_i|}(A_i-O_j)^{\text{ T }}(\dot{A}_i-\dot{O}_j)\nonumber \\ =&\overrightarrow{|O_j A_i|}^{\text{ T }}J_{ai}(\theta )\dot{\theta }-\overrightarrow{|O_j A_i|}^{\text{ T }}\dot{O}_j, \end{aligned}$$

(4.7)

where $\overrightarrow{|O_j A_i|}=(A_i-O_j)^{\text{ T }}/|O_j A_i|\in \mathbb {R}^{1\times m}$ is the unit vector of $\overrightarrow{A_i-O_j}$. Therefore, the collision between critical point $A_i$ and object $O_j$ can be obtained by ensuring the following inequality

$$\begin{aligned} J_{oi}\dot{\theta }\le \text {sgn}(D)g(|D|)-\overrightarrow{|O_j A_i|}^{\text{ T }}\dot{O}_j, \end{aligned}$$

(4.8)

where $J_{oi}=-\overrightarrow{|O_j A_i|}^{\text{ T }}J_{ai}\in \mathbb {R}^{1\times n}$. Based on the inequality description (4.8), the collision between robot and obstacle can be avoided by ensuring

$$\begin{aligned} J_{o}\dot{\theta }\le B, \end{aligned}$$

(4.9)

where $J_o=[\underbrace{J_{o1}^{\text{ T }},\cdots ,J_{o1}^{\text{ T }}}_{b},\cdots ,\underbrace{J_{oa}^{\text{ T }},\cdots ,J_{oa}^{\text{ T }}}_{b}]^{\text{ T }}\in \mathbb {R}^{ab\times n}$ is the concatenated form of $J_{oi}$ considering all pairs between $A_i$ and $O_j$, $B=[B_{11},\cdots ,B_{1b},\cdots ,B_{a1},\cdots ,B_{ab}]^{\text{ T }}\in \mathbb {R}^{ab}$ is the vector of upper-bounds, in which $B_{ij}=\text {sgn}(D)g(|D|)-\overrightarrow{|O_j A_i|}^{\text{ T }}\dot{O}_j$.

Remark 4.1

According to Eq. 4.5 and the definition of class-K functions, the original escape velocity based obstacle avoidance methods in [34, 35] can be regarded as a special case of 4.5, in which G(|D|) is selected as $G(|D|)=k|D|$. Therefore, in this chapter, the proposed obstacle avoidance strategy is more general than traditional methods.

4.2.3 QP Type Problem Description

As to redundant manipulators, in order to take full advantage of the redundant DOFs, the robot can be designed to fulfill a secondary task when tracking a desired trajectory. In this chapter, the secondary task is set to minimize joint velocity while avoiding obstacles. In real implementations, both joint angles and velocities are limited because of physical limitations such as mechanical constraints and actuator saturation. Because of the fact that $\text {rank}(J)<n$, there might be infinity solutions to achieve kinematic control. In this chapter, we aim to design a kinematic controller which is capable of avoiding obstacles while tracking a pre-defined trajectory in the Cartesian space. For safety, the robot is wished to move at a low speed, in addition, lower energy consumption is guaranteed. By defining an objective function scaling joint velocity as $\dot{\theta }^{\text{ T }}\dot{\theta }/2$, the tracking control of a redundant manipulator while avoiding obstacles can be described as

$$\begin{aligned} \text {min}~~~~&\dot{\theta }^{\text{ T }}\dot{\theta }/2, \end{aligned}$$

(4.10a)

$$\begin{aligned} s.t.~~~~&x=x_{\text{ d }},\end{aligned}$$

(4.10b)

$$\begin{aligned}&\theta ^{-}\le \theta \le \theta ^{+},\end{aligned}$$

(4.10c)

$$\begin{aligned}&\dot{\theta }^{-}\le \dot{\theta }\le \dot{\theta }^{+},\end{aligned}$$

(4.10d)

$$\begin{aligned}&J_{o}\dot{\theta }\le B. \end{aligned}$$

(4.10e)

It is remarkable that the constraints Eq. (4.10b)–(4.10e) and the objective function (4.10a) to be optimized are built in different levels, which is very difficult to solve directly. Therefore, we will transform the original QP problem (4.10) in the velocity level. In order to realize precise tracking control to the desired trajectory $x_{\text{ d }}$, by introducing a negative feedback in the outer-loop, the equality constraint can be ensured by letting the end-effector moves at a velocity of $\dot{x}=\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x)$. In terms with (4.10c), according to the escape velocity method, it can be obtained by limiting joint speed as $\alpha (\theta ^{-}-\theta )\le \dot{\theta }\le \alpha (\theta ^{+}-\theta )$, where $\alpha$ is a positive constant. Combing the kinematic property (4.2), the reformulated QP problem is

$$\begin{aligned} \text {min}~~~~&\dot{\theta }^{\text{ T }}\dot{\theta }/2,\end{aligned}$$

(4.11a)

$$\begin{aligned} s.t.~~~~&J(\theta )\dot{\theta }=\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x),\end{aligned}$$

(4.11b)

$$\begin{aligned}&\text {max}(\alpha (\theta ^{-}-\theta ),\dot{\theta }^{-})\le \dot{\theta }\le \text {min}(\dot{\theta }^{+},\alpha (\theta ^{+}-\theta )),\end{aligned}$$

(4.11c)

$$\begin{aligned}&J_{o}\dot{\theta }\le B. \end{aligned}$$

(4.11d)

It is noteworthy that both the formula (4.11a) and (4.11d) are nonlinear. The QP problem cannot be solved directly by traditional methods. Using the parallel computing and learning ability, a deep RNN will be established later.

4.3 Deep RNN Based Solver Design

In this chapter, a deep RNN is proposed to solve the obstacle avoidance problem (4.11) online. To ensure the constraints (4.11b), (4.11c), and (4.11d), a group of state variables are introduced in the deep RNN. The stability is also proved by Lyapunov theory.

4.3.1 Deep RNN Design

Firstly, by defining a group of state variables $\lambda _1\in \mathbb {R}^m$, $\lambda _2\in \mathbb {R}^{ab}$, a Lagrange function is selected as

$$\begin{aligned} L=\dot{\theta }^{\text{ T }}\dot{\theta }/2+\lambda _1^{\text{ T }}(\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x)-J(\theta )\dot{\theta })+\lambda _2^{\text{ T }}(J_{o}\dot{\theta }-B), \end{aligned}$$

(4.12)

$\lambda _1$ and $\lambda _2$ are the dual variables corresponding to the constraints (4.11b) and (4.11d). According to Karush-Kuhn-Tucker conditions, the optimal solution of the optimization problem (4.12) can be equivalently formulated as

$$\begin{aligned}&\dot{\theta }=P_{\varOmega }(\dot{\theta }-\dfrac{\partial L}{\partial \dot{\theta }}),\end{aligned}$$

(4.13a)

$$\begin{aligned}&J(\theta )\dot{\theta }=\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x),\end{aligned}$$

(4.13b)

$$\begin{aligned}&\left\{ \begin{array}{ll} \lambda _2>0~~~~~\text {if}~~~J_{o}\dot{\theta }=B,\\ \lambda _2=0~~~~~\text {if}~~~J_{o}\dot{\theta }\le B, \end{array}\right. \end{aligned}$$

(4.13c)

where $P_{\varOmega }(x)=\text {argmin}_{y\in \varOmega }||y-x||$ is a projection operator to a restricted interval $\varOmega$, which is defined as $\varOmega =\{\dot{\theta }\in \mathbb {R}^n|\text {max}(\alpha (\theta ^{-}-\theta ),\dot{\theta }^{-})\le \dot{\theta }\le \text {min}(\dot{\theta }^{+},\alpha (\theta ^{+}-\theta ))\}$. Notable that Equation (4.13c) can be simply described as

$$\begin{aligned} \lambda _2=(\lambda _2+J_o\dot{\theta }-B)^+, \end{aligned}$$

(4.14)

where $(\bullet )^+$ is a projection operation to the non-negative space, in the sense that $y^{+}=\text {max}(y,0)$.

Although the solution of (4.13) is exact the optimal solution of the constrained-optimization problem (4.11), it is still a challenging issue to solve (4.13) online since its inherent nonlinearity. In this chapter, in order to solve (4.13), a deep recurrent neural network is designed as

$$\begin{aligned} \varepsilon \ddot{\theta }&=-\dot{\theta }+P_{\varOmega }(J^{\text{ T }}\lambda _1-J_o^{\text{ T }}\lambda _2),\end{aligned}$$

(4.15a)

$$\begin{aligned} \varepsilon \dot{\lambda }_1&=\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x)-J(\theta )\dot{\theta },\end{aligned}$$

(4.15b)

$$\begin{aligned} \varepsilon \dot{\lambda }_2&=-\lambda _2+(\lambda _2+J_o\dot{\theta }-B)^{+}, \end{aligned}$$

(4.15c)

with $\varepsilon$ is a positive constant scaling the convergence of (4.15).

Remark 4.2

As to the established deep RNN (4.15), the first dynamic equation is also the output dynamics, which combines the dynamics of state variables $\lambda _1$ and $\lambda _2$, as well as the physical limitations including joint angles and velocities. State variable $\lambda _1$ is used to ensure the equality constraint (4.11b), as shown in (4.15b), $\lambda _1$ is updated according to the difference between reference speed $\dot{x}_{\text{ d }}+k(x_{\text{ d }}-x)$ and actually speed $J(\theta )\dot{\theta }$. Similarly, $\lambda _2$ is used to ensure the inequality constraint (4.11d), which will be further discussed later. It is remarkable that $\varepsilon$ plays an important role in the convergence of the deep RNN. The smaller $\varepsilon$, the faster the deep RNN converges.

Remark 4.3

By introducing the model information such as J, $J_o$, etc., the established deep RNN consists of three class of nodes, namely $\dot{\theta }$, $\lambda _1$ and $\lambda _2$, and the total number of nodes is $n+m+ab$. Comparing to traditional neural networks in [19], the complexity of neural networks is greatly reduced.

4.3.2 Stability Analysis

In this part, we offer stability analysis of the obstacle avoidance method based on deep RNN based. First of all, some basic Lemmas are given as below.

Definition 4.1

A continuously differentiable function $F(\bullet )$ is said to be monotone, if $\nabla F+\nabla F^{\text{ T }}$ is positive semi-definite, where $\nabla F$ is the gradient of $F(\bullet )$.

Lemma 4.1

A dynamic neural network is said to converge to the equilibrium point if it satisfies

$$\begin{aligned} \kappa \dot{x}=-\dot{x}+P_S(x-\rho F(x)), \end{aligned}$$

(4.16)

where $\kappa >0$ and $\rho >0$ are constant parameters, and $P_S=\text {argmin}_{y\in S}||y-x||$ is a projection operator to closed set S.

Lemma 4.2

[37] Let $V: [0,\infty )\times B_d\rightarrow \mathbb {R}$ be a $C^1$ function, $\alpha _1$, $\alpha _2$ be class-K functions defined on [0, d) which satisfy $\alpha _1(||x||)\le V(t,x)\le \alpha _2(||x||)$, $\forall (t,x)\in [0,d)\times B_d$, then $x=0$ is a uniformly asymptotically stable equilibrium point if there exist some class-K function g defined on [0, d), to make the following inequality hold

$$\begin{aligned} \dfrac{\partial V}{\partial t}+\dfrac{\partial V}{\partial x}f(t,x)\le -\alpha (||x||), \forall (t,x)\in [0,\infty )\times B_d, \end{aligned}$$

(4.17)

About the stability of the closed-loop system, we offer the following theorem.

Theorem 4.1

Given the obstacle avoidance problem for a redundant manipulator in kinematic control tasks, the proposed deep recurrent neural network is stable and will globally converge to the optimal solution of (4.10).

Proof

The stability analysis consists of two parts: firstly, we will show that the equilibrium of the deep RNN is exactly the optimal solution of the control objective described in (4.11), which prove that the output of deep RNN will realize obstacle avoidance while tracking a given trajectory, and then we will prove that the deep recurrent neural network is stable in sense of Lyapunov.

Part I. As to the deep recurrent neural network (4.15), let $(\dot{\theta }^*,\lambda _1^*,\lambda _2^*)$ be the equilibrium of the deep RNN, then $(\dot{\theta }^*,\lambda _1^*,\lambda _2^*)$ satisfies

$$\begin{aligned} -\dot{\theta }^*+P_{\varOmega }(J^{\text{ T }}\lambda _1^*-J_o^{\text{ T }}\lambda _2^*)=0,\end{aligned}$$

(4.18a)

$$\begin{aligned} \dot{x}_{\text{ d }}+k(x_{\text{ d }}-x)-J(\theta )\dot{\theta }^*=0,\end{aligned}$$

(4.18b)

$$\begin{aligned} -\lambda _2^*+(\lambda _2^*+J_o\dot{\theta }^*-B)^{+}=0, \end{aligned}$$

(4.18c)

with $\dot{\theta }^*$ be the output of deep RNN. By comparing Equation (4.18) and (4.13), we can readily obtain that they are identical, i.e., the equilibrium point satisfies the KKT condition of (4.10).

Then we will show that the equilibrium point (output of the proposed deep RNN) is capable of dealing with kinematic tracking as well as obstacle avoidance problem. Define a Lyapunov function V about the tracking error $e=x_{\text{ d }}-x$ as $V=e^{\text{ T }}e/2$, by differentiating V with respect to time and combining (4.11b), we have

$$\begin{aligned} \dot{V}&=e^{\text{ T }}\dot{e}=e^{\text{ T }}(\dot{x}_{\text{ d }}-J(\theta )\dot{\theta }^*)\nonumber \\&=-ke^{\text{ T }}e\le 0, \end{aligned}$$

(4.19)

in which the dynamic equation (4.18b) is also used. It can readily obtained that the tracking error would eventually converge to zero.

It is notable that the dynamic equation (4.18c) satisfies

$$\begin{aligned} \lambda _2^*+J_o\dot{\theta }^*-B-(\lambda _2^*+J_o\dot{\theta }^*-B)^{+}=J_o\dot{\theta }^*-B. \end{aligned}$$

(4.20)

According to the property of projection operator $(\bullet )^+$, $y-(y)^+\le 0$ holds for any y, then we have $J_o\dot{\theta }^*-B\le 0$, together with (4.5), the inequality (4.5) is satisfied. Notable that (4.5) can be rewritten as

$$\begin{aligned} \dot{D}\ge -\text {sgn}(D)g(|D|). \end{aligned}$$

(4.21)

As to (4.21), we first consider the situation when equality holds. Since g(|D|) is a function belonging to class K, it can be easily obtained that $D=0$ is the only equilibrium of $\dot{D}=-\text {sgn}(D)g(|D|)$. Define a Lyapunov function as $V_2(t,D)=D^2/2$, and select two functions as $\alpha _1(|D|)=\alpha _2(|D|)=D^2/2$. It is obvious that $\alpha _1(|D|)=\alpha _2(|D|)$ belongs to class-K, and the following inequality will always hold

$$\begin{aligned} \alpha _1(|D|)\le V_2(t,D)\le \alpha _2(|D|). \end{aligned}$$

(4.22)

Taking the time derivative of $V_2(t,D)$, we have

$$\begin{aligned} \dfrac{\partial V_2}{\partial \text{ t }}+\dfrac{\partial V}{\partial D}\dot{D}=-|D|g(|D|)\le 0. \end{aligned}$$

(4.23)

According to Lemma 4.2, the equilibrium $x=0$ is uniformly asymptotically stable. Then we arrive at the conclusion that if the equality $\text{ d }(|O_j A_i|)/\text{ d }t=-\text {sgn}(D)g(|D|)$ holds, $|D|=0$ will be guaranteed, i.e., $|O_jA_i|-d$ for all $i=1\cdots a$, $=1\cdots b$. Based on comparison principle, we can readily obtain that $|O_jA_i|\ge d$ when $\text{ d }(|O_j A_i|)/\text{ d }t\ge -\text {sgn}(D)g(|D|)$.

Part II. Then we will show the stability of the deep RNN (4.15). Let $\xi =[\dot{\theta }^{\text{ T }},\lambda _1^{\text{ T }},\lambda _2^{\text{ T }}]^{\text{ T }}$ be the a concatenated vector of state variables of the proposed deep RNN, then (4.15) can be rewritten as

$$\begin{aligned} \varepsilon \dot{\xi }=-\xi +P_{\bar{\varOmega }}[\xi -F(\xi )], \end{aligned}$$

(4.24)

where $P_S(\bullet )$ is a projection operator to a set S, and $F(\xi )=[F_1(\xi ),F_2(\xi ),F_3(\xi )]^{\text{ T }}\in \mathbb {R}^{n+m+ab}$, in which

$$\begin{aligned} \begin{bmatrix} F_1\\ F_2\\ F_3 \end{bmatrix}=\begin{bmatrix} \dot{\theta }-J^{\text{ T }}\lambda _1+J_o^{\text{ T }}\lambda _2\\ J\dot{\theta }-\dot{x}_{\text{ d }}-k(x_{\text{ d }}-x)\\ -J_o\dot{\theta }^*-B \end{bmatrix}.\nonumber \end{aligned}$$

Let $\nabla F=\partial F/\partial \xi$, we have

$$\begin{aligned} \nabla F(\xi )=\begin{bmatrix} I&{}&{}-J^{\text{ T }}&{}&{}J_o^{\text{ T }}\\ J&{}&{}0&{}&{}0\\ -J_o^{\text{ T }}&{}&{}0&{}&{}0 \end{bmatrix}. \end{aligned}$$

(4.25)

According to the definition of monotone function, we can readily obtain that $F(\xi )$ is monotone. From the description of (4.24), the projection operator $P_S$ can be formulated as $P_S=[P_{\varOmega };P_{R};P_{\Lambda }]$, in which $P_{\varOmega }$ is defined in (4.13), $P_{R}$ can be regarded as a projection operator of $\lambda _1$ to R, with the upper and lower bounds being $\pm \infty$, and $P_{\Lambda }=(\bullet )^+$ is a special projection operator to closed set $\mathbb {R}_+^{ab}$. Therefore, $P_S$ is a projection operator to closed set $[\varOmega ;\mathbb {R}^m;\mathbb {R}_+^{ab}]$. Based on Lemma 4.1, the proposed neural network (4.15) is stable and will globally converge to the optimal solution of (4.10). The proof is completed. $\Box$

4.4 Numerical Results

In this chapter, the proposed deep RNN based controller is applied on a planar 4-DOF robot. Firstly, a basic case where the obstacle is described as a single point is discussed, and then the controller is expanded to multiple obstacles and dynamic ones. Comparisons with existing methods are also listed to indicate the superiority of the deep RNN based scheme.

Fig. 4.2

The planar robot to be simulated in this chapter

4.4.1 Simulation Setup

The physical structure of the 4-link planar robot to be simulated is shown in Fig. 4.2, in which the critical points of the robot are also marked. As shown in Fig. 4.2, critical points $A_2$, $A_4$, $A_6$ are selected at the joint centers, and $A_1$, $A_3$, $A_5$, $A_7$ are selected at the center of robot links. It is notable that $A_i$ and the Jacobian matrix $J_{oi}$ are essential in the proposed control scheme. Based on the above description of $A_i$, the D-H parameters of $A_1$ is $a_1=0.15$, $a_2=a_3=0$, $\alpha _1=\alpha _2=\alpha _3=0$, $d_1=d_2=d_3=0$, then both the position and Jacobian matrix $J_{a1}$ of $A_1$ can be calculated readily. Based on the definition in Eq. (4.8), $J_{o1}$ can be obtained. $A_i$ and $J_{oi}$ can be calculated similarly. The control parameters are set as $\varepsilon =0.001$, $\alpha =8$, $k=8$. As to the physical constraints, the limits of joint angles and velocities are selected as $\theta _i^-=-3$rad, $\theta _i^+=3$rad, $\dot{\theta }_i^-=-1$rad/s, $\dot{\theta }_i^+=1$rad/s for $i=1\dots 4$. The safety distance d is set to be 0.1 m.

Fig. 4.3

Numerical results of single obstacle avoidance. a is the motion trajectories when ignoring obstacle avoidance scheme, b is the motion trajectories when considering obstacle avoidance scheme, c is the profile of tracking errors when considering obstacle avoidance scheme, d is the profile of distances between critical points and obstacle, e is the profile of joint velocities, f is the profile of joint angles

4.4.2 Single Obstacle Avoidance

In this simulation, the obstacle is assumed to be centered at $[-0.1,0.2]^{\text{ T }}$m, the desired path is set as $x_{\text{ d }}=[0.4+0.1cos(0.5t),0.4+0.1sin(0.5t)]^{\text{ T }}$m, and the initial joint angles are set to be $\theta _0=[\pi /2,-\pi /3,-\pi /4,0]^{\text{ T }}$rad. The class-K function is selected as $G(|D|)=K_1|D|$ with $K_1=200$. In order to show the effectiveness of the proposed obstacle avoidance method, contrast simulations with and without inequality constraint (4.10e) are conducted. Simulation results are shown in Fig. 4.3. When ignoring the obstacle, the end-effector trajectories and the corresponding incremental configurations are shown in Fig. 4.3a, although the robot achieves task space tracking to $x_{\text{ d }}$, obviously the first link of the robot would collide with the obstacle. After introducing obstacle avoidance scheme, the robot moves other joints rather than the first joint, and then avoids the obstacle effectively (Fig. 4.3b). Simultaneously, the tracking errors when tracking the given circle are shown in Fig. 4.3c. From the initial state, the end-effector moves towards the circle quickly and smoothly, after that, the tracking error in stable state keeps less than $1\times 10^{-4}$m, showing that the robot could achieve kinematic control as well as obstacle avoidance tasks. To show more details of the proposed deep RNN based method, some important process data is given. As the obstacle is close to the first joint, critical points $A_1$ and $A_2$ are more likely to collide with the obstacle, therefore, as a result, the distances between the obstacle $O_1$ and $A_1$, $A_2$ are shown in Fig. 4.3d, from $t=2$s to $t=6.5$s, $||A_1O_1||$ remains at the minimum value $d=0.1$, that is to say, using the proposed obstacle avoidance method, the robot maintains minimum distance from the obstacle. The profile of joint velocities are shown in Fig. 4.3e, at the beginning of simulation, the robot moves at maximum speed, which leads to the fast convergence of tracking errors. The curve of joint angles change over time is shown in Fig. 4.3f.

Fig. 4.4

Discussions on different obstacle avoidance functions. a is the comparative curves of different obstacle avoidance functions. b is the profile of minimum distance of the robot and obstacle using different obstacle avoidance functions

4.4.3 Discussion on Class-K Functions

In this part, we will discuss the influence of different class-K functions in the avoidance scheme (4.5). Four functions are selected as $G_1(|D|)=K|D|^2$, $G_2(|D|)=K|D|$, $G_3(|D|)=K\text {tanh}(5|D|)$, $G_4(|D|)=K\text {tanh}(10|D|)$, Fig. 4.4a shows the comparative curves the these functions. Other simulation settings are the same as the previous one. Simulation results are shown in Fig. 4.4b. When selecting the same positive gain K, the minimum distance is about 0.08 m, which shows the robot can avoid colliding with the obstacle using the avoidance scheme (4.5). The close-up graph of the tracking error is also shown, it is remarkable that the minimum distance deceases, as the gradient of the class-K function increases near zero. Therefore, one conclusion can be drawn that the function can be more similar with sign function, to achieve better obstacle avoidance.

Fig. 4.5

Numerical results of multiple obstacle avoidance. a is the motion trajectories when ignoring obstacle avoidance scheme. b is the motion trajectories when considering obstacle avoidance scheme. c is the profile of tracking errors when considering obstacle avoidance scheme. d is the profile of distances between critical points and obstacles. e is the profile of joint velocities. f is the profile of $\lambda _2$. g is the profile of joint angles. h is the profile of $\lambda _1$

4.4.4 Multiple Obstacles Avoidance

In this part, we consider the case where there are two obstacles in the workspace. The obstacles are set at $[0.1,0.25]^{\text{ T }}$m and $[0,0.4]^{\text{ T }}$m, respectively. Simulation results are shown in Fig. 4.5. The desired path is defined as $x_{\text{ d }}=[0.45+0.1cos(0.5t),0.4+0.1sin(0.5t)]^{\text{ T }}$. The initial joint angle of the robot is selected as $\theta _0=[1.5,-1-1,0]^{\text{ T }}$. To further show the effectiveness of the proposed obstacle avoidance strategy 4.5, g|D| is selected as $g|D|=K_1/(1+e^{-|D|})-K_1/2$ with $K_1=200$. When $\lambda _2$ is set to 0, as shown in Fig. 4.5a, the inequality constraint (4.11d) will not work, in other words, only kinematic tracking problem is considered rather than obstacle avoidance, in this case, the robot would collide with the obstacles. After introducing online training of $\lambda _2$, the simulation results are given in Fig. 4.5b–h. The tracking errors are shown in Fig. 4.5c, with the transient time being about 4s, and steady state error less than $1\times 10^{-3}$m. Correspondingly, the robot moves fast in the transient stage, ensuring the quick convergence of the tracking errors. It is remarkable that the distances between the critical points and obstacle points are kept larger than 0.1m at all times, showing the effectiveness of the proposed method. At $t=14$s, from Fig. 4.5d and g, when the distance between $A_3$ and $O_1$ is close to 0.1m, the corresponding dual variable $\lambda _2$ becomes positive, making the inequality constraint (4.11d) hold, and the boundary between the robot and obstacle is thus guaranteed. After $t=18$s, $||A_3O_1||$ becomes greater, then $\lambda _2$ converges to aero. Notable that although $\lambda _1$ and $\lambda _2$ do not converge to certain values, the dynamic change of $\lambda _1$ and $\lambda _2$ ensures the regulation of the proposed deep RNN.

Fig. 4.6

Numerical results of enveloping shape obstacles. a is the motion trajectories when ignoring obstacle avoidance scheme. b is the motion trajectories when considering obstacle avoidance scheme. c is the profile of tracking errors when considering obstacle avoidance scheme. d is the profile of distances between critical points and obstacles. e is the profile of joint velocities. f is the profile of joint angles. g is the profile of $\lambda _2$. h is the profile of $\lambda _1$

4.4.5 Enveloping Shape Obstacles

In this part, we consider obstacles of general significance. Suppose that there is a rectangular obstacle in the workspace, with the vertices being $[0,0.5]^{\text{ T }}$, $[0.4,0.5]^{\text{ T }}$, $[0.4,0.6]^{\text{ T }}$ and $[0.5,0.6]^{\text{ T }}$, respectively. By selecting the safety distance $d=0.1$m, and obstacle points as $O_1=[0.05,0.55]^{\text{ T }}$, $O_2=[0.15,0.55]^{\text{ T }}$, $O_3=[0.25,0.55]^{\text{ T }}$ and $O_4=[0.35,0.55]^{\text{ T }}$. It can be readily obtained that the rectangular obstacle is totally within the envelope defined by $O_i$ and d. The incremental configurations when tracking the path while avoiding the obstacle are shown in Fig. 4.6b, in which a local amplification diagram is also given, showing that the critical points $A_3$ is capable of avoiding $O_2$ and $O_3$. It is worth noting that by selecting proper point group and safety distance, the obstacle can be described by the envelope shape effectively. Figure 4.6c, h also give important process data of the system under the proposed controller, including tracking errors, joint angles, angular velocities, and state variables of deep RNNs. We can observe that the physical constraints as well as kinematic control task are realized using the controller.

4.4.6 Comparisons

To illustrate the priority of the proposed scheme, a group of comparisons are carried out. As shown in Table 4.1, all the controllers in [12, 16, 34, 35] achieve the avoidance of obstacles. Comparing to APF method in [12, 16] and JP based method in [12, 16], the proposed controller can realize a secondary task, at the same time, we present a more general formulation of the obstacle avoidance strategy, which is helpful to gain a deeper understanding of the mechanism for avoidance of obstacles. Moreover, in this chapter, both dynamic trajectories and obstacles are considered. The comparisons above also highlight the main contributions of this paper.

4.5 Summary

In this chapter, a novel obstacle avoidance strategy is proposed based on a deep recurrent neural network. The robots and obstacles are presented by sets of critical points, then the distance between the robot and obstacle can be approximately described as point-to-points distances. By understanding the nature escape velocity methods, a more general description of obstacle avoidance strategy is proposed. Using minimum-velocity-norm (MVN) scheme, the obstacle avoidance together with path tracking problem is formulated as a QP problem, in which physical limits are also considered. By introducing model information, a deep RNN with simple structure is established to solve the QP problem online. Simulation results show that the proposed method can realize the avoidance of static and dynamic obstacles.

Table 4.1 Comparisons among different obstacle avoidance controllers on manipulators