Abstract
This paper investigates a feature tracking control method for visual servoing (VS) manipulators adaptive dynamic programming (ADP)-based the unknown dynamics. The major superiority of ADP-based optimal control lies in that the visual tracking problem is converted to the feature tracking error control with optimal cost function. Moreover, an adaptive neural network observer is developed to approximate the entire uncertainties, which are utilized to construct an improved cost function. By establishing a critic neural network, the Hamilton–Jacobi–Bellman (HJB) equation is solved, and the approximate optimal error control policy is derived. The closed-loop VS manipulator system is verified to be ultimately uniformly bounded with the developed ADP-based feature tracking control strategy according to the Lyapunov theory. Finally, simulation results under various situations demonstrate that the proposed method achieves higher tracking accuracy than other methods, as well as satisfies energy optimal requirements.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
To comply with the requirements of modern manufacturing for efficiency, visualization and wireless communication [1,2,3], manipulators equipped with different sensors are competent to adapt extreme ambient conditions by means of the non-contact detection. Manipulators with visual sensors simulate human vision which allows the feedback controller to be measured in non-contact positions and directions. At present, visual servoing (VS) manipulators have a wide application potential in many scenarios such as disaster rescue, medical detection, space exploration, etc [4,5,6]. It is well known that the tracking control strategy is demanded to provide the more precision and less consumption of the VS systems especially in the unstructured environments, but only a few advanced techniques have been applied to the VS tracking control to ensure the optimum of system tracking performance.
Related work
Generally, the goal of the feature trajectory tracking control is to drive the system outputs track specified desired trajectories in the VS system. Hence, visual feedback signals have been used as significant information in robotics to tackle the positioning or motion control in unstructured environments. The visual tracking problem for manipulators has been studied over the past few years and a wide range of technologies have been explored.
Kang et al. [7] adopted a reinforcement learning method to adaptively adjust the servoing gain to improve the convergence rate and stability. Li et al. [8] combined proportional derivative (PD) control with sliding mode control (SMC) to tackle the disturbance and uncertainties on a 6-degree-of-freedom (DOF) VS manipulator. Sharma et al. [9] proposed a fractional order SMC method to drive vehicle motion using visual information of image plane. Furthermore, considering the uncertainties, they designed a new adaptive rule to adjust the sliding surface parameters to ensure the finite time stability of the system. Qiu et al. [10] presented a depth-independent interaction matrix based on model predictive control method by taking the input and output constraints into account.
Unlike the above kinematic VS control problems, the dynamic VS control problems can be solved by establishing the composite Jacobian matrix mapping from the image space to the robot joint space which is usually adopted. Based on the obtained system parameters, an effective controller can be designed. For example, Wang et al. [11] presented a new adaptive algorithm based on the estimated image depth, and proved the global stability by Lyapunov method. Li et al. [12] addressed an effective controller design problem for an uncalibrated camera–manipulator system to ensure the finite-time convergence. In addition, some scholars have also paid attention to the large measurement error caused by external interference or system modeling deviation. Based on the parameter uncertainties in manipulator kinematics and dynamics, Cheah et al. [13] proposed an adaptive regression strategy to estimate the Jacobian matrix adaptively. Hua et al. [14] adopted immersion and invariance observer to identify an uncalibrated VS system without measuring the joint velocity on the basis of depth independent matrix. Wang et al. [15] designed a new nonlinear observer to dynamically track the motion of a target in Cartesian space. Wang et al. [16] proposed a novel adaptive observer controller which employed the feature velocity term contained in the unknown kinematics. The superiority of the proposed image-space observer lied in its simple structure in handling uncertainties; thus, it avoided the over parametrization in the existing works. Leite et al. [17] developed a cascade control strategy based on an indirect/direct adaptive method, which introduced uncertainties in robot kinematics and dynamics of visual tracking problem. Considered the output nonlinearity and unknown dynamics, Wang et al. [18] investigated an adaptive neural network control for the VS manipulator system whose dynamic model is not required to be linearly decomposable. Zhang et al. [19] developed an adaptive neural network controller with the Barrier Lyapunov Function (BLF) to overcome nonlinearities and visibility constraint problems.
In addition, the key points of optimal control problems of nonlinear systems are to design a suitable controller tackling input/output constraints, external disturbance, uncertainties, etc [20,21,22,23]. During the past few years, adaptive dynamic programming (ADP) algorithm which is proposed by Werbos [24] has extensively developed the optimal control schemes for robots or manipulators to enhance the control performance and reduce the energy consumption of the controller [25]. Kong et al. [26] introduced an approximate optimal strategy to resolve the non-linearity saturation problem of n-DOF manipulators. In [27], an adaptive fuzzy neural network control method with impedance learning was presented for robots with constraints. In [28], the optimal coordination control which was applied to multi-robots to follow expected trajectories was presented by means of reinforcement learning. Tang et al. [29] employed a reinforcement learning-based adaptive optimal control method to realize the optimal tracking of n-DOF manipulators. Li et al. [30] established the nonlinear discrete-time dynamic model of wheeled mobile robots, where the reinforcement learning and ADP method were adopted to tackle the tracking problem for systems with skidding and slipping constraints. In [31], an artificial potential field scheme cooperated with ADP method was proposed for path planning of bio-mimetic robot fish, where heuristic learning programming was applied to obtain the position and angle. Lian et al. [32] presented a receding-horizon dual heuristic programming algorithm for tracking control of wheeled mobile robots, and developed a backstepping kinematic controller. Zhan et al. [33] proposed an ADP-based control approach to deal with tracking problem for robots with environment interactions. Li et al. [34] proposed a policy iteration-based fault compensation control for modular reconfigurable robots subject to actuator failures. Zhao et al. [35] developed an event-triggered ADP algorithm of decentralized tracking control which can reduce communication frequency and extend the service life of mechanical and electronic devices. Dong et al. [36] designed a novel force/position control scheme based on zero-sum optimal ADP decentralized control strategy for decentralized system by considering the influence of unknown and interconnected dynamics for reconfigurable manipulators. The application of ADP-based optimal control in the field of robotics has progressed in the recent years, the optimal feature tracking control for VS system which is expected in practical systems is still an open problem.
Motivation and contribution
In recent few years, many kinematics-based visual controllers have been proposed by assuming that the VS manipulator has an accurate positioning device with negligible dynamics [37,38,39]. However, in the control perspective, it is difficult to ensure the dynamic performance and stable control when neglecting the nonlinearities in kinematic control due to the existence of both parameter uncertainties in robot dynamics and errors in camera calibration. Therefore, the controller design is a challenging task with considering the influence of control error and system stability, especially in the robot positioning or trajectory tracking control [12, 40]. Unfortunately, there is few discussion on the difficulties of VS manipulator systems, especially with uncertainties of intrinsic parameters, camera calibration errors, external disturbance, friction, etc. From the aforementioned literature, we conclude that the difficulties in designing controllers lie in how to handle unmodeled dynamics and external disturbance without linearity-in-parameters. Furthermore, it is expected to optimize the performance index in VS manipulator systems.
Inspired by the above literature, this paper investigates a feature tracking controller based on ADP scheme for VS manipulators subject to unknown dynamics by taking optimal performance index into account. Based on the radial basis function (RBF) estimated uncertain dynamics of the VS manipulator model, an adaptive NN observer is proposed to identify the uncertainties (e.g., unmodeled dynamics, external disturbance, joint friction, etc) in real time. The cost function is improved by inserting the estimated uncertainties, and the visual tracking problem is converted to the feature error control. Then, the optimal feature tracking control is derived directly. Therefore, the stability of VS manipulator systems is guaranteed by utilizing Lyapunov stability theorem. Finally, in order to show the robustness and effectiveness of the designed controller, a 3-DOF (Degree of Freedom) eye-to-hand (ETH) manipulator is employed to simulation.
The main contributions of the presented scheme can be summarized as follows.
-
1.
In this paper, the proposed feature tracking control strategy, which directly acts on image feature, is facilitated more feasibility and intuitively. Thus, the designing controller based on the camera–manipulator model does not need to obtain regression matrix and avoids complicated calculation.
-
2.
It is the first time to develop the ADP technique to feature-based visual tracking control for VS manipulator systems with unknown dynamics. Unlike the existing visual tracking control approaches, the critic NN-based controller is designed in an optimal manner, which saves the energy cost and is significant in practice.
-
3.
The major advantage of the improved cost function lies in that the estimated uncertainties is introduced and given full consideration in controller design. Simultaneously, the closed-loop VS manipulator system can be guaranteed to be ultimately uniformly bounded (UUB) using the proposed ADP control scheme.
The remainder of this paper is organized as follows. In “Preliminaries and problem statement”, the basic preliminaries and dynamic model are presented. In “ADP-based feature tracking controller”, the unknown dynamics of VS manipulator systems is approximated by an adaptive NN observer, and the optimal error controller is designed in detail. Then, the stability is analyzed. In “Simulation tests”, simulation examples are provided to illustrate the effectiveness of the proposed control scheme. Finally, a brief conclusion is given in “Conclusion”.
Preliminaries and problem statement
Camera–robot kinematics model
In this paper, the ETH structure is selected for the VS system that is shown in Fig. 1, and a n-DOF VS manipulator is employed to construct the forward kinematics. Denote the image coordinate of feature point as \(f_{uv}=[u,v]^\mathrm{{T}}\). The mapping from feature point to robot position [14] can be expressed as
where \(r(t) \in {\mathbb {R}}^3\) is the Cartesian coordinate of robot end-effector with respect to the base frame, \({{D_\mathrm{{epth}}}(t) \in {{{\mathbb {R}}}}}\) is the depth of feature point in the camera frame, \({M_\mathrm{{c}}} \in {\mathbb {R}}^{{3}\times {4}}\) is the perspective projection matrix which can be expressed as
where \({M_\mathrm{{in}}} \in {\mathbb {R}}^{{3}\times {4}}\) is the intrinsic matrix of the camera, \({M_\mathrm{{ex}}} \in {\mathbb {R}}^{{4}\times {4}}\) is the homogenous transformation matrix computed via forward kinematics, which also represents the extrinsic matrix.
By separating \(f_{uv}(t)\) from (1), we can obtain
where \({M_\mathrm{{sub}}} \in {\mathbb {R}}^{{2}\times {3}}\) is the sub-matrix of perspective projection matrix \({M_\mathrm{{c}}}\), which is given by
where \(m_{ij}\) is the ijth component of \({M_\mathrm{{c}}}\).
The depth of the feature point can be given by
where \(M_\mathrm{{D}}=[m_{31}, m_{32}, m_{33}]^\mathrm{{T}}\). Assume \({D_\mathrm{{epth}}}(t)\) be a positive and bounded constant; i.e.,
By differentiating (2) and (3), one obtains
where \({J_\mathrm{{I}}}\in {\mathbb {R}}^{{2s}\times {3}}\) is the feature Jacobian matrix (or interaction matrix), s is the number of feature points. Let \(q(t)\in {\mathbb {R}}^n\) be the joint angle vector. From the robot kinematics, the velocity relationship of joint space to Cartesian space can be expressed as
where \(J_\mathrm{{R}}\in {\mathbb {R}}^{{3}\times {n}}\) is the robot Jacobian matrix. Combining (4) with (5), we obtain
where \(J_\mathrm{{com}}\in {\mathbb {R}}^{{2s}\times {n}}\) denotes the compound Jacobian matrix. We can rewrite (6) as
where \(J_\mathrm{{com}}^ + = {(J_\mathrm{{com}}^\mathrm{{T}}{J_\mathrm{{com}}})^{ - 1}}J_\mathrm{{com}}^\mathrm{{T}}\) is the pseudo-inverse of the compound Jacobian matrix. In practice, the manipulator is required to perform a servoing task in a reachable finite task-space [41]. To avoid the Jacobian matrix singularity, \(M_\mathrm{{c}}\) should be full rank. Hence, \(J_\mathrm{{com}}\) is full rank and its pseudo-inverse matrix exists, whose detailed illustration can be found in [42, 43].
By differentiating (7), the acceleration of joint angle q(t) is formulated as
Camera–manipulator dynamic model
Considering a general n link manipulator, whose dynamic model can be mathematically formulated as
where \(N(q)\in {\mathbb {R}}^{{n}\times {n}}\) is the inertia matrix, \(B(q,\dot{q})\in {\mathbb {R}}^{{n}\times {n}}\) is the centrifugal and coriolis force, \(G(q)\in {\mathbb {R}}^n\) is the gravitational term, \(A(\dot{q})\in {\mathbb {R}}^n\) denotes the friction term, \(F_\mathrm{{d}}\in {\mathbb {R}}^n\) indicates the external disturbance, and \(\tau \in {\mathbb {R}}^n\) denotes the output torque.
Combining (7)–(9), the dynamics of VS manipulators can be expressed as
Multiplying the term \({(J_\mathrm{{com}}^ + )^\mathrm{{T}}}\) on both sides of (10), the dynamics of the VS manipulators is expressed in the workspace as
where \(N_\mathrm{{o}} (f_{uv}) = {(J_\mathrm{{com}}^ +)^\mathrm{{T}}}N(q)J_\mathrm{{com}}^ + \), \({C_\mathrm{{o}}}(f_{uv},\dot{f}_{uv}) = {(J_\mathrm{{com}}^ + ) ^\mathrm{{T}}}(N(q)\frac{\mathrm{{d}}}{{{\mathrm{{d}}t}}}(J_\mathrm{{com}}^ + ) + B(q,\dot{q})J_\mathrm{{com}}^ +)\), \({G_\mathrm{{o}}}(f_{uv}) = {(J_\mathrm{{com}}^ +) ^\mathrm{{T}}}G(q)\), \({A_\mathrm{{o}}}(\dot{f}_{uv}) = {(J_\mathrm{{com}}^ +)^\mathrm{{T}}}A(\dot{q})\), \({F _\mathrm{{do}}} = {(J_\mathrm{{com}}^ + )^\mathrm{{T}}}\)
\({F _\mathrm{{d}}}\), and \({\tau _\mathrm{{o}}} = {(J_\mathrm{{com}}^ + )^\mathrm{{T}}}\tau \).
Due to the uncertain kinematic parameters and unmodeled dynamics, the actual parameters of system can be decomposed into the nominal part and uncertainties, so (11) can be written as
where \({\bar{N}}_\mathrm{{o}}(f_{uv})\), \({{{\bar{B}}}_\mathrm{{o}}}(f_{uv},\dot{f}_{uv})\), \({{{\bar{G}}}_\mathrm{{o}}}(f_{uv})\) and \({{{\bar{A}}}_\mathrm{{o}}}(\dot{f}_{uv})\) are the nominal part, \(\varDelta N_\mathrm{{o}}(f_{uv}))\), \(\varDelta {B_\mathrm{{o}}}(f_{uv},\dot{f}_{uv})\), \(\varDelta {G_\mathrm{{o}}}(f_{uv})\) and \(\varDelta {A_\mathrm{{o}}}(\dot{f}_{uv})\) are the uncertainties.
By separating the uncertainties from the dynamics of camera–manipulator model, (12) can be reformulated as
where the uncertainties \(D(f_{uv})\) is given as
Before designing and analyzing the optimal feature tracking controller, the camera–manipulator dynamic system (13) is supposed to satisfy the following properties.
Property 1
The inertia matrix \({N_\mathrm{{o}}}(f_{uv})\) is symmetric, positive scalar, and satisfies
where \(\lambda _1\) and \(\lambda _2\) are positive constants.
Property 2
The time-derivative of the inertia matrix \({N_\mathrm{{o}}}(f_{uv})\) and centripetal and coriolis matrix \({B_\mathrm{{o}}}(f_{uv},\dot{f}_{uv})\) satisfies the skew-symmetric relationship as
Assumption 1
The friction torque \({A_\mathrm{{o}}}(\dot{f}_{uv})\) is bounded by \(\left\| {{A_\mathrm{{o}}}(\dot{f}_{uv})} \right\| \le {\varphi _1}\) , where \({\varphi _1} \in {\mathbb {R}}\) is an unknown positive constant.
Assumption 2
The uncertain dynamics \({D}(f_{uv})\) is bounded by \(\left\| {{D}( f_{uv})} \right\| \le {\varphi _2}\) , where \({\varphi _2} \in {\mathbb {R}}\) is an unknown positive constant.
From the properties above, the VS manipulator systems can be rewritten to facilitate the ADP design. By transforming the dynamic model (13), the state space expression of VS system is proposed as
where \(x = {[{x_1},{x_2}]^\mathrm{{T}}} = {[f_{uv},\dot{f}_{uv}]^\mathrm{{T}}}\) with \({x_1},{x_2} \in {\mathbb {R}}^{2s}\) is the system state vector, y is the output vector, k(x) and g(x) can be defined as
Assumption 3
k(x) and g(x) are locally Lipschitz and continuous in their arguments with \(k(x)=0\).
Remark 1
It is observed from (13), the input and output of dynamic model are mapped from the direct form (i.e., \({\tau _\mathrm{{o}}} \rightarrow f_{uv},\dot{f}_{uv},\ddot{f}_{uv}\)) to the indirect form (i.e., \(\tau \rightarrow \ddot{q},\dot{q} \rightarrow \dot{r} \rightarrow \dot{f}_{uv}\), see (4), (6) and (9)). Moreover, a camera–manipulaotr dynamic model is established by taking the uncertainties \(D(f_{uv})\) into account. In this way, linearity in parameter features cannot be employed in the VS manipulator systems. In this paper, the ADP-based control approach is presented to solve the feature tracking control problem of VS manipulator systems with uncertainties. This implies that the proposed scheme guarantees the closed-loop VS manipulator systems to converge to zero, i.e., the actual trajectories can follow their desired trajectories.
ADP-based feature tracking controller
Optimal visual control
As we know, the aim of the optimal feature tracking control is to design an effective tracking control policy which follows the desired feature trajectory. To achieve this objective, the feature tracking control can be obtained by combining the desired visual tracking control and feature tracking error control.
Assumption 4
The desired feature trajectory \(f_{uv_\mathrm{{d}}}\) , the desired feature velocity \(\dot{f}_{uv_\mathrm{{d}}}\) and the desired feature acceleration \({\ddot{f}_{uv_\mathrm{{d}}}}\) are all bounded and known.
Letting \({x_{f_{uv_\mathrm{{d}}}}} = {[{f_{uv_\mathrm{{d}}}},{\dot{f}_{uv_\mathrm{{d}}}}]^\mathrm{{T}}}\) and \({\dot{x}_{f_{uv_\mathrm{{d}}}}} = {[{\dot{f}_{uv_\mathrm{{d}}}},{\ddot{f}_{uv_\mathrm{{d}}}}]^\mathrm{{T}}}\), the desired feature trajectory can be described as
where \(\tau _{f_{uv_\mathrm{{d}}}}\) denotes the desired control torque. Then, the desired visual tracking controller can be obtained by
From the state space expression of system (15), the feature tracking error dynamics can be expressed by
where \(e_\mathrm{{f}}\) indicates the feature error and \({\dot{e}}_\mathrm{{f}}\) denotes the time derivative of \(e_\mathrm{{f}}\). For the state space expression of camera–manipulator system (15), the optimal objective is to derive the control law by minimizing the following infinite horizon cost function
where \(P\left( {{e_\mathrm{{f}}},{\tau _\mathrm{{fe}}}} \right) = e_\mathrm{{f}}^\mathrm{{T}}Q{e_\mathrm{{f}}} + \tau _\mathrm{{fe}}^\mathrm{{T}}R{\tau _\mathrm{{fe}}}\) denotes the utility function, \(P\left( {{e_\mathrm{{f}}},{\tau _\mathrm{{fe}}}} \right) \ge 0\) with \(P\left( {0,0} \right) = 0\), \({\tau _{f_{e}}} = {\tau _\mathrm{{o}}} - {\tau _{f_{uv_\mathrm{{d}}}}}\) is the optimal control input error, \(Q\in {\mathbb {R}}^{{2s}\times {2s}}\) and \(R\in {\mathbb {R}}^{{2s}\times {2s}}\) are the positive definite matrices, \({{\hat{D}}}(t)\) is the estimation of uncertainties, and \(\alpha > 0\) is an unknown constant.
Definition 1
[44] For the dynamic system (15), a control policies \(\tau _\mathrm{{fe}}\) is admissible with respect to the cost function (19) on a compact set \(\varOmega \) , if \(\tau _\mathrm{{fe}}\) is continuous on \(\varOmega \) with \(\tau _\mathrm{{fe}}(0)=0\), \(\tau _\mathrm{{fe}}\) stabilizes on \(\varOmega \), and \(U\left( {{e_\mathrm{{f}}}} \right) \) is finite \(\forall {e_\mathrm{{f}}} \in \varOmega \).
Given a series of admissible control policies \({\tau _\mathrm{{fe}}} \in \varXi (\varOmega )\), then the infinitesimal version of (19) is the the so-called Lyapunov equation as
where \(U(0) = 0\) and \(\nabla U({e_\mathrm{{f}}}) = \frac{{\partial U({e_\mathrm{{f}}})}}{{\partial {e_\mathrm{{f}}}}}\) is the partial derivative of \(U({e_\mathrm{{f}}})\) with respect to \(e_\mathrm{{f}}\). The Hamiltonian and the improved cost function can be given by
Thus, the solution of the HJB equation can be obtained by
where \(\nabla {U^*}\left( {{e_\mathrm{{f}}}} \right) = \frac{{\partial {U^*}\left( {{e_\mathrm{{f}}}} \right) }}{{\partial {e_\mathrm{{f}}}}}\). If \({U^*}\left( {{e_\mathrm{{f}}}} \right) \) is continuously differentiable, the optimal feature tracking error controller of the VS system will be derived as
According to (21) and (23), we can obtain
Adaptive neural network observer design
The uncertainties are estimated by an adaptive NN observer, which can be formulated by
where \({{{\hat{x}}}_{\mathrm{{fo}}_{uv}}}\) denotes the observation of the system state x, \(\beta \) is the positive definite observation gain matrix, and \({O_\mathrm{{e}}} = x - {{{\hat{x}}}_{\mathrm{{fo}}_{uv}}}\) denotes the state observer error.
Combining (15) with (26), we can present the observation error dynamics as
where \(\varGamma (x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\! =\! {k_e}(x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}}) + {g_e}(x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})({\tau _\mathrm{{o}}} - D(x))\), \({k_e}(x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\! =\! k(x) - k({{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\) and \({g_e}(x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}}) = g(x) - g({{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\) are the observation errors of k(x) and g(x), respectively. According to Assumption 3, \(\varepsilon \) is a positive constant such that \(||g(\hat{x}_{fouv}) ||= \varepsilon \).
Assumption 5
\(\varGamma (x,{{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\) is norm-bounded as \(\left\| {\varGamma (x,{{{{\hat{x}}}}_{\mathrm{{fo}}_{uv}}})} \right\| \le {\omega _1}\), where \(\omega _1\) is a positive constant.
To estimate the uncertainties D(x) , RBFNN is constructed as
where \({W_\mathrm{{D}}} \in {\mathbb {R}}^{{l_1} \times 2s}\) denotes the ideal weight matrix, \(\varPhi \left( x \right) \in {\mathbb {R}}^{l_1}\) denotes the NN activation function, \(l_1\) indicates the number of neurons in the hidden layer, and \({\chi _\mathrm{{D}}}\) indicates the NN approximation error.
Let \({{\hat{W}}}_\mathrm{{D}} \) be the estimation of \(W_\mathrm{{D}}\). \({{\hat{D}}}({{{\hat{x}}}_{\mathrm{{fo}}_{uv}}})\) is the estimation of D(x) , which can be expressed as
where \({{\hat{W}}}_\mathrm{{D}}\) can be updated by
where \(\mu \) is a positive definite matrix. From (28) and (29), one obtains
where \({{\tilde{W}}}_\mathrm{{D}} = {W_\mathrm{{D}}} - {{{\hat{W}}}_\mathrm{{D}}}\) is the weight estimation error, \({{\tilde{\varPhi }}} \left( {x,{{{{\hat{x}}}}_{\mathrm{{fo}}_{uv}}}} \right) = \varPhi \left( x \right) - {{\hat{\varPhi }}} \left( {{{{{\hat{x}}}}_{\mathrm{{fo}}_{uv}}}} \right) \) is the estimation error of the activation function.
Assumption 6
The local observation error \({W_\mathrm{{e}}} = W_\mathrm{{D}}^{\mathrm{T}}{\tilde{\varPhi }} \left( {x,}\right. \left. {{{{{\hat{x}}}}_{\mathrm{{fo}}_{uv}}}} \right) + {\chi _\mathrm{{D}}}\) is norm-bounded as \(\left\| {{W_\mathrm{{e}}}} \right\| \le {\omega _2}\), where \(\omega _2 > 0\) is an unknown constant.
Theorem 1
For the VS manipulator systems with uncertainties, the proposed adaptive NN observer can ensure the observation error to be UUB with the help of the NN updating law (30).
Proof
Choose a Lyapunov function candidate as
The time derivative of (32) is
where \({\lambda _{\min }}\left( \beta \right) \) is the minimum eigenvalue of the matrix. Thus, substituting (29) into (33), we have
It can be seen that \({\dot{L}_1} \le 0\) when \(O_\mathrm{{e}}\) lies outside the compact set \({\varOmega _1} = \left\{ {{O_\mathrm{{e}}}:\left\| {{O_\mathrm{{e}}}} \right\| \le \frac{{{\omega _1} + {\omega _2}\varepsilon }}{{{\lambda _{\min }}\left( \beta \right) }}} \right\} \). According to the Lyapunov’s direct method, the state observation error can be guaranteed to be UUB. This concludes the proof.
Critic NN and implementation
As an excellent learning tool of nonlinear functions, NN is widely considered to approximate the cost function (22). Thereby, the improved cost function can be expressed by a critic NN on the compact set \({\varOmega _2}\), which is given by
where \({W_U} \in {\mathbb {R}}{^{{l_2}}}\) denotes the ideal weight matrix, \({\varPhi _U}({e_\mathrm{{f}}}) \in {\mathbb {R}}{^{{l_2}}}\) indicates the NN basis function, \(l_2\) denotes the number of neurons in the hidden layer, and \(\chi _U\) is the NN approximation error. The partial derivative of \({T_U}({e_\mathrm{{f}}})\) with respect to \(e_\mathrm{{f}}\) is
where \(\nabla {\varPhi _U}({e_\mathrm{{f}}}) = \frac{{\partial {\varPhi _U}({e_\mathrm{{f}}})}}{{\partial {e_\mathrm{{f}}}}}\) and \(\nabla {\chi _U}\) are the partial derivatives of the basis function \({\varPhi _U}({e_\mathrm{{f}}})\) and the NN approximation error \(\chi _U\), respectively. A critic NN is utilized to approximate the improved cost function as
Thus, the partial derivative of \({{{\hat{T}}}_U}({e_\mathrm{{f}}})\) with respect to \(e_\mathrm{{f}}\) is
Considering (23), the ideal optimal feature tracking error control policy can be described by
Thus, according to (37) and (38), the approximation optimal feature tracking error control can be given by
For the uncertain system (15), considering (20) and (36), one can obtain
Therefore, the Hamiltonian can be expressed by
where \({E_{UH}} = - {(\nabla {\chi _U})^\mathrm{{T}}}{\dot{e}_\mathrm{{f}}}\) is the approximation residual. And the approximate Hamiltonian is derived in the same manner, which is expressed as
Defining the error function as \({E_U} = H\left( {{e_\mathrm{{f}}},{\tau _\mathrm{{fe}}},{W_U}} \right) - {{\hat{H}}}\left( {{e_\mathrm{{f}}},{\tau _\mathrm{{fe}}},{{{{\hat{W}}}}_U}} \right) \) , then combining (42) with (43), we have
where \({{{\tilde{W}}}_U} = {W_U} - {{{\hat{W}}}_U}\) is the weight estimation error.
Assumption 7
The NN function \(\delta = \nabla {\varPhi _U}({e_\mathrm{{f}}}){\dot{e}_\mathrm{{f}}}\) is norm-bounded as \(\left\| \delta \right\| \le {\delta _e}\), where \(\delta _e\) is a positive constant.
To adjust the critic NN weight vector \({{{\hat{W}}}_U}\), we can minimize the objective function \({E_\mathrm{{obj}}} = \frac{1}{2}E_U^\mathrm{{T}}{E_U}\) with the updating law as
where \(\mu _U\) is the learning rate of the critic NN. Hence, considering (44) and (45), one can obtain the updating law of weight estimation error as
Theorem 2
For the uncertain camera–manipulator system, the weight vector approximation error of the critic NN can be guaranteed to be UUB with the updating law (45).
Proof
Choose a Lyapunov function candidate as
The time derivative of (47) is
According to Young’s inequality, we can obtain
Therefore, \({\dot{Z}_2} < 0\) when the weight approximation error \({{{{\tilde{W}}}}_U}\) lies outside the compact set \({\varOmega _3} = \left\{ {{{{{\tilde{W}}}}_U}:\left\| {{{{{\tilde{W}}}}_U}} \right\| \le \left\| {\frac{{{E_{UH}}}}{{{\delta _e}}}} \right\| } \right\} \). Thus, the weight approximation error can be guaranteed to be UUB. This concludes the proof.
Stability analysis
Unlike existing visual tracking control methods which neglected the optimal control performance, this paper improves the cost function with the information from an adaptive NN observer. Furthermore, via the ADP approach, we develop a novel optimal feature tracking error control method that optimizes the control performance and ensures the system stability.
The optimal feature tracking controller which composes of the desired tracking controller \(\tau _\mathrm{{fd}}\) and feature tracking error controller \(\tau _\mathrm{{fe}}\) is derived by
Theorem 3
Consider system dynamics of VS manipulator (15) and improved cost function (19), the closed-loop VS system is UUB under the optimal tracking control policy (50).
Proof
Choose a Lyapunov function candidate as
Considering (15), (16) and (24), the time derivative of (51) is expressed as
where \({k_\mathrm{{d}}}(x,{x_\mathrm{{fd}}}) = k(x) - k({x_\mathrm{{fd}}})\) and \({g_\mathrm{{d}}}(x,{x_\mathrm{{fd}}}) = g(x) - g({x_\mathrm{{fd}}})\). According to Assumption 3, \(\varepsilon _f\) is a positive constant such that \(\left\| {{k_\mathrm{{d}}}(x,{x_\mathrm{{fd}}})} \right\| \le {\varepsilon _f}\left\| {{e_\mathrm{{f}}}} \right\| \). Assuming \(\left\| {g(x)} \right\| \le {\kappa _1}\), \(\left\| {g({x_\mathrm{{fd}}})} \right\| \le {\kappa _2}\) and \(\left\| {g(x) - g({x_\mathrm{{fd}}})} \right\| \le {\kappa _3}\), we have
Assuming \(\left\| {{\tau _\mathrm{{fd}}}} \right\| \le {\zeta _1}\) and \(\left\| {D(x) - {{\hat{D}}}({{\hat{x}}})} \right\| \le {\zeta _2}\), where \(\zeta _1\) and \(\zeta _2\) are positive constant. We have
Therefore, it can be seen that \({\dot{Z}_3} < 0\) when \(e_\mathrm{{f}}\) lies outside the compact set \({\varOmega _4} = \left\{ {{e_\mathrm{{f}}}:\left\| {{e_\mathrm{{f}}}} \right\| \le \sqrt{\frac{{\kappa _3^2\zeta _1^2 + \kappa _1^2\zeta _2^2}}{{2({\lambda _{\min }}(Q) - {\varepsilon _f} - \frac{3}{2})}}} } \right\} \) with the following conditions hold.
Simulation tests
In this section, we employ a 3-DOF humanoid manipulator with one feature point marked on the end-effector for simulation tests [45, 46]. The performance of the proposed ADP-based feature tracking control are implemented in two cases, i.e., without/with uncertainties.
The 3-DOF manipulator system and the control parameters are presented in Tables 1 and 2. The intrinsic matrix \(M_\mathrm{{in}}\) and the extrinsic matrix \(M_\mathrm{{ex}}\) are given by
Define the desired feature trajectories as
In the adaptive NN observer, Gaussian type function is selected as the activation function, the center of the basis function Bc is
and the width of the activation function Bb = 80. The improved cost function (19) is approximated by a critic NN, and the weight vector is \({{{{\hat{W}}}}_U} = {[{{{{\hat{W}}}}_{U1}},{{{{\hat{W}}}}_{U2}},{{{{\hat{W}}}}_{U3}},{{{{\hat{W}}}}_{U4}},}{{{{{\hat{W}}}}_{U5}},{{{{\hat{W}}}}_{U6}},{{{{\hat{W}}}}_{U7}},{{{{\hat{W}}}}_{U8}},{{{{\hat{W}}}}_{U9}},{{{{\hat{W}}}}_{U10}}]^\mathrm{{T}}}\) with its initial value \({{{{\hat{W}}}}_U} = {[7,3,50,5,4,5,15,1,0.5,1.5]^\mathrm{{T}}}\), the activation function is chosen as \({\varPhi _U} = {[e_{f1}^2,{e_{f1}}{e_{f2}},{e_{f1}}{e_{f3}},{e_{f1}}{e_{f4}}}{e_{f2}^2,{e_{f3}}{e_{f2}},{e_{f2}}{e_{f4}},e_{f3}^2,e_{f4}^2,}\) \({{e_{f3}}{e_{f4}}]^\mathrm{{T}}}\).
Case 1: VS system without uncertainties
We employ three different initial feature points to verify the visual tracking performance using the proposed ADP scheme. Moreover, it is assumed that the uncertainties can be neglected. The initial feature points are given by
As shown in Figs. 2, 3, 4 and 5, simulation results illustrate the trajectories of feature position, feature velocity, and feature tracking error, respectively. The visual tracking trajectories are illustrated in Fig. 2. It is observed that the actual feature trajectories can follow their desired ones with different initial feature points. The image tracking errors are displayed to illustrate the visual tracking performance intuitively in Fig. 3. We can see that the desired trajectory with the initial point \(f_2\) has a faster convergence rate than others. It is obvious that velocity trajectory curves of the VS manipulator systems are smooth and continuous except a slight oscillation at the beginning as shown in Fig. 4. Feature curves on the image plane are described in Fig. 5. From the feature tracking trajectories and their error curves, the VS system is performed to be asymptotically stable.
Case 2: VS system with uncertainties
To test the robustness of our proposed method, we consider a simple servoing task by introducing different uncertainties. Let the initial state be [460, 70, 0, 0], the initial observation state be [461, 69, 0, 0]. The uncertainties as constant vector and sinusoidal noise, which are given by
Simulation results are shown in Figs. 6, 7, 8 and 9. The velocity tracking trajectories still accompany the desired ones in both uncertain cases, which are shown in Figs. 6 and 8. From Figs. 7 and 9, the uncertainties can be estimated well within a short period of time by using the developed scheme.
To further exhibit the performance of the proposed ADP-based controller, the comparison results with adaptive neural network (ANN) scheme [47] and adaptive sliding mode control (ASMC) scheme [48] are also provided. The uncertainty is set as constant vector \(D_1\). The image feature position and their velocity trajectories of the proposed scheme, ANN scheme and ASMC scheme are illustrated in Figs. 10 and 11. The settling time of VS system under the proposed scheme (about 1.8 s) is longer than that under ANN scheme (about 0.4 s) and ASMC scheme (about 0.2 s). The image tracking errors of three methods are depicted in Fig. 12. It can be seen the error curve of the ASMC scheme has the fastest convergence rate in spite of an obvious fluctuation in Fig. 12b. Furthermore, the error curve of the ANN scheme has a small overshoot compared to our method. It is obvious that the responses of feature tracking exhibit no oscillations and overshoot, and smooth transient performance in Fig. 12c. The comparison of image feature position of three methods on the image plane are shown in Fig. 13. We can observe that the results of tracking in a complete circle period, which validates the accurate of our method. To quantize the tracking accuracy, three performance index functions, where \(E_\mathrm{{max}}\) denotes maximum value of absolute of image feature error, \(E_\mathrm{{min}}\) denotes minimum value of absolute of image feature error and MSE denotes mean-square-error (MSE) measure of image feature error, are defined as
After 2 s, the numerical quantitative comparison results of the proposed ADP-based scheme, ANN scheme and ASMC scheme are listed in Table 3. The significance underline points out the minimum value of the row in Table 3. It is clearly indicated that the proposed ADP-based scheme has a more precise tracking accuracy in contrast to other two methods.
It concludes that the proposed scheme can fulfill prominent tracking tasks by considering uncertainties.
Conclusion
In this paper, a feature tracking control scheme for VS manipulator with uncertainties based on ADP has been proposed. Under the effective estimation of uncertainties based on the adaptive NN observer, an improved cost function is designed to account for the influence of system uncertainties. The improved HJB equation is solved by a critic neural network, and the approximated optimal feature tracking error controller can be derived directly. Thus, the feature tracking controller is obtained by combining the optimal feature error controller and the desired controller. Moreover, the VS system is guaranteed to be UUB based on Lyapunov stability analysis. Simulation results illustrate the effectiveness of the proposed feature tracking control scheme. It is shown that the proposed controller is capable of controlling the VS manipulator which are regarded as highly nonlinear dynamic systems successfully.
In this study, we investigate the visual servoing control problem for manipulators subject to the unknown dynamics with energy cost optimization. In our future work, the dynamic control of manipulators with time delay, uncertainties in Jacobian matrix, and depth information, as well as VS control with image processing are potential research topics.
References
Wang H, Wang S, Zuo S (2020) Development of visible manipulator with multi-gear array mechanism for laparoscopic surgery. IEEE Robot Autom Lett 5(2):3090–3097
Arya KV, Bhadoria RS, Chaudhari NS (eds) (2018) Emerging wireless communication and network technologies: principle. Paradigm and performance. Springer, Singapore
Bhadoria RS, Chaudhari NS (2019) Pragmatic sensory data semantics with service-oriented computing. J Organ End User Comput 31(2):22–36
Dai H, Cao X, Jing X et al (2020) Bio-inspired anti-impact manipulator for capturing non-cooperative spacecraft: theory and experiment. Mech Syst Signal Proc 142:106785
Gao J, Zhang G, Wu P, Zhao X, Wang T, Yan W (2019) Model predictive visual servoing of fully-actuated underwater vehicles with a sliding mode disturbance observer. IEEE Access 7:25516–25526
Li W, Chiu PWY, Li Z (2020) An accelerated finite-time convergent neural network for visual servoing of a flexible surgical endoscope with physical and rcm constraints. IEEE Trans Neural Netw Learn Syst 31(12):5272–5284
Kang M, Chen H, Dong J (2020) Adaptive visual servoing with an uncalibrated camera using extreme learning machine and q-leaning. Neurocomputing 402(18):384–394
Li S, Ghasemi A, Xie W, Gao Y (2018) An enhanced IBVS controller of a 6-Dof manipulator using hybrid PD-SMC method. Int J Control Autom Syst 16:844–855
Sharma RS, Nair RR, Agrawal P, Behera L, Subramanian VK (2019) Robust hybrid visual servoing using reinforcement learning and finite-time adaptive fosmc. IEEE Syst J 13(3):3467–3478
Qiu Z, Hu S, Liang X (2019) Model predictive control for constrained image-based visual servoing in uncalibrated environments. Asian J Control 21(3):1–17
Wang H, Jiang M, Chen W, Liu Y (2012) Visual servoing of robots with uncalibrated robot and camera parameters. Mechatronics 22(6):661–668
Li T, Zhao H (2017) Global finite-time adaptive control for uncalibrated robot manipulator based on visual servoing. ISA Trans 68:402–411
Cheah CC, Liu C, Slotine J (2006) Adaptive Jacobian tracking control of robots with uncertainties in kinematic, dynamic and actuator models. IEEE Trans Autom Control 51(6):1024–1029
Hua C, Liu Y, Yang Y (2015) Image-based robotic control with unknown camera parameters and joint velocities. Robotica 33(8):1718–1730
Wang H (2015) Adaptive visual tracking for robotic systems without image-space velocity measurement. Automatica 55:294–301
Wang H, Liu YH, Chen W, Wang Z (2011) A new approach to dynamic eye-in-hand visual tracking using nonlinear observers. IEEE ASME Trans Mechatron 16(2):387–394
Leite AC, Lizarralde F (2016) Passivity-based adaptive 3d visual servoing without depth and image velocity measurements for uncertain robot manipulators. Int J Adapt Control Signal Process 30:1269–1297
Wang F, Liu Z, Chen CLP et al (2018) Adaptive neural network-based visual servoing control for manipulator with unknown output nonlinearities. Inf Sci 451:16–33
Zhang Y, Hua C, Li Y, Guan X (2019) Adaptive neural networks-based visual servoing control for manipulator with visibility constraint and dead-zone input. Neurocomputing 332:44–55
Zhao B, Liu D, Luo C (2019) Reinforcement learning-based optimal stabilization for unknown nonlinear systems subject to inputs with uncertain constraints. IEEE Trans Neural Netw Learn Syst 31(10):4330–4340
Zhao B, Liu D, Cesare A (2020) Sliding mode surface-based approximate optimal control for uncertain nonlinear systems with asymptotically stable critic structure. IEEE Trans Cybern 99:1–12
Zhao B, Luo F, Lin H, Liu D (2021) Particle swarm optimized neural networks based local tracking control scheme of unknown nonlinear interconnected systems. Neural Netw 134:54–63
Lin H, Zhao B, Liu D, Cesare A (2020) Data-based fault tolerant control for nonlinear systems through particle swarm optimized critic learning. IEEE/CAA J Autom Sin 7(4):954–964
Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy and adaptive approaches, chapter 13. Van Nostrand, New York
Liu D, Xue S, Zhao B, Luo B, Wei Q (2021) Adaptive dynamic programming for control: a survey and recent advances. IEEE Trans Syst Man Cybern 51(1):142–160
Kong L, Zhang S, Yu X (2020) Approximate optimal control for an uncertain robot based on adaptive dynamic programming. Neurocomputing 423(29):308–317
Zhang C, Zou W, Cheng N, Gao J (2017) Trajectory tracking control for rotary steerable systems using interval type-2 fuzzy logic and reinforcement learning. J Frankl Inst 355(2):803–826
Li Y, Chen L, Tee K, Li Q (2015) Reinforcement learning control for coordinated manipulation of multi-robots. Neurocomputing 170:168–175
Tang L, Liu Y, Tong S (2014) Adaptive neural control using reinforcement learning for a class of robot manipulator. Neural Comput Appl 25(1):135–141
Li S, Ding L, Gao H, Chen C, Liu Z, Deng Z (2017) Adaptive neural network tracking control-based reinforcement learning for wheeled mobile robots with skidding and slipping. Neurocomputing 283(29):20–30
Wang Z, Dai Y, Li Y (2011) Research of path planning based on adaptive dynamic programming for bio-mimetic robot fish. Int J Model Identif Control 13(3):144–151
Lian C, Xu X, Chen H, He H (2016) Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans Cybern 46(11):2484–2496
Zhan H, Huang D, Chen Z, Wang M, Yang C (2020) Adaptive dynamic programming-based controller with admittance adaptation for robot–environment interaction. Int J Adv Robot Syst 17(3):172988142092461
Li Y, Xia H, Bo Z (2018) Policy iteration algorithm based fault tolerant tracking control: an implementation on reconfigurable manipulators. J Electr Eng Technol 13(4):1739–1750
Zhao B, Liu D (2020) Event-triggered decentralized tracking control of modular reconfigurable robots through adaptive dynamic programming. IEEE Trans Ind Electron 6(4):3054–3064
Dong B, An T, Zhou F, Liu K, Li Y (2019) Decentralized robust zero-sum neuro-optimal control for modular robot manipulators in contact with uncertain environments: theory and experimental verification. Nonlinear Dyn 97(13):503–524
Al-Junaid H (2015) Ann based robotic arm visual servoing nonlinear system. Procedia Comput Sci 62:23–30
Li W, Ye G, Wan H, Zheng S, Lu Z (2015) Decoupled control for visual servoing with SVM-based virtual moments. In: IEEE international conference on information and automation. IEEE, pp 2121–2126
Oliveira TR, Leite AC, Peixoto AJ, Hsu L (2014) Overcoming limitations of uncalibrated robotics visual servoing by means of sliding mode control and switching monitoring scheme. Asian J Control 16(3):752–764
Li T, Zhao H, Chang Y (2018) Visual servoing tracking control of uncalibrated manipulators with a moving feature point. Int J Syst Sci 49(9–12):2410–2426
Liang X, Wang HS, Liu YH et al (2016) Adaptive task-space cooperative tracking control of networked robotic manipulators without task-space velocity measurements. IEEE Trans Cybern 46(10):2386–2398
Chaumette F, Hutchinson S (2006) Visual servo control, part i: basic approaches. IEEE Robot Autom Mag 13(4):82–90
Chaumette F (1998) Potential problems of stability and convergence in image-based and position-based visual servoing. In: Kriegman DJ, Hager GD, Morse AS (eds) The confluence of vision and control. Lecture notes in control and information sciences, vol 237. Springer, London
Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791
Siciliano B, Sciavicco L, Villani L, Oriolo G (2008) Robotics: modelling. Planning and control. Springer, London
Lafmejani HS, Zarabadipour H (2014) Modeling, simulation and position control of 3DOF articulated manipulator. Indones J Electr Eng Inform 2(3):132–140
Liu J (2013) Radial basis function (RBF) neural network control for mechanical systems. Springer, Berlin
Liu J, Wang X (2011) Advanced sliding mode control for mechanical systems, design. Analysis and matlab simulation. Springer, Berlin
Funding
Funding was provided by National Natural Science Foundation of China (61773075), Innovative Research Group Project of the National Natural Science Foundation of China (61703055), Jilin Scientific and Technological Development Program (20200801056GH, 20190103004JH), Department of Science and Technology of Jilin Province (CN) (JJKH20200672KJ, JJKH20200673KJ, JJKH20200674KJ).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ren, X., Li, H. Adaptive dynamic programming-based feature tracking control of visual servoing manipulators with unknown dynamics. Complex Intell. Syst. 8, 255–269 (2022). https://doi.org/10.1007/s40747-021-00367-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40747-021-00367-0