Abstract
Manipulators are widely used in various fields of industrial automation, and hydraulic manipulators have more extensive application in future by virtue of high power to weight ratio. However, compared with the traditional motor manipulators, the dynamic model of hydraulic manipulator is more complex such as higher model order characteristics, which makes the design of precision motion controller of hydraulic manipulators more challenging. In addition, the application demand of hydraulic manipulator is gradually increasing even under various harsh conditions or extreme environments, which puts forward higher requirements for the autonomous environmental perception ability of hydraulic manipulators. Therefore, the visual servoing control for hydraulic manipulator is of great research value. In this paper, a position-based visual servoing control method for multi-joint hydraulic manipulator is proposed. To be specific, based on the obtained target position, the discrete desired path points can be got by velocity-limited path interpolation, which will be transformed into a discrete sequence of desired joint angles according to well-constrained inverse kinematics. Then the continuous desired angle trajectory of manipulator joint is generated by B-spline trajectory planner. Finally, the ideal angle trajectory is input into backstepping controller to contrive the angle tracking of each joint, so as to realize the visual target following control of multi-joint hydraulic manipulator.
Similar content being viewed by others
Code Availability
The code used during the current study are available from the corresponding author on reasonable request.
References
Mohammad Hossein Fallah, M., Janabi-Sharifi, F.: Conjugated visual predictive control for constrained visual servoing. J. Intell. Robot. Syst 101(2), 33 (2021). https://doi.org/10.1007/s10846-020-01299-6https://doi.org/10.1007/s10846-020-01299-6
Mattila, J., Koivumki, J., Caldwell, D.G., Semini, C.: A survey on control of hydraulic robotic manipulators with projection to future trends. IEEE ASME Trans. Mechatron. 22(2), 669–680 (2017). https://doi.org/10.1109/TMECH.2017.2668604
Kornuta, T., Zieliński, C.: Robot control system design exemplified by multi-camera visual servoing. J. Intell. Robot. Syst 77(3), 499–523 (2015). https://doi.org/10.1007/s10846-013-9883-x
Sadeghzadeh, M., Calvert, D., Abdullah, H.A.: Self-learning visual servoing of robot manipulator using explanation-based fuzzy neural networks and q-learning. J. Intell. Robot. Syst 78(1), 83–104 (2015). https://doi.org/10.1007/s10846-014-0151-5
Shen, W., Wang, J.: An integral terminal sliding mode control scheme for speed control system using a double-variable hydraulic transformer. ISA Transactions. https://doi.org/10.1016/j.isatra.2019.08.068https://doi.org/10.1016/j.isatra.2019.08.068 (2019)
Li, H., Zhang, X.m., Zeng, L., Huang, Y.j.: A monocular vision system for online pose measurement of a 3rrr planar parallel manipulator. J. Intell. Robot. Syst 92(1), 3–17 (2018). https://doi.org/10.1007/s10846-017-0720-5
Yao, J., Deng, W.: Active disturbance rejection adaptive control of hydraulic servo systems. IEEE Trans. Ind. Electron. 64(10), 8023–8032 (2017). https://doi.org/10.1109/TIE.2017.2694382
Daniilidis, K., Bayro-Corrochano, E.: The dual quaternion approach to hand-eye calibration. In: Proceedings of 13th International Conference on Pattern Recognition, vol. 1, pp. 318–3221. https://doi.org/10.1109/ICPR.1996.546041(1996)
Ali, Z.A., Israr, A., Alkhammash, E.H., Hadjouni, M.: A leader-follower formation control of multi-uavs via an adaptive hybrid controller. Complexity 2021. https://doi.org/10.1155/2021/9231636https://doi.org/10.1155/2021/9231636 (2021)
Elsisi, M., Mahmoud, K., Lehtonen, M., Darwish, M.M.: An improved neural network algorithm to efficiently track various trajectories of robot manipulator arms. Ieee Access 9, 11911–11920 (2021). https://doi.org/10.1109/ACCESS.2021.3051807
Ali, Z.A., Xinde, L.: Modeling and controlling the dynamic behavior of an aerial manipulator. Fluctuation and Noise Letters, 2150044. https://doi.org/10.1142/S0219477521500449 (2021)
Bodie, K., Tognon, M., Siegwart, R.: Dynamic end effector tracking with an omnidirectional parallel aerial manipulator. IEEE Robot. Autom. Lett 6(4), 8165–8172 (2021). https://doi.org/10.1109/LRA.2021.3101864
Thelen, A., Frey, S., Hirsch, S., Hering, P.: Improvements in shape-from-focus for holographic reconstructions with regard to focus operators, neighborhood-size, and height value interpolation. IEEE Trans. Image Process. 18(1), 151–157 (2009). https://doi.org/10.1109/TIP.2008.2007049
Yao, J., Deng, W., Jiao, Z.: Rise-based adaptive control of hydraulic systems with asymptotic tracking. IEEE Trans. Autom. Sci. Eng. 14(3), 1524–1531 (2017). https://doi.org/10.1109/TASE.2015.2434393
Sun, X., Zhu, X., Wang, P., Chen, H.: A review of robot control with visual servoing. In: 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 116–121. https://doi.org/10.1109/CYBER.2018.8688060 (2018)
Keshmiri, M., Xie, W.F.: Visual servoing of a robotic manipulator using an optimized trajectory planning technique. In: 2014 IEEE 27th Canadian Conference on Electrical and Computer Engineering (CCECE), pp.1–6. https://doi.org/10.1109/CCECE.2014.6901078 (2014)
Ren, Y., Sun, H., Tang, Y., Wang, S.: Vision based object grasping of robotic manipulator. In: 2018 24th International Conference on Automation and Computing (ICAC), pp. 1–5. https://doi.org/10.23919/IConAC.2018.8749001 (2018)
Pan, W., Lyu, M., Hwang, K.S., Ju, M.Y., Shi, H.: A neuro-fuzzy visual servoing controller for an articulated manipulator. IEEE Access 6, 3346–3357 (2018). https://doi.org/10.1109/ACCESS.2017.2787738https://doi.org/10.1109/ACCESS.2017.2787738
Lin, C.Y., Hsieh, P.J., Chang, F.A.: Dsp based uncalibrated visual servoing for a 3-dof robot manipulator. In: 2016 IEEE International Conference on Industrial Technology (ICIT), pp. 1618–1621. https://doi.org/10.1109/ICIT.2016.7475003 (2016)
Zhan, G., Du, D., Wang, H.: Experimental analysis of networked visual servoing inverted pendulum system under noise attacks. In: 2018 IEEE 27th International Symposium on Industrial Electronics (ISIE), pp. 971–975. https://doi.org/10.1109/ISIE.2018.8433805 (2018)
Li, J., Huang, H., Xu, Y., Wu, H., Wan, L.: Uncalibrated visual servoing for underwater vehicle manipulator systems with an eye in hand configuration camera. Sensors 19(24). https://doi.org/10.3390/s19245469(2019)
Bae, S.H., Kim, E.J., Yang, S.J., Park, J.K., Kuc, T.Y.: A dynamic visual servoing of robot manipulator with eye-in-hand camera. In: 2018 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–4. https://doi.org/10.23919/ELINFOCOM.2018.8330640 (2018)
Burger, W., Dean-Leon, E., Cheng, G.: Robust second order sliding mode control for 6d position based visual servoing with a redundant mobile manipulator. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 1127–1132. https://doi.org/10.1109/HUMANOIDS.2015.7363494https://doi.org/10.1109/HUMANOIDS.2015.7363494 (2015)
Hwang, K.S., Lee, J.L., Hwang, Y.L., Jiang, W.C.: Image base visual servoing base on reinforcement learning for robot arms. In: 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 566–569. https://doi.org/10.23919/SICE.2017.8105453 (2017)
Zhang, Y., Li, S., Liao, B., Jin, L., Zheng, L.: A recurrent neural network approach for visual servoing of manipulators. In: 2017 IEEE International Conference on Information and Automation (ICIA), pp. 614–619. https://doi.org/10.1109/ICInfA.2017.8078981https://doi.org/10.1109/ICInfA.2017.8078981 (2017)
Wang, K., Ding, N., Dai, F.: Visual servoing based pickup of stationary objects with a dynamically controlled manipulator. In: 2017 IEEE International Conference on Industrial Technology (ICIT), pp. 902–907. https://doi.org/10.1109/ICIT.2017.7915479 (2017)
Helian, B., Chen, Z., Yao, B.: Precision motion control of a servomotor-pump direct-drive electrohydraulic system with a nonlinear pump flow mapping. IEEE Trans. Ind. Electron. 67(10), 8638–8648 (2019). https://doi.org/10.1109/TIE.2019.2947803
Bartlett, H.L., Lawson, B.E., Goldfarb, M.: Design, control, and preliminary assessment of a multifunctional semipowered ankle prosthesis. IEEE/ASME Transactions on Mechatronics 24(4), 1532–1540 (2019). https://doi.org/10.1109/TMECH.2019.2918685https://doi.org/10.1109/TMECH.2019.2918685
Cheng, M., Zhang, J., Xu, B., Ding, R., Wei, J.: Decoupling compensation for damping improvement of the electrohydraulic control system with multiple actuators. IEEE ASME Trans. Mechatron. 23(3), 1383–1392 (2018). https://doi.org/10.1109/TMECH.2018.2834936https://doi.org/10.1109/TMECH.2018.2834936
Hyon, S., Suewaka, D., Torii, Y., Oku, N.: Design and experimental evaluation of a fast torque-controlled hydraulic humanoid robot. IEEE ASME Trans. Mechatron. 22(2), 623–634 (2017). https://doi.org/10.1109/TMECH.2016.2628870
Lyu, L., Chen, Z., Yao, B.: Advanced valves and pump coordinated hydraulic control design to simultaneously achieve high accuracy and high efficiency. IEEE Transactions on Control Systems Technology, pp. 1–13. https://doi.org/10.1109/TCST.2020.2974180 (2020)
Ding, R., Cheng, M., Jiang, L., Hu, G.: Active fault-tolerant control for electro-hydraulic systems with an independent metering valve against valve faults. IEEE Trans. Ind. Electron. 68(8), 7221–7232 (2021). https://doi.org/10.1109/TIE.2020.3001808
Tripathi, A., Dasrath, D., Sun, Z., Northrop, W., Kittelson, D., Stelson, K.A.: Design and control of a controlled trajectory rapid compression and expansion machine. IEEE ASME Trans. Mechatron. 24(4), 1711–1722 (2019). https://doi.org/10.1109/TMECH.2019.2917820https://doi.org/10.1109/TMECH.2019.2917820
Huang, J., An, H., Yang, Y., Wu, C., Wei, Q., Ma, H.: Model predictive trajectory tracking control of electro-hydraulic actuator in legged robot with multi-scale online estimator. IEEE Access 8, 95918–95933 (2020). https://doi.org/10.1109/ACCESS.2020.2995701https://doi.org/10.1109/ACCESS.2020.2995701
Lyu, L., Chen, Z., Yao, B.: Development of pump and valves combined hydraulic system for both high tracking precision and high energy efficiency. IEEE Transactions on Industrial Electronics 66(9), 7189–7198 (2019). https://doi.org/10.1109/TIE.2018.2875666https://doi.org/10.1109/TIE.2018.2875666
Shen, W., Wang, J.: An integral terminal sliding mode control scheme for speed control system using a double-variable hydraulic transformer. ISA transactions. https://doi.org/10.1016/j.isatra.2019.08.068https://doi.org/10.1016/j.isatra.2019.08.068 (2019)
Chen, Z., Huang, F., Sun, W., Gu, J., Yao, B.: Rbf-neural-network-based adaptive robust control for nonlinear bilateral teleoperation manipulators with uncertainty and time delay. IEEE ASME Trans. Mechatron. 25(2), 906–918 (2020). https://doi.org/10.1109/TMECH.2019.2962081
Sivcev, S., Rossi, M., Coleman, J., Dooly, G., Omerdic, E., Toal, D.: Fully automatic visual servoing control for work-class marine intervention rovs. Control Eng. Pract. 74, 153–167 (2018). https://doi.org/10.1016/j.conengprac.2018.03.005
Chen, Z., Huang, F., Chen, W., Zhang, J., Sun, W., Chen, J., Gu, J., Zhu, S.: Rbfnn-based adaptive sliding mode control design for delayed nonlinear multilateral telerobotic system with cooperative manipulation. IEEE Trans. Industr. Inform. 16(2), 1236–1247 (2020). https://doi.org/10.1109/TII.2019.2927806
Coquillart, S.: Computing offsets of b-spline curves. Computer-Aided Design 19(6), 305–309 (1987). https://doi.org/10.1016/0010-4485(87)90284-3https://doi.org/10.1016/0010-4485(87)90284-3
Wang, M., Li, X., Xu, K., Jiang, R.: Smooth trajectory planning for manipulator of cotton harvesting machinery based on quaternion and b-spline. In: 2012 International Symposium on Instrumentation Measurement, Sensor Network and Automation (IMSNA), vol. 1, pp. 134–137. https://doi.org/10.1109/MSNA.2012.6324531 (2012)
Liang, F., Yan, G., Fang, F.: Global time-optimal b-spline feedrate scheduling for a two-turret multi-axis nc machine tool based on optimization with genetic algorithm. Robotics and Computer-Integrated . Manufacturing 75, 102308 (2022). https://doi.org/10.1016/j.rcim.2021.102308
Funding
This work is supported by National Natural Science Foundation of China (No.52075476), Hainan Provincial National Natural Science Foundation of China (No. 521MS065), and Key R&D Program of Zhejiang Province (No.2021C03013).
Author information
Authors and Affiliations
Contributions
Conceptualization: Zheng Chen, Shizhao Zhou; Methodology: Shizhao Zhou, Chong Shen, Fengye Pang; Formal analysis and investigation: Shizhao Zhou, Chong Shen, Fengye Pang; Writing original draft preparation: Shizhao Zhou, Chong Shen; Writing - review and editing: Shizhao Zhou, Chong Shen, Fengye Pang; Funding acquisition: Zheng Chen; Resources: Zheng Chen, Jason Gu, Shiqiang Zhu; Supervision: Zheng Chen, Jason Gu, Shiqiang Zhu.
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Availability of data and material
The datasets used or analysed during the current study are available from the corresponding author on reasonable request.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix : A: Proof for Theorem
Appendix : A: Proof for Theorem
Proof Proof for Theorem I
The Lyapunov function is defined as:
Combine (??),(??), (??), (??) and Eq. A1 the differential of V2 can be simplified as:
Combine (??), (??), (??) and (A2), the differential of V3 can be simplified as:
According to the Condition III, (A4) can be convert to inequality form as follow:
\({{ {{{\dot {V}}}_{2}} \rvert }_{{{z}_{3}}=0}}\)can be written as:
According to the Condition I, (A6) can be convert to inequality form as follow:
where \({{\eta }_{2}}=2{{M}_{j}}^{-1}{{k}_{2}}\), the following equation can be obtained by solving (A8) in time domain:
According to Eq. A9, \({{ {{V}_{2}} \rvert }_{{{z}_{3}}=0}}\le \frac {{{\zeta }_{2}}}{{{\eta }_{2}}}\), and \({{ {{V}_{2}} \rvert }_{{{z}_{3}}=0}}\to 0\) as \(t\to \infty \).
Combined (A6) and Eq. A9, the following inequality of \({{\dot {V}}_{3}}\) can be obtained the as:
where η3 = 2k3, the following equation can be obtained by solving (A8) in time domain:
where \(\frac {{{\zeta }_{c}}}{{{\eta }_{c}}}=\frac {{{\zeta }_{2}}}{{{\eta }_{2}}}+\frac {{{\zeta }_{3}}}{{{\eta }_{3}}}\).
According to Eq. A11, \({{V}_{3}}\le \frac {{{\zeta }_{c}}}{{{\eta }_{c}}}\), and V3 → 0 as \(t\to \infty \).
It can be proved that V3 is ultimately bounded to \(\frac {{{\zeta }_{c}}}{{{\eta }_{c}}}\) and z3 is bounded. On the premise that z3 is bounded, according to eq.49, V2 is ultimately bounded to \(\frac {{{\zeta }_{2}}}{{{\eta }_{2}}}\) and z2 is bounded.
In addition, by frequency domain transformation of Eq. ??, the conversion function L(s) between z1 and z2 can be obtained as follows:
According to Eq. ??, z1 is bounded on the premise that z2 is bounded. The proof for Theorem I is complete. □
Proof Proof for Theorem II
When disturbance error, Δ = 0, gets zero and the dynamic model parameters are accurate (\(\tilde {\theta }=0\)), the differential of Lyapunov function such as Eqs. A2 and A4 can be rewritten as:
According to Condition II, the upper bound of Eq. 13 can be expressed as:
According to Condition IV, (14) can be rewritten as:
Thus, the asymptotic output tracking of Step II in the process of control law design is achieved, i.e., z3 → 0 as \(t\to \infty \). And on this premise, the asymptotic output tracking of Step I is also achieved.
That is,
as \(t\to \infty \).
And according to Eq. ??, z1 → 0 can be achieved on the premise of that z2 → 0 as \(t\to \infty \). The proof for Theorem 2 is complete. □
Rights and permissions
About this article
Cite this article
Zhou, S., Shen, C., Pang, F. et al. Position-Based Visual Servoing Control for Multi-Joint Hydraulic Manipulator. J Intell Robot Syst 105, 33 (2022). https://doi.org/10.1007/s10846-022-01628-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-022-01628-x