Abstract
Aiming at satisfying the increasing demand of family service robots for housework, this paper proposes a robot visual servoing scheme based on the randomized trees to complete the visual servoing task of unknown objects in natural scenes. Here, “unknown” means that there is no prior information on object models, such as template or database of the object. Firstly, an object to be manipulated is randomly selected by user prior to the visual servoing task execution. Then, the raw image information about the object can be obtained and used to train a randomized tree classifier online. Secondly, the current image features can be computed using the well-trained classifier. Finally, the visual controller can be designed according to the error of image feature, which is defined as the difference between the desired image features and current image features. Five visual positioning of unknown objects experiments, including 2D rigid object and 3D non-rigid object, are conducted on a MOTOMAN-SV3X six degree-of-freedom (DOF) manipulator robot. Experimental results show that the proposed scheme can effectively position an unknown object in complex natural scenes, such as occlusion and illumination changes. Furthermore, the developed robot visual servoing scheme has an excellent positioning accuracy within 0.05 mm positioning error.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Change history
07 December 2022
A Correction to this paper has been published: https://doi.org/10.1007/s10846-022-01775-1
References
Huebner, K., Welke, K., Przybylski, M., Vahrenkamp, N., Asfour, T., Kragic, D., Dillmann, R.: Grasping known objects with humanoid robots: a box-based approach. Proc. of the IEEE Int. Conf. on Adv Robot. (2009)
Rasolzadeh, B., Björkman, M., Huebner, K., Kragic, D.: An active vision system for detecting, fixating and manipulating objects in real world. Int. J. Robot. Res. 29(2–3), 133–154 (2010)
Azad, P., Asfour, T., Dillmann, R.: Stereo-based 6D object localization for grasping with humanoid robot systems. Proc. of the IEEE/RSJ Int. Conf. on Intell Robots Sys (IROS). (2007)
Xin, J., Liu, D., Xu, Q.K.: LS-SVR-based robot uncalibrated 4DOF visual positioning. Contr Theory Appl. 27(1), 77–85 (2010)
Hauck, A., Ruttinger, J., Sorg, M., Farber, G.: Visual determination of 3D grasping points on unknown objects with a binocular camera system. Proc. of the IEEE/RSJ Int. Conf. on Intell Robots Sys (IROS). 272–278 (2009)
Kim, J., Cauli, N., Vicente, P., Damas, B., Bernardino, A., Santo Victor, J., Cavallo, F.: Cleaning tasks knowledge transfer between heterogeneous robots: a deep learning approach. J. Intell. Robot. Syst. 98(1), 191–205 (2019)
Reitelshöfer, S., Meister, S., Franke, J.: Recognition and description of unknown everyday objects by using an image based meta-search engine for service robots. Adv Engin Forum. 19(2), 132–138 (2016)
Schiffer, S.: Integrating Qualitative Reasoning and Human-Robot Interaction for Domestic Service Robots. PhD Thesis, RWTH Aachen University (2015)
Zhao, Z.: Towards 3D Reconstruction and Semantic Mapping for Indoor Scenes. PhD Thesis. University of Science and Technology of China (2016)
Dune, C., Remazeilles, A., Marchand, E., Leroux, C.: Vision-based grasping of unknown objects to improve disabled people autonomy. Proc. of Robotics: Sci Syst. (2008)
Gratal, X., Romero, J., Bohg, J., Kragic, D.: Visual servoing on unknown objects. Mechatronics. 22(4), 423–435 (2012)
Wang, H., Yang, B., Wang, J., Liang, X.W., Chen, W.D., Liu, Y.-H.: Adaptive visual servoing of contour features. IEEE/ASME Transac Mechatro. 23(2), 811–822 (2018)
Wang, Z., Hirai, S.: Modeling and estimation of rheological properties of food products for manufacturing simulations. J. Food Eng. 102(2), 136–144 (2011)
Higashimori, M., Yoshimoto, K., Kaneko, M.: Active shaping of an unknown rheological object based on deformation decomposition into elasticity and plasticity. Proc. IEEE Int. Conf. Robotics Automat. 5120–5126 (2010)
Shibata, M., Hirai, S.: Soft object manipulation by simultaneous control of motion and deformation. Proc. IEEE Int. Conf. Robotics Automat. 2460–2465 (2006)
Tokumoto, S., Hirai, S.: Deformation control of rheological food dough using a forming process model, in proc. IEEE Int. Conf. Robotics Automat. 2, 1457–1464 (2002)
Das, J., Sarkar, N.: Autonomous shape control of a deformable object by multiple manipulators. J. Intell. Robot. Syst. 62, 3–27 (2001)
Wu, H., Andersen, T.T., Andersen, N.A., Ravn, O.: Application of visual servoing for grasping and placing operation in slaughterhouse. Proc. of the Int. Conf. on Control, Autom Robot. 457–462 (2017)
Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. Proc. of the IEEE Int. Conf. on Robot Autom (ICRA). 316–1322 (2015)
Zhong, X.G., Xu, M., Zhong, X.Y., Peng, X.F.: A robot grasping discrimination approach based on multimode feature deep learning. J Autom. 42(7), 1022–1029 (2016)
Navarro-Alarcon, D., Liu, Y., Romero, J., Li, P.: Model Free Visually Served Deformation Control of Elastic Objects by Robot Manipulators. IEEE Trans. Robot. (2013)
Navarro-Alarcon, D., Liu, Y., Romero, J., Li, P.: Visually served deformation control by robot manipulators. Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA). 21(1), 5239–5244 (2013)
Berenson,D.: Manipulation of deformable objects without modeling and simulating deformation, IEEE/RSJ Int. Conf. on Intell Robots Syst (2013)
Berry, F., Martinet, P., Gallice, I.: Real time visual servoing around a complex object. IEICE Trans. Inf. Syst. 83(7), 1358–1368 (2000)
Ma, Y., Liu, X., Zhang, J., et.al. Robotic grasping and alignment for small size components assembly based on visual servoing. Int. J. Adv. Manuf. Technol. 106(11–12):4827–4843 (2020)
Li, J.A., Xie, H., Ma, R., Low, K.H.: Output feedback image-based visual servoing of rotorcrafts. J. Intell. Robot. Syst. 93(1), 277–287 (2019)
Ren, S., Girshick, R., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transac Pattern Anal & Mach Intell. 39(6), 1137–1149 (2017)
Yin, Y., Li, H., Fu, W.: Faster-YOLO: an accurate and faster object detection method. Digit Signal Proc. 102(7), 102756 (2020)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grants 61833013, 61873200 and U20A20225, and the Natural Sciences and Engineering Research Council of Canada. The authors would like to express their sincere gratitude to the Editor-in-Chief, the Associate Editor, and all the anonymous reviewers whose insightful comments that have helped to improve the quality and presentation of this paper considerably.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: In this article the affiliation 'Department of Mechanical, Industrial and Aerospace Engineering, Concordia University, Montreal, QC H3G 1M8, Canada' for Author Youmin Zhang was missing. The original article has been corrected.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Xin, J., Dong, C., Zhang, Y. et al. Visual Servoing of Unknown Objects for Family Service Robots. J Intell Robot Syst 104, 10 (2022). https://doi.org/10.1007/s10846-021-01516-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-021-01516-w