Abstract
As flexibility becomes an important factor in factory automation, the bin-picking system, where a robot performs pick-and-place tasks for randomly piled parts in a bin through measuring the 3D pose of an object by a 3D vision sensor, has been actively studied. However, conventional bin-picking systems that are employed for particular tasks are limited by such things as the FOV (Field of View), the shape of landmark features, and computation time. This paper proposes a general-purpose stereo vision based bin-picking system. To detect the workpiece to be picked, a geometric pattern matching (GPM) method with respect to the 2D image with a wide FOV is applied. The accurate 3D pose of a selected workpiece among the pick-up candidates is acquired by measuring the 3D positions of three features in the workpiece using the stereo camera. In order to improve the 3D position estimation performance, the GPM method is also used instead of the stereo matching method. The multiple pattern registration and ellipse fitting techniques are additionally applied to increase the reliability. The grasp position of a workpiece without collision is determined using the pose of the object and the bin information. By using these methods a practical bin-picking strategy is established to operate robustly with minimum help from the human workers in the factory. Through experiments on commercial industrial workpieces and industrial robot, we validated that the proposed vision system accurately measures the 3D pose of part and the robot successfully manipulates the workpiece among randomly stacked parts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
B. Habibi, “Single camera 3D™(SC3D™) a novel technology for guidance of industrial robots in three-dimensional space,” Braintech, 2002.
J.-K. Oh, C.-H. Lee, S.-H. Lee, J.-S. Hur, and S. Lee “Development of a 3D vision guided robotic system using stereo vision,” Proc. of the 39th International Symposium on Robotics, pp. 2–7, October 2008.
W. Hardin, “Vision enables freestyle bin-picking,” Vision System Design, Jun. 2007.
W. Iversen, “Vision-guided robotics: in search of the holy grail,” Automation World, February 2006.
K. Ban, F. Warashina, I. Kanno, and H. Kumiya, “Industrial intelligent robot,” FANUC Tech. Rev., vol. 16, no. 2, pp. 29–34, 2003.
K. Rahardja and A. Kosaka, “Vision-based bin-picking: recognition and localization of multiple complex objects using simple visual cues,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1448–1457, Nov. 1996.
M. Berger, G. Bachler, and S. Scherer, “Vision guided bin picking and mounting in a flexible assembly cell,” Proc. of the 13th International Conference on Industrial & Engineering Applications of Artificial Intelligence and Expert Systems, pp. 109–118, 2000.
Point Grey Research Inc., http://www.ptgrey.com
Videre Design LLC, http://www.videredesign.com
R. D. Schraft and T. Ledermann, “Intelligent picking of chaotically stored objects,” Assembly Automation, vol. 23, no. 1, pp. 38–42, 2003.
J.-K. Oh and C.-H. Lee, “Development of a stereo vision based position measurement algorithm for the randomly piled objects,” Proc. of International Conference on Advanced Robotics, pp. 64–69, August 2007.
C. R. Hema, M. P. Paulraj, R. Nagarajan, and Y. Sazali, “Segmentation and location computation of bin objects,” International Journal of Advanced Robotic Systems, vol. 4, no. 1, pp. 57–62, 2007.
Cognex Corporation, “PatMax and PatQuick,” Cognex, 2005.
D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” International Journal of Computer Vision, vol. 47, no. 1/2/3, pp. 7–42, 2002.
F. Boughorbel, Y. Zhang, S. Kang, U. Chidambaram, B. Abidi, A. Koschan, and M. Abidi, “Laser ranging and video imaging for bin picking,” Assembly Automation, vol. 23, no. 1, pp. 53–59, 2003.
F. Blais, “Review of 20 years of range sensor development,” Journal of Electronic Imaging, vol. 13, no. 1, pp. 231–243, 2004.
R. Hartley and A. Zisserman, “Multiple view geometry in computer vision,” Cambridge Press, 2000.
C. J. Harris and M. Stephens, “A combined corner and edge detector,” Proc. 4th Alvey Vision Conf., Manchester, pp. 147–151, 1988.
R. Horaud and F. Dornaika, “Hand-eye calibration,” Int. Journal of Robotics Research, vol. 14, no. 3, pp. 195–210, 1995.
Hyundai Heavy Industry Co., Ltd., Hi5 Robot Controller Manual, Hyundai Heavy Industry, 2008.
P. Mittrapiyanuruk, G. N. DeSouza, and A. C. Kak, “Calculating the 3D-pose of rigid objects using active appearance models,” Proc. of the Int. Conference in Robotics and Automation, pp. 5147–5152, 2004.
D.-J. Kang, J.-E. Ha, and M.-H. Jeong, “Detection of calibration patterns for camera calibration with irregular lighting and complicated backgrounds,” International Journal of Control, Automation, and Systems, vol. 6, no. 5, pp. 746–754, 2008.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Editorial Board member Kang-Hyun Jo under the direction of Editor Hyouk Ryeol Choi.
This work was supported in part by the grant from the Industrial Source Technology Development Program of, in part by the Intelligent Robot Program under the Frontier R&D Initiative of, also in part by the ITRC Program NIPA-2012-(C1090-1221-0008) of MKE, Korea, as well as in part by WCU Program (R31-10062-0) and PRCP (2011-0018397) of NRF / MEST, Korea.
Jong-Kyu Oh received his B.S. and M.S. degrees in Electronic Engineering from Pusan National University in 1998 and 2000, respectively. He received his Ph.D. degree in Electrical and Computer Engineering from Sungkyunkwan University in 2012. Since 2000, he has been worked with Hyundai Heavy Industries Co., Ltd. His research interests include 3D vision, robot vision, and medical robot.
Sukhan Lee received his B.S. and M.S. degrees in Electrical Engineering from Seoul National University in 1972 and 1974, respectively. He received his Ph.D. degree in Electrical Engineering from Purdue University, West Lafayette in 1982. From 1983 to 1997, he was with the Departments of Electrical Engineering and of Computer Science at the University of Southern California and, from 1990 to 1997, with the Jet Propulsion Laboratory, California Institute of Technology, as a Senior Member of Technical Staff. From 1998 to 2003, he was an Executive Vice President and a Chief Research Officer at the Samsung Advanced Institute of Technology. Since 2003, he has been a Professor of Information and Communication Engineering and WCU professor of Interaction Science at the Sungkyunkwan University, while serving as the Director of the Intelligent Systems Research Institute. Since 2011, he assumed the Dean of the Graduate School of Sungkyunkwan University. Prof. Sukhan Lee has his research interest in the areas of Cognitive Robotics, Intelligent Systems, and Micro/Nano Electro-Mechanical systems. He is currently a fellow of IEEE and a fellow of Korean National Academy of Science and Technology.
Chan-Ho Lee received his B.S. and M.S. degrees in Electronic Engineering from Hanyang University in 1986 and 1991, respectively. Since 1991, he has been worked with Hyundai Heavy Industries Co., Ltd. His research interests include 3D vision, robot vision, and medical robot.
Rights and permissions
About this article
Cite this article
Oh, JK., Lee, S. & Lee, CH. Stereo vision based automation for a bin-picking solution. Int. J. Control Autom. Syst. 10, 362–373 (2012). https://doi.org/10.1007/s12555-012-0216-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-012-0216-9