Abstract
This research aims to develop a vision sensor system and a recognition algorithm to enable a humanoid to operate autonomously in a disaster environment. In disaster response scenarios, humanoid robots that perform manipulation and locomotion tasks must identify the objects in the environment from those challenged by the call by the United States’ Defense Advanced Research Projects Agency, e.g., doors, valves, drills, debris, uneven terrains, and stairs, among others. In order for a humanoid to undertake a number of tasks, we construct a camera–laser fusion system and develop an environmental recognition algorithm. Laser distance sensor and motor are used to obtain 3D cloud data. We project the 3D cloud data onto a 2D image according to the intrinsic parameters of the camera and the distortion model of the lens. In this manner, our fusion sensor system performs functions such as those performed by the RGB-D sensor generally used in segmentation research. Our recognition algorithm is based on super-pixel segmentation and random sampling. The proposed approach clusters the unorganized cloud data according to geometric characteristics, namely, proximity and co-planarity. To assess the feasibility of our system and algorithm, we utilize the humanoid robot, DRC-HUBO, and the results are demonstrated in the accompanying video.
Similar content being viewed by others
References
D. Comaniciu and P. Meer, Robust analysis of feature spaces: Color image segmentation, Proceedings of 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE (1997).
S. C. Zhu and A. Yuille, Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 18 (9) (1996) 884–900.
B. Douillard et al., On the segmentation of 3D LIDAR point clouds, 2011 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2011).
H. Woo et al., A new segmentation method for point cloud data, International Journal of Machine Tools and Manufacture, 42 (2) (2002) 167–178.
D. Holz et al., Real-time plane segmentation using RGB-D cameras, RoboCup 2011: Robot Soccer World Cup XV, Springer Berlin Heidelberg (2012) 306–317.
U. Asif, M. Bennamoun and F. Sohel, A model-free approach for the segmentation of unknown objects, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), IEEE (2014).
L. Cruz, D. Lucio and L. Velho, Kinect and rgbd images: Challenges and applications, Graphics, 25th SIBGRAPI Conference on Patterns and Images Tutorials (SIBGRAPI-T), IEEE (2012).
J. Han et al., Enhanced computer vision with microsoft kinect sensor: A review, IEEE Transactions on Cybernetics, 43 (5) (2013) 1318–1334.
S. Izadi et al., KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera, Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, ACM (2011).
P. Henry et al., RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments, The International Journal of Robotics Research, 31 (5) (2012) 647–663.
J. Owens, Object detection using the Kinect. No. ARL-TN-0474, Army Research Lab Aberdeen Proving Ground MD Vehicle Technology Directorate (2012).
O. Wulf and B. Wagner, Fast 3D scanning methods for laser measurement systems, International Conference on Control Systems and Computer Science (CSCS14) (2003).
R. Unnikrishnan and M. Hebert, Fast extrinsic calibration of a laser rangefinder to a camera, Carnegie Mellon University (2005).
I. Lee and J.-H. Oh, Humanoid posture selection for reaching motion and a cooperative balancing controller, Journal of Intelligent & Robotic Systems (2015) 1-16.
J. Bohg and D. Kragic, Learning grasping points with shape context, Robotics and Autonomous Systems, 58 (4) (2010) 362–377.
R. B. Rusu et al., Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009), IEEE (2009).
R. B. Rusu et al., Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009), IEEE (2009).
Z.-C. Marton et al., General 3D modelling of novel objects from a single view, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE (2010).
D. Holz et al., Real-time plane segmentation using RGB-D cameras, RoboCup 2011: Robot Soccer World Cup XV, Springer Berlin Heidelberg (2012) 306–317.
J. J. Koenderink and A. J. van Doorn, The internal representation of solid shape with respect to vision, Biological Cybernetics, 32 (4) (1979) 211–216.
D. G. Lowe, Local feature view clustering for 3D object recognition, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001 (CVPR 2001), IEEE, 1 (2001).
J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (8) (2000) 888–905.
R. Schnabel, R. Wahl and R. Klein, Efficient RANSAC for point-cloud shape detection, Computer Graphics Forum, Blackwell Publishing Ltd., 26 (2) (2007).
J. Papon et al., Voxel cloud connectivity segmentationsupervoxels for point clouds, 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2013).
U. Asif, M. Bennamoun and F. Sohel, A model-free approach for the segmentation of unknown objects, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), IEEE (2014).
G. Kootstra, N. Bergstrom and D. Kragic, Fast and automatic detection and segmentation of unknown objects, 2010 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids), IEEE (2010).
N. Bergstrom, M. Bjorkman and D. Kragic, Generating object hypotheses in natural scenes through human-robot interaction, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE (2011).
R. Achanta et al., SLIC superpixels compared to state-of-theart superpixel methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34 (11) (2012) 2274–2282.
Z. Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11) (2000) 1330–1334.
J. Heo and J. Oh, Biped walking pattern generation using an analytic method for a unit step with a stationary time interval between steps, IEEE Transactions on Industrial Electronics, 62 (2) (2015) 1091–1100.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Deok Jin Lee
Inho Lee received his B.S., M.S., and Ph.D. degrees in Mechanical Engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea, in 2009, 2011 and 2016, respectively. He worked on the development of humanoid robots through the following projects: HUBO, HUBO2, and DRC-HUBO. He is currently working in Institute for Human and Machine Cognition Robotics Lab and his research interests include motion planning, quadruped and bipedal walking, and stabilization control for humanoid robots, manipulation, sensors, actuators, and microprocessor applications.
Jaesung Oh received his B.S. degree in Mechanical and Control Engineering from the Handong Global University, Pohang, South Korea, and his M.S. degree in Mechanical Engineering from KAIST in 2013 and 2015, respectively. He is currently a doctorate student of the Department of Mechanical Engineering, KAIST. His research interests include humanoid robot, as well as the applications of robotics and control.
Inhyeok Kim received his B.S. degree in Mechanical Engineering from Yonsei University, Seoul, South Korea, in 2004. He received his M.S. and Ph.D. degrees in Mechanical Engineering from KAIST, Daejeon, South Korea, in 2006 and 2013, respectively. He is currently working in Naver Labs and his research interests include humanoid robot and whole-body inverse kinematics, motor control, and microprocessor applications.
Junho Oh received his B.S. and M.S. degrees in Mechanical Engineering from Yonsei University, Seoul, South Korea, and his Ph.D. degree in Mechanical Engineering from the University of California, Berkeley, in 1977, 1979, and 1985, respectively. Since 1985, he has been with the Department of Mechanical Engineering, KAIST, where he is currently a Professor and a Director of the Humanoid Robot Research Center. His research interests include humanoid robots, sensors, actuators, and applications of microprocessors. He is a member of the IEEE, KSME, KSPE, and ICROS.
Rights and permissions
About this article
Cite this article
Lee, I., Oh, J., Kim, I. et al. Camera-laser fusion sensor system and environmental recognition for humanoids in disaster scenarios. J Mech Sci Technol 31, 2997–3003 (2017). https://doi.org/10.1007/s12206-017-0543-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12206-017-0543-0