Abstract
This paper proposes a depth measurement error model for consumer depth cameras such as the Microsoft Kinect, and a corresponding calibration method. These devices were originally designed as video game interfaces, and their output depth maps usually lack sufficient accuracy for 3D measurement. Models have been proposed to reduce these depth errors, but they only consider camera-related causes. Since the depth sensors are based on projector-camera systems, we should also consider projector-related causes. Also, previous models require disparity observations, which are usually not output by such sensors, so cannot be employed in practice. We give an alternative error model for projector-camera based consumer depth cameras, based on their depth measurement algorithm, and intrinsic parameters of the camera and the projector; it does not need disparity values. We also give a corresponding new parameter estimation method which simply needs observation of a planar board. Our calibrated error model allows use of a consumer depth sensor as a 3D measuring device. Experimental results show the validity and effectiveness of the error model and calibration procedure.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Zhang, Z. Microsoft Kinect sensor and its effect. IEEE Multimedia Vol. 19, No. (2), 4–10, 2012.
Han, J.; Shao, L.; Xu, D.; Shotton, J. Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Transactions on Cybernetics Vol. 43, No. (5), 1318–1334, 2013.
Smisek, J.; Jancosek, M.; Pajdla, T. 3D with Kinect. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 1154–1160, 2011.
Herrera, D.; Kannala, J.; Heikkilä, J. Joint depth and color camera calibration with distortion correction. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. (10), 2058–2064, 2012.
Yamazoe, H.; Habe, H.; Mitsugami, I.; Yagi, Y. Easy depth sensor calibration. In: Proceedings of the 21st International Conference on Pattern Recognition, 465–468, 2012.
Raposo, C.; Barreto, J. P.; Nunes, U. Fast and accurate calibration of a Kinect sensor. In: Proceedings of the International Conference on 3D Vision, 342–349, 2013.
Xiang, W.; Conly, C.; McMurrough, C. D.; Athitsos, V. A review and quantitative comparison of methods for Kinect calibration. In: Proceedings of the 2nd International Workshop on Sensor-based Activity Recognition and Interaction, Article No. 3, 2015.
Darwish, W.; Tang, S.; Li, W.; Chen, W. A new calibration method for commercial RGB-d sensors. Sensors Vol. 17, No. 6, 1204, 2017.
Weiss, A.; Hirshberg, D.; Black, M. J. Home 3D body scans from noisy image and range data. In: Proceedings of the International Conference on Computer Vision, 1951–1958, 2011.
Jin, B.; Lei, H.; Geng, W. Accurate intrinsic calibration of depth camera with cuboids. In: Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, Vol. 8693. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer, Cham, 788–803, 2014.
Di Cicco, M.; Iocchi, L.; Grisetti, G. Nonparametric calibration for depth sensors. Robotics and Autonomous Systems Vol. 74, 309–317, 2015.
Teichman, A.; Miller, S.; Thrun, S. Unsupervised intrinsic calibration of depth sensors via SLAM. Robotics: Science and Systems Vol. 248, 3, 2013.
Wang, H.; Wang, J.; Liang, W. Online reconstruction of indoor scenes from RGB-D streams. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3271–3279, 2016.
Park, J.-H.; Shin, Y.-D.; Bae, J.-H.; Baeg, M.-H. Spatial uncertainty model for visual features using a Kinect™ sensor. Sensors Vol. 12, No. (7), 8640–8662, 2012.
Nguyen, C. V.; Izadi, S.; Lovell, D. Modeling Kinect sensor noise for improved 3D reconstruction and tracking. In: Proceedings of the 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, 524–530, 2012.
Zhang, Z. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 22, No. (11), 1330–1334, 2000.
Khoshelham, K.; Elberink, S. O. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors Vol. 12, No. (2), 1437–1454, 2012.
Heikkila, J. Geometric camera calibration using circular control points. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 22, No. (10), 1066–1077, 2000.
Dal Mutto, C.; Zanuttigh, P.; Cortelazzo, G. M. Time-of-Flight Cameras and Microsoft Kinect™. Springer Science & Business Media, 2012.
Freedman, B.; Shpunt, A.; Arieli, Y. Distance-varying illumination and imaging techniques for depth mapping. U.S. Patent 8,761,495. 2014.
Acknowledgements
This work was supported by the JST CREST “Behavior Understanding based on Intention-Gait Model” project.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Hirotake Yamazoe received his B.E., M.E., and Ph.D. degrees in engineering from Osaka University in 2000, 2002, and 2005, respectively. He was with the Advanced Telecommunications Research Institute International (ATR) during 2005–2011, the Institute of Scientific and Industrial Research, Osaka University during 2011–2012, and Osaka School of International Public Policy, Osaka University during 2012–2015. He is now a lecturer in the College of Information Science and Engineering, Ritsumeikan University. His research interests include computer vision and wearable computing. He is a member of IEICE, IPSJ, ITE, HIS, IEEE, and ACM.
Hitoshi Habe received his B.E. and M.E. degrees in electrical engineering and D.Info. degree in intelligence science and technology from Kyoto University, Japan, in 1997, 1999, and 2006, respectively. After working for Mitsubishi Electric Corporation from 1999 to 2002, Kyoto University from 2002 to 2006, Nara Institute of Science and Technology from 2006 to 2011, and Osaka University from 2011 to 2012, he is now a lecturer in the Department of Informatics, Kindai University, Japan. From 2010 to 2011, he was a visiting researcher at the Department of Engineering, University of Cambridge, UK. His research interests include computer vision, pattern recognition, and image processing. He is a member of IEEE, ACM, IEICE, IPSJ, and JSAI.
Ikuhisa Mitsugami received his B.S. degree in engineering from Kyoto University in 2001, and M.S. and Ph.D. degrees in engineering from Nara Institute of Science and Technology in 2003 and 2007, respectively. He then started working for the Academic Center for Computing and Media Studies, Kyoto University, and in 2010 became an assistant professor of the Institute of Scientific and Industrial Research, Osaka University. He is currently an associate professor of the Graduate School of Information Sciences, Hiroshima City University. His research interests are computer vision, human interfaces, and human behavior analysis. He is a member of IEEE, IEICE, IPSJ, and VRSJ.
Yasushi Yagi is the Executive Vice President of Osaka University. He received his Ph.D. degree from Osaka University in 1991. In 1985, he joined the Product Development Laboratory, Mitsubishi Electric Corporation, where he worked on robotics and inspection. He became a research associate in 1990, a lecturer in 1993, an associate professor in 1996, and a professor in 2003 at Osaka University. He was also director of the Institute of Scientific and Industrial Research, Osaka University from 2012 to 2015. International conferences for which he has served as Chair include: FG1998 (Financial Chair), OMINVIS2003 (Organizing Chair), ROBIO2006 (Program Co-chair), ACCV2007 (Program Chair), PSVIT2009 (Financial Chair), ICRA2009 (Technical Visit Chair), ACCV2009 (General Chair), ACPR2011 (Program Co-chair), and ACPR2013 (General Chair). He has also served on the IEEE ICRA Conference Editorial Board (2007–2011). He is an editorial board member of IJCV and Editor-in-Chief of IPSJ Transactions on Computer Vision & Applications. He was awarded an ACM VRST2003 Honorable Mention Award, IEEE ROBIO2006 Finalist for T.J. Tan Best Paper in Robotics, IEEE ICRA2008 Finalist for Best Vision Paper, MIRU2008 Nagao Award, and PSIVT2010 Best Paper Award. His research interests are computer vision, medical engineering, and robotics. He is a fellow of IPSJ and a member of IEICE, RSJ, and IEEE.
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Yamazoe, H., Habe, H., Mitsugami, I. et al. Depth error correction for projector-camera based consumer depth cameras. Comp. Visual Media 4, 103–111 (2018). https://doi.org/10.1007/s41095-017-0103-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-017-0103-7