Abstract
In this paper, we introduce a method to build 3D object models from RGB-D images automatically by interleaving model building with robotic manipulation. Using a fixed RGB-D camera and starting from the first view of the object, our approach gradually builds and extends a partial model (based on what has been visible) into a complete object model. In the process, the partial model is also used to guide a robot manipulator to change the pose of the object to make more surfaces visible for continued model building. The alternation of perception-based model building and pose changing continues until a complete object model is built with all object surfaces covered. The method is implemented, and experimental results show the effectiveness and robustness of this approach.
This work was supported by the NSF grants IIP-1432983, IIP-1439695, EPRI, and ABB.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choi, C., Christensen, H.I.: 3D pose estimation of daily objects using an RGB-D camera. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3342–3349, October 2012
Singh, A., Sha, J., Narayan, K.S., Achim, T., Abbeel, P.: BigBIRD: A large-scale 3D database of object instances. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516, May 2014
Koller, D., Turitzin, M., Levoy, M., Tarini, M., Croccia, G., Cignoni, P., Scopigno, R.: Protected interactive 3D graphics via remote rendering. ACM Trans. Graphics (TOG) 23(3), 695–703 (2004)
Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: Real-time dense surface mapping and tracking. In: 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 127–136 (2011)
Connolly, C.: The determination of next best views. In: ICRA, pp. 432–435 (1985)
Scott, W.R., Roth, G., Rivest, J.F.: View planning for automated 3D object reconstruction inspection. ACM Comput. Surv. (CSUR) 35(1), 64–96 (2003)
Tarbox, G., Gottschlich, S.: Planning for complete sensor coverage in inspection. Comput. Vis. Image Underst. 61(1), 84–111 (1995)
Bowyer, K., Dyer, C.: Aspect graphs: an introduction and survey of recent results. Int. J. Imaging Syst. Technol. 2(4), 315–328 (1990)
Maver, J., Bajcsy, R.: Occlusions as a guide for planning the next view. IEEE Trans. Pattern Anal. Mach. Intell. 15(5), 417–433 (1993)
Massios, N., Fisher, R.: A best next view selection algorithm incorporating a quality criterion. In: British Machine Vision Conference, pp. 780–789 (1998)
Foissotte, T., Stasse, O., Escande, A., Wieber, P., Kheddar, A.: A two-steps next-best-view algorithm for autonomous 3D object modeling by a humanoid robot. In: ICRA, pp. 1159–1164 (2009)
Kriegel, T., Bodenmuller, T., Suppa, M., Hirzinger, G.: A surface-based next-best-view approach for automated 3D model completion of unknown objects. In: ICRA, pp. 4869–4874 (2011)
Vasquez-Gomez, J., Sucar, L., Murrieta-Cid, R., Lopez-Damian, E.: Volumetric next-best-view planning for 3D object reconstruction with positioning error. Int. J. Adv. Rob. Syst. 11(159), 1–13 (2014)
Krainin, M., Curless, B., Fox, D.: Autonomous generation of complete 3D object models using next best view manipulation planning. In: ICRA, pp. 5031–5037, May 2011
Ma, L., Ghafarianzadeh, M., Coleman, D., Correll, N., Sibley, G.: Simultaneous localization, mapping, and manipulation for unsupervised object discovery. In: ICRA, pp. 1344–1351 (2015)
Browatzki, B., Tikhanoff, V., Metta, G., Bulthoff, H., Wallraven, C.: Active in-hand object recognition on a humanoid robot. IEEE Trans. Robot. 30(5), 1260–1269 (2014)
Plantinga, H., Dyer, C.R.: Visibility, occlusion, and the aspect graph. Intl. J. Comput. Vis. (IJCV) 5(2), 137–160 (1990)
Besl, P., McKay, H.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Yu, G., Morel, J.M.: ASIFT: an algorithm for fully affine invariant comparison. In: Image Process. On Line 1, 1–28 (2011)
Eggert, D.W., Lorusso, A., Fisher, R.B.: Estimating 3-D rigid body transformations: a comparison of four major algorithms. Mach. Vis. Appl. 9(5–6), 272–290 (1997)
Pulli, K.: Multiview registration for large data sets. In: 1999, Proceedings, Second International Conferences on 3-D Digital Imaging and Modeling, pp. 160–168 (1999)
Gottschalk, S.: Collision queries using oriented bounding boxes. Ph.D. thesis, The University of North Carolina at Chapel Hill (2000)
Klingensmith, M., Hermann, M., Srinivasa, S.: Object modeling and recognition from sparse, noisy data via voxel depth carving. In: International Symposium on Experimental Robotics (ISER) (2014)
Sanchiz, J., Fisher, R.: A next-best-view algorithm for 3D scene recovery with 5 degrees of freedom. In: British Machine Vision Conference, pp. 163–172 (1999)
Albalate, M.T.L., Devy, M., Miguel, J., Marti, S.: Perception planning for an exploration task of a 3D environment. In: Pattern Recognition, 2002. In: 16th International Conference on Proceedings, vol. 3, pp. 704–707 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Teng, Z., Mao, H., Xiao, J. (2017). Automatic Object Modeling Through Integrating Perception and Robotic Manipulation. In: Kulić, D., Nakamura, Y., Khatib, O., Venture, G. (eds) 2016 International Symposium on Experimental Robotics. ISER 2016. Springer Proceedings in Advanced Robotics, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-50115-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-50115-4_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50114-7
Online ISBN: 978-3-319-50115-4
eBook Packages: EngineeringEngineering (R0)