Abstract
Understanding physical relations between objects, especially their support relations, is crucial for robotic manipulation. In this paper, we propose a method for extracting more detailed physical knowledge from a set of RGB-D images taken from the same scene but from different views using qualitative reasoning and intuitive physical models. Rather than providing a simple contact relation graph and approximating stability over convex shapes, our method is able to provide a detailed supporting relation analysis based on a volumetric representation. Specifically, true supporting relations between objects (e.g., if an object supports another object by touching it on the side or if the object above contributes to the stability of the object below) are identified. We apply our method to real-world structures captured in warehouse scenarios and show our method works as desired.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Date set 2 can be provided upon request.
References
Allen, J.F., Koomen, J.A.: Planning using a temporal world model. In: IJCAI 1983, pp. 741–747. Morgan Kaufmann Publishers Inc., Burlington (1983)
Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures. vol. 1611, pp. 586–607. International Society for Optics and Photonics (1992)
Chen, K., Lai, Y., Hu, S.: 3d indoor scene modeling from RGB-D data: a survey. CVM 1(4), 267–278 (2016)
Cholewiak, S.A., Fleming, R.W., Singh, M.: Visual perception of the physical stability of asymmetric three-dimensional objects. JOV 13(4), 12–12 (2013)
Ciocarlie, M., Hsiao, K., Jones, E.G., Chitta, S., Rusu, R.B., Şucan, I.A.: Towards reliable grasping and manipulation in household environments. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) ISER 2014, pp. 241–252. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-28572-1_17
Davis, E., Marcus, G.: The scope and limits of simulation in automated reasoning. AIJ 233, 60–72 (2016)
Fahlman, S.E.: A planning system for robot construction tasks. AIJ 5(1), 1–49 (1974)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Manhattan-world stereo. In: CVPR 2009, pp. 1422–1429. IEEE (2009)
Ge, X., Renz, J.: Representation and reasoning about general solid rectangles. IJCAI 2013, 905–911 (2013)
Ge, X., Renz, J., Abdo, N., Burgard, W., Dornhege, C., Stephenson, M., Zhang, P.: Stable and robust: stacking objects using stability reasoning. In: Workshop on Human-Robot Engagement in the Home, Workplace and Public Spaces, IJCAI 2017, p. 27 (2017)
Ge, X., Renz, J., Zhang, P.: Visual detection of unknown objects in video games using qualitative stability analysis. TCIAIG 8(2), 166–177 (2016)
Guesgen, H.W.: Spatial reasoning based on Allen’s temporal logic. International Computer Science Institute Berkeley (1989)
Jia, Z., Gallagher, A., Saxena, A., Chen, T.: 3d-based reasoning with blocks, support, and stability. CVPR 2013, 1–8 (2013)
Kemp, C.C., Edsinger, A., Torres-Jara, E.: Challenges for robot manipulation in human environments [grand challenges of robotics]. RAM 14(1), 20–29 (2007)
Ku, J., Harakeh, A., Waslander, S.L.: In defense of classical image processing: fast depth completion on the CPU. In: 2018 15th Conference on Computer and Robot Vision (CRV), pp. 16–22. IEEE (2018)
Kuipers, B.: Qualitative reasoning: modeling and simulation with incomplete knowledge. Automatica 25(4), 571–585 (1989)
Laboratory, R.P.I.I.P., Meagher, D.: Octree Encoding: a New Technique for the Representation, Manipulation and Display of Arbitrary 3-D Objects by Computer (1980)
Lee, J.H., Renz, J., Wolter, D., et al.: Starvars-effective reasoning about relative directions. IJCAI 2013, 976–982 (2013)
Li, W., Leonardis, A., Fritz, M.: Visual stability prediction for robotic manipulation. In: ICRA 2017. IEEE (2017)
Ligozat, G.É.: Reasoning about cardinal directions. JVLC 9(1), 23–44 (1998)
Liu, W., Li, S., Renz, J., et al.: Combining RCC-8 with qualitative direction calculi: algorithms and complexity. IJCAI 2009, 854–859 (2009)
Liu, Y., Wang, X., Jin, X., Wu, L.: On internal cardinal direction relations. In: Cohn, A.G., Mark, D.M. (eds.) COSIT 2005. LNCS, vol. 3693, pp. 283–299. Springer, Heidelberg (2005). https://doi.org/10.1007/11556114_18
Livesley, R.: Limit analysis of structures formed from rigid blocks. IJNME 12(12), 1853–1871 (1978)
Mojtahedzadeh, R., Bouguerra, A., Lilienthal, A.J.: Automatic relational scene representation for safe robotic manipulation tasks. In: IROS 2013, pp. 1335–1340. IEEE (2013)
Navarrete, I., Sciavicco, G.: Spatial reasoning with rectangular cardinal direction relations. In: ECAI 2006, vol. 6, pp. 1–10 (2006)
Panda, S., Abdul Hafez, A.H., Jawahar, C.: Single and multiple view support order prediction in clutter for manipulation (2016)
Papon, J., Abramov, A., Schoeler, M., Worgotter, F.: Voxel cloud connectivity segmentation-supervoxels for point clouds. CVPR 2013, 2027–2034 (2013)
Pomerleau, F., Colas, F., Siegwart, R., et al.: A review of point cloud registration algorithms for mobile robotics. Found. Trends® Robot. 4(1), 1–104 (2015)
Randell, D.A., Cui, Z., Cohn, A.G.: A spatial logic based on regions and connection. KR 1992(92), 165–176 (1992)
Shao, T., Monszpart, A., Zheng, Y., Koo, B., Xu, W., Zhou, K., Mitra, N.: Imagining the unseen: stability-based cuboid arrangements for scene understanding. TOG 33(6), 10 (2014)
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_54
Skiadopoulos, S., Koubarakis, M.: Composing cardinal direction relations. AIJ 152(2), 143–171 (2004)
Song, S., Yu, F., Zeng, A., Chang, A.X., Savva, M., Funkhouser, T.: Semantic scene completion from a single depth image. arXiv preprint arXiv:1611.08974 (2016)
Stein, S.C., Wörgötter, F., Schoeler, M., Papon, J., Kulvicius, T.: Convexity based object partitioning for robot applications. In: ICRA 2014, pp. 3213–3220. IEEE (2014)
Stoyanov, T., et al.: No more heavy lifting: robotic solutions to the container unloading problem. IEEE Robot. Autom. Mag. 23(4), 94–106 (2016)
Whiting, E., Ochsendorf, J., Durand, F.: Procedural modeling of structurally-sound masonry buildings. In: TOG, vol. 28, p. 112. ACM (2009)
Zhang, P., Renz, J.: Qualitative spatial representation and reasoning in angry birds: the extended rectangle algebra. In: KR 2014 (2014)
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE MultiMedia 19(2), 4–10 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, P., Ge, X., Renz, J. (2020). Support Relation Analysis for Objects in Multiple View RGB-D Images. In: El Fallah Seghrouchni, A., Sarne, D. (eds) Artificial Intelligence. IJCAI 2019 International Workshops. IJCAI 2019. Lecture Notes in Computer Science(), vol 12158. Springer, Cham. https://doi.org/10.1007/978-3-030-56150-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-56150-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-56149-9
Online ISBN: 978-3-030-56150-5
eBook Packages: Computer ScienceComputer Science (R0)