Abstract
This paper presents a lightweight 3D vision system called Equal Baseline Camera Array (EBCA). EBCA can work in different light conditions and it can be applied for measuring large range of distances. The system is a useful alternative to other known distance measuring devices such as structured-light 3D scanners, time-of-flight cameras, Light Detection and Ranging (LIDAR) devices and structure from motion techniques. EBCA can be mounted on a robotic arm without putting significant load on its construction. EBCA consists of a central camera and a ring of side cameras. The system uses stereo matching algorithms to acquire disparity maps and depth maps similarly as in case of using stereo cameras. This paper introduces methods of adapting stereo matching algorithms designed for stereo cameras to EBCA. The paper also presents the analysis of local, semi-global and global stereo matching algorithms in the context of the EBCA usage. Experiments show that, on average, results obtained from EBCA contain 37.49% less errors than the results acquired from a single stereo camera used in the same conditions.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Pérez, L., Rodríguez, İ., Rodríguez, N., Usamentiaga, R., García, D.F.: Robot guidance using machine vision techniques in industrial environments: A comparative review. Sensors 16(3), 335 (2016)
Jang, W., Je, C., Seo, Y., Lee, S.W.: Structured-light stereo: Comparative analysis and integration of structured-light and active stereo for measuring dynamic shape. Opt. Lasers Eng. 51(11), 1255–1264 (2013)
Eichhardt, I., Chetverikov, D., Jankó, Z.: Image-guided tof depth upsampling: a survey. Mach. Vis. Appl. 28(3), 267–282 (2017)
Tse, RO.C., Gold, C., Kidner, D.: Advances in 3D Geoinformation Systems, chapter 3D City Modelling from LIDAR Data, pp. 161–175. Springer, Berlin (2008)
Kazmi, W., Foix, S., Alenya, G.: Plant Leaf Imaging Using Time of Flight Camera under Sunlight, Shadow and Room Conditions. In: 2012 IEEE International Symposium On Robotic and Sensors Environments (ROSE), pp. 192–197 (2012)
Venkataraman, K., Lelescu, D., Duparré, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., Nayar, S.: Picam: An ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32(6), 166:1–166:13 (2013)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 519–528 (2006)
Okutomi, M., Kanade, T.: A multiple-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993)
Park, J.-I., Inoue, S.: Acquisition of sharp depth map from multiple cameras. Signal Process. Image Commun. 14(1-2), 7–19 (1998)
Kaczmarek, A.L.: Improving depth maps of plants by using a set of five cameras. J. Electron. Imaging 24(2), 023018 (2015)
Kaczmarek, A.L.: Stereo vision with equal baseline multiple camera set (ebmcs) for obtaining depth maps of plants. Comput. Electron. Agricult. 135, 23–37 (2017)
Kaczmarek, A.L.: Stereo Camera Upgraded to Equal Baseline Multiple Camera Set (Ebmcs). In: 2017 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4 (2017)
Bi, Z.M., Wang, Lihui: Advances in 3d data acquisition and processing for industrial applications. Robot. Comput.-Integr. Manuf. 26(5), 403–413 (2010)
Daneshmand, M., Bilici, O., Bolotnikova, A., Anbarjafari, G.: Medical robots with potential applications in participatory and opportunistic remote sensing: A review. Robot. Auton. Syst. 95, 160–180 (2017)
Ludvigsen, M., Sørensen, A.J.: Towards integrated autonomous underwater operations for ocean mapping and monitoring. Ann. Rev. Control 42, 145–157 (2016)
Jiang, W., Wang, Z.: Calibration of visual model for space manipulator with a hybrid lm-ga algorithm. Mech. Syst. Signal Process. 66-67, 399–409 (2016)
Murphy, R.R., Kravitz, J., Stover, S.L., Shoureshi, R.: Mobile robots in mine rescue and recovery. IEEE Robot. Autom. Mag. 16(2), 91–103 (2009)
Fjerdingen, S.A., Bjerkeng, M., Transeth, A.A., Kyrkjebø, E., Røyrøy, A.: A learning camera platform for remote operations with industrial manipulators. IFAC Proc. Vol. 45(8), 39–46 (2012). 1st IFAC Workshop on Automatic Control in Offshore Oil and Gas Production
Hayashi, S., Shigematsu, K., Yamamoto, S., Kobayashi, K., Kohno, Y., Kamata, J., Kurita, M.: Evaluation of a strawberry-harvesting robot in a field test. Biosyst. Eng. 105(2), 160–171 (2010)
Baeten, J., Donné, K., Boedrij, S., Beckers, W., Claesen, E.: Autonomous fruit picking machine: A robotic apple harvester. In: Laugier, C., Siegwart, R. (eds.) Field and Service Robotics, volume 42 of Springer Tracts in Advanced Robotics, pp 531–539. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-75404-6_51
van Henten, E.J., Hemming, J., van Tuijl, B.A.J., Kornet, J.G., Meuleman, J., Bontsema, J., van Os, E.A.: An autonomous robot for harvesting cucumbers in greenhouses. Auton. Robot. 13(3), 241–258 (2002)
Muscato, G., Prestifilippo, M., Abbate, N., Rizzuto, I.: A prototype of an orange picking robot: past history, the new robot and experimental results. Indust. Robot: Int. J. 32(2), 128–138 (2005)
Wan, W., Lu, F., Wu, Z., Harada, K.: Teaching robots to do object assembly using multi-modal 3d vision. Neurocomputing 259(Supplement C), 85–93 (2017). Multimodal Media Data Understanding and Analytics
Sanchez-Lopez, J.R., Marin-Hernandez, A., Palacios-Hernandez, E.R., Rios-Figueroa, H.V., Marin-urias, L.F.: A real-time 3d pose based visual servoing implementation for an autonomous mobile robot manipulator, vol. 7. 3rd Iberoamerican Conference on Electronics Engineering and Computer Science CIIECC (2013)
Shin, B.-S., Mou, X., Mou, W., Wang, H.: Vision-based navigation of an unmanned surface vehicle with object detection and tracking abilities. Mach. Vis. Appl. 29(1), 95–112 (2018)
Jia, B., Chen, J., Zhang, K.: Recursive drivable road detection with shadows based on two-camera systems. Mach. Vis. Appl. 28(5), 509–523 (2017)
Lin, L., Song, Y., Yang, Y., Feng, H., Cheng, Y., Pan, H.: Computer vision system r&d for east articulated maintenance arm robot. Fusion Eng. Des. 100, 254–259 (2015)
Plebe, A., Grasso, G.: Localization of spherical fruits for robotic harvesting. Mach. Vis. Appl. 13(2), 70–79 (2001)
Palli, G., Moriello, L., Scarcia, U., Melchiorri, C.: An underwater robotic gripper with embedded force/torque wrist sensor. IFAC-PapersOnLine 50(1), 11209–11214 (2017). 20th IFAC World Congress
Chéné, Y., Rousseau, D., Lucidarme, P., Bertheloot, J., Caffier, V., Morel, P., Belin, É., Chapeau-Blondeau, F.: On the use of depth camera for 3d phenotyping of entire plants. Comput. Electron. Agricult. 82, 122–127 (2012)
Bradski, G.R., Opencv, A.K.: Learning Gary Rost, 1st edn. O’Reilly Media, Inc., Newton (2008)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002). Microsoft Research Technical Report MSR-TR-2001-81, November 2001
Menze, M., Heipke, C., Geiger, A.: Joint 3D Estimation of Vehicles and Scene Flow. In: ISPRS Workshop on Image Sequence Analysis (ISA) (2015)
Menze, M., Heipke, C., Geiger, A.: Object scene flow ISPRS Journal of Photogrammetry and Remote Sensing (JPRS) (2018)
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, ., Levoy, M.: High performance imaging using large camera arrays. In: ACM SIGGRAPH 2005 Papers, SIGGRAPH’05, pp. 765–776. ACM, New York (2005)
Fehrman, B., McGough, J.: Depth Mapping Using a Low-Cost Camera Array. In: 2014 Southwest Symposium on Image Analysis and Interpretation, pp. 101–104 (2014)
Fehrman, B., McGough, J.: Handling Occlusion with an Inexpensive Array of Cameras. In: 2014 Southwest Symposium on Image Analysis and Interpretation, pp. 105–108 (2014)
Kaczmarek, A.L.: Influence of aggregating window size on disparity maps obtained from equal baseline multiple camera set (ebmcs) Choraś, R.S. (ed.) . Springer International Publishing, Cham (2017)
Microsoft lifecam studio product information. http://download.microsoft.com/download/0/9/5/0952776D-7A26-40E1-80C4-76D73FC729DF/TDS_LifeCamStudio.pdf. Accessed: 2019-09-08
Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20(4), 401–406 (1998)
Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)
Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc. Ser. B (Methodol.) 48(3), 259–302 (1986)
Tappen, M.F., Freeman, W.T.: Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical Mrf Parameters. In: 2003. Proceedings. Ninth IEEE International Conference On Computer Vision, pp. 900–906 Vol.2 (2003)
Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Map estimation via agreement on trees: message-passing and linear programming. IEEE Trans. Inf. Theory 51(11), 3697–3717 (2005)
Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1568–1583 (2006)
Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) Computer Vision ACCV 2010, 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8-12, 2010, Revised Selected Papers, Part I, volume 6492 of Lecture Notes in Computer Science, pp 25–38. Springer, Berlin (2011)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Kaczmarek, A.L. 3D Vision System for a Robotic Arm Based on Equal Baseline Camera Array. J Intell Robot Syst 99, 13–28 (2020). https://doi.org/10.1007/s10846-019-01117-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-019-01117-8