Skip to main content
SpringerLink
Log in

Surface profile-guided scan method for autonomous 3D reconstruction of unknown objects using an industrial robot

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a novel method called surface profile-guided scan for 3D surface reconstruction of unknown objects. This method benefits from the advantages of two types of sensors: one having a wide field of view but of low resolution (type I) and the other of high resolution but with a narrow field of view (type II) for the autonomous reconstruction of highly accurate 3D models. It employs a range sensor (type II) mounted on an industrial manipulator, a rotary stage, and a color camera (type I). No prior knowledge of the geometry of the object is required. The only information available is that the object is located on a rotary table and is within the field of view of the camera and in the working space of the industrial robot. The camera provides a set of vertical surface profiles around the object, which are used to obtain scan paths for the range sensor. Then, the robot manipulator moves the range sensor along the scan paths. Finally, the 3D surface model is completed by detecting and rescanning holes on the surface. The quality of the surface model obtained from real objects by the proposed 3D reconstruction method proves its effectiveness and versatility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

References

  1. Kriegel, S., Rink, C., Bodenmüller, T., Suppa, M.: Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects. J. Real-Time Image Proc. 10(4), 611–631 (2015)

    Article  Google Scholar 

  2. Khalfaoui, S., Seulin, R., Fougerolle, Y., Fofi, D.: An efficient method for fully automatic 3D digitization of unknown objects. Comput. Ind. 64, 1152–1160 (2013)

    Article  Google Scholar 

  3. Torabi, L., Gupta, K.: An autonomous six-DOF eye-in-hand system for in situ 3D object modeling. Int. J. Robot. Res. 31(1), 82–100 (2012)

    Article  Google Scholar 

  4. Shi, J., Sun, Z., Bai, S.: 3D reconstruction framework via combining one 3D scanner and multiple stereo trackers. Vis. Comput. 34, 377–389 (2018)

    Article  Google Scholar 

  5. Aleotti, J., Lodi Rizzini, D., Monica, R., Caselli, S., Global registration of mid-range 3D observations and short range next best views. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, Illinois, USA, 14–18 September 2014

  6. Karaszewski, M., Sitnik, R., Bunsch, E.: On-line, collision-free positioning of a scanner during fully automated three-dimensional measurement of cultural heritage objects. Robot. Auton. Syst. 60, 1205–1219 (2012)

    Article  Google Scholar 

  7. Uyanik, C., Secil, S., Ozkan, M., Dutagaci, H., Turgut, K., Parlaktuna, O.: SPGS: A new method for autonomous 3D reconstruction of unknown objects by an industrial robot. In: Towards Autonomous Robotic Systems (TAROS 2018), pp. 15–27, Bristol, UK, 25–27 July 2018

  8. Mavrinac, A., Chen, X., Luis Alarcon-Herrera, J.: Semiautomatic model-based view planning for active triangulation 3-D inspection systems. IEEE/ASME Trans. Mechatron. 20(2), 799–811 (2015)

    Article  Google Scholar 

  9. Lanman, D., Crispell, D., Taubin, G.: Surround structured lighting: 3-D scanning with orthographic illumination. Comput. Vis. Image Underst. 113(11), 1107–1117 (2009)

    Article  Google Scholar 

  10. Zexiao, X., Jianguo, W., Ming, J.: Study on a full field of view laser scanning system. Int. J. Mach. Tools Manuf. 47, 33–43 (2007)

    Article  Google Scholar 

  11. Hosseininaveh, A., Sargeant, B., Erfani, T., Robson, S., Shortis, M., Hess, M., Boehm, J.: Towards fully automatic reliable 3D acquisition: from designing imaging network to a complete and accurate point cloud. Robot. Auton. Syst. 62, 1197–1207 (2014)

    Article  Google Scholar 

  12. He, B.W., Long, Z.X., Lii, Y.F.: The research of an automatic object reconstruction method based on limit visible region of the laser-scanning vision system. Robot. Comput.-Integr. Manuf. 26, 711–719 (2010)

    Article  Google Scholar 

  13. Loriot, B., Seulin, R., Gorria, P.: Non-model based method for an automation of 3D acquisition and post-processing. Electr. Lett. Comput. Vis. Image Anal. 7(3), 67–82 (2008)

    Google Scholar 

  14. Ahmadabadian, A.H., Yazdan, R., Karami, A., Moradi, M., Ghorbani, F.: Clustering and selecing vantage images in a low cost system for 3D reconstruction of texture-less objects. Measurement 99, 185–191 (2017)

    Article  Google Scholar 

  15. Santosi, Z., Budak, I., Stojakovic, V., Sokac, M., Vukelic, D.: Evaluation of synthetically generated patterns for image-based 3D reconstruction of texture-less object. Measurement 147, 106883 (2019)

    Article  Google Scholar 

  16. Vincenzo, N., Cesare, R., Sergio, S.: A new real-time shape acquisition with a laser scanner: first test results. Robot. Comput.-Integr. Manuf. 26, 543–550 (2014)

    Google Scholar 

  17. Monica, R., Aleotti, J.: Contour-based next-best view planning from point cloud segmentation of unknown objects. Auton. Robot. 42, 443–458 (2018)

    Article  Google Scholar 

  18. Monica, R., Aleotti, J.: Surfel-based next best view planning. IEEE Robot. Autom. Lett. 3(4), 3324–3331 (2018)

    Article  Google Scholar 

  19. Wang, Y., James, S., Stathopoulou, E.K., Beltrán-González, C., Konishi, Y., Del Bue, A.: Autonomous 3-D reconstruction, mapping, and exploration of indoor environments with a robotic arm. IEEE Robot. Autom. Lett. 4(4), 3340–3347 (2019)

    Article  Google Scholar 

  20. Mavrinac, A., Chen, X.: Modeling coverage in camera networks: A survey. Int. J. Comput. Vis. 101(1), 205–226 (2013)

    Article  MathSciNet  Google Scholar 

  21. Cowan, C.K., Kovesi, P.D.: Automatic sensor placement from vision task requirements. IEEE Trans. Pattern Anal. Mach. Intell. 10(3), 407–416 (1988)

    Article  Google Scholar 

  22. Tarabanis, K.A., Tsai, R.Y., Allen, P.K.: The MVP sensor planning system for robotic vision tasks. IEEE Trans. Robot. Autom. 11(1), 72–85 (1995)

    Article  Google Scholar 

  23. Schwager, M., Julian, B.J., Angermann, M., Rus, D.: Eyes in the sky: decentralized control for the deployment of robotic camera networks. Proc. IEEE 99(9), 1541–1561 (2011)

    Article  Google Scholar 

  24. Mavrinac, A., Chen, X., Tan, Y.: Coverage quality and smoothness criteria for online view selection in a multi-camera network. ACM Trans. Sensor Netw. 10(2), 1–19 (2014)

    Article  Google Scholar 

  25. Zhang, X., Chen, X., Farzadpour, F., Fang, Y.: Visual distance approach for multicamera deployment with coverage optimization. IEEE/ASME Trans. Mechatron. 23(3), 1007–1018 (2018)

    Article  Google Scholar 

  26. Mendozaa, M., Vasquez-Gomez, I., Taud, H., Sucar, L.E., Reta, C.: Supervised learning of the next-best-view for 3d object reconstruction. Pattern Recognit. Lett. 133, 224–231 (2020)

    Article  Google Scholar 

  27. Doumanoglou, A., Kouskouridas, R., Malassiotis, S., Kim, T.: Recovering 6D object pose and predicting next-best-view in the crowd. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 3583–3592 (2016)

  28. Zivkovic, Z.: Improved adaptive gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, (2004)

  29. Zivkovic, Z., Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)

    Article  Google Scholar 

  30. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Article  Google Scholar 

  31. Sklansky, J.: Finding the convex hull of a simple polygon. Pattern Recogn. Lett. 1(2), 79–83 (1982)

    Article  MATH  Google Scholar 

  32. Bockman, S.F.: Generalizing the formula for areas of polygons to moments. Am. Math. Mon. 96(2), 131–132 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  33. Uyanik, C., Ozkan, M.: A method for determining 3D surface points of objects by a single camera and rotary stage. In: International Conference on Computer Science and Engineering (UBMK 2017), Antalya, Turkey (2017)

  34. Zhang, Z., Medioni, G., Kang, S.B. (eds.): Emerging Topics in Computer Vision. Prentice Hall Professional Technical Reference, New Jersey (2004)

    Google Scholar 

  35. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  36. Turgut, K., Dutagaci, H., Soyleyici, C., Secil, S., Ozkan, M., Parlaktuna, O., Parlaktuna, M.: A method for determining local coordinate frames attached to objects by a laser profile sensor in industrial robot workspaces. In: The 17th International Conference on Autonomous Robot Systems and Competitions (ICARSC 2017), Coimbra, Portugal (2017)

  37. K. Turgut, Autonomous 3-D object modeling with robot manipulator and laser profile sensor. M.Sc. Thesis, Eskisehir Osmangazi University, pp.35–45, Eskisehir, Turkey (2016)

  38. Loriot, B., Ralph, S., Gorria, P.: Non-model based method for an automation of 3D acquisition and post-processing. ELCVIA 7(3), 67–82 (2008)

    Article  Google Scholar 

  39. Guo, X., Xiao, J., Wang, Y.: A survey on algorithms of hole filling in 3D surface reconstruction. Vis. Comput. 34, 93–103 (2018)

    Article  Google Scholar 

  40. Zhao, W., Gao, S., Lin, H.: A robust hole-filling algorithm for triangular mesh. Vis. Comput. 23, 987–997 (2007)

    Article  Google Scholar 

  41. Marton, Z.C., Rusu, R.B., Beetz, M.: On fast surface reconstruction methods for large and noisy point clouds. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3218–3223, Kobe, Japan (2009)

  42. Secil, S., Turgut, K., Soyleyici, C., Ozkan, M., Parlaktuna, O., Dutagaci, H., Parlaktuna, M.: A robotic system for autonomous 3-D surface reconstruction of objects: design and implementation. In: The 3rd International Conference on Control, Automation and Robotics (ICCAR 2017), Nagoya, Japan, 22–24 April 2017

  43. Huang, Y., Qian, X.: Dynamic B-spline surface reconstruction: closing the sensing-and-modeling loop in 3D digitization. Comput. Aided Des. 39, 987–1002 (2007)

    Article  Google Scholar 

  44. Quinsat, Y., Lartigue, C.: Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int. J. Adv. Manuf. Technol. 81, 411–421 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under project number 115E374 and project title “Fully Automated 3D Modeling of Objects by Using an Industrial Robot Manipulator.” This work was also supported in part by the Scientific Research Project Commission of Eskisehir Osmangazi University (ESOGU-BAP) under the project number 201515A105 and project title “3D reconstruction of unknown objects with a laser sensor and robot manipulator.”

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Eskisehir Osmangazi University, Eskisehir, Turkey

    Metin Ozkan

  2. Department of Electrical and Electronics Engineering, Eskisehir Osmangazi University, Eskisehir, Turkey

    Sezgin Secil, Kaya Turgut, Helin Dutagaci & Osman Parlaktuna

  3. Department of Health Technology, Technical University of Denmark, Copenhagen, Denmark

    Cihan Uyanik

Authors
  1. Metin Ozkan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sezgin Secil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaya Turgut
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Helin Dutagaci
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cihan Uyanik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Osman Parlaktuna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Metin Ozkan.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ozkan, M., Secil, S., Turgut, K. et al. Surface profile-guided scan method for autonomous 3D reconstruction of unknown objects using an industrial robot. Vis Comput 38, 3953–3977 (2022). https://doi.org/10.1007/s00371-021-02241-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02241-z

Keywords

Search

Navigation