Skip to main content
SpringerLink
Log in

Fast restoration of geometric details of automobile castings scanned by RGB-D sensor

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The depth data of automobile castings obtained by RGB-D sensor are usually combined with noise, the classical regularization method can eliminate the noise efficiently. Yet the regularization step is too time-consuming to reconstruct the geometric details of automobile castings efficiently. Given this, we present a fast method called fast restoration of automobile castings (FRAC) to restore the geometric details of automobile castings in fast manner. First, the implicit surface data is extracted from globally aligned RGB-D images, the voxel data structure is extended to index and process the implicit surface in real time. Then, an inverse shading formula is constructed to compute TSDF (truncated signed distance field) values of casting surfaces quickly, and an objective function is designed to optimize the geometric details of casting surfaces in real time. Finally, a GPU-based Gauss–Newton solver is used to accelerate restoration of castings further. The defective casting models scanned by RGB-D sensor are quickly refined to a complete model with better accuracy. Experimental results show that with respect to the sampled automobile castings which include 359,470 points in average, the average optimization time reaches 0.66 s per frame, the average restoration time is about 6.48 s. Computing TSDF requires only about 34.8 MB GPU caches in average. FRAC is able to restore the geometric details of automobile castings in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Werner, D., Al-Hamadi, A., Werner, P.: Truncated signed distance function: experiments on voxel size. ICIAR 2014, pp. 357–364 (2014)

  2. Rasmusson, A., Sørensen, T.S., Ziegler, G.: Connected Components Labeling on the GPU with Generalization to Voronoi Diagrams and Signed Distance Fields. ISVC 2013, pp. 206–215 (2013)

  3. Slavcheva, M., Kehl, W., Navab, N., Ilic, S.: Sdf-2-sdf registration for real-time 3d reconstruction from RGB-D data. Int. J. Comput. Vis. 2, 1–22 (2017)

    Google Scholar 

  4. Li, D.W., Dai, N., Jiang, X.T., Chen, X.S.: Interior structural optimization based on the density-variable shape modeling of 3D printed objects. Int. J. Adv. Technol. J. 83(9–12), 1627–1635 (2015)

    Google Scholar 

  5. Wang, X.C., Zhang, W.W., Wu, X.C., Xiao, L.Y., Qian, Y.B., Fang, Z.: Real-time vehicle type classification with deep convolutional neural networks. J. Real Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0712-5

    Article  Google Scholar 

  6. Zhu, X., Meng, Q., Gu, L.: Real-time image recognition using weighted spatial pyramid networks. J. Real Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0743-y

    Article  Google Scholar 

  7. Takimoto, R.Y., Tsuzuki, M.D.S.G., Vogelaar, R., Martins, T.D.C., Sato, A.K., Iwao, Y., et al.: 3d reconstruction and multiple point cloud registration using a low precision RGB-D sensor. Mechatronics 35(January), 11–22 (2016)

    Article  Google Scholar 

  8. Fuhrmann, S., Goesele, M.: Floating scale surface reconstruction. ACM Trans. Graph. 33(4), 1–11 (2014)

    Article  Google Scholar 

  9. Coutinho, M.G.: Appendix G: Constructing signed distance fields for 3D polyhedra. Guide to dynamic simulations of rigid bodies and particle systems, pp. 351–364 (2013)

  10. Barclay, J., Dhokia, V., Nassehi, A.: Additive manufacturing simulation using signed distance fields. Sustainable design and manufacturing, pp. 435–444 (2016)

  11. Garcia-Garcia, A., Orts-Escolano, S., Garcia-Rodriguez, J., et al.: Interactive 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors. J. Real Time Image Proc. 14(3), 585–604 (2018)

    Article  Google Scholar 

  12. Whelan, T., Johannsson, H., Kaess, M.: Robust real-time visual odometry for dense RGB-D mapping. IEEE international conference on robotics and automation, pp. 5724–5731 (2013)

  13. Richardt, C., Stoll, C., Dodgson, N.A.: Coherent spatio temporal filtering, up sampling and rendering of RGBZ videos. Comput. Graph. Forum 31(2pt1), 247–256 (2012)

    Article  Google Scholar 

  14. Denker, K., Hamann, B., Umlauf, G.: On-line CAD reconstruction with accumulated means of local geometric properties. International conference on curves and surfaces 2014, pp: 181–201 (2015)

  15. Wöhler, C.: Geometric approaches to three-dimensional scene reconstruction. 3D computer vision, pp. 3–126 (2009)

  16. Cavallari, T., Stefano, L.D.: On-Line Large Scale semantic fusion. ECCV 2016, pp. 83–99 (2016)

  17. Nießner, M., Izadi, S., Stamminger, M.: Real-time 3d reconstruction at scale using voxel hashing. Acm Trans. Graph. 32(6), 1–11 (2013)

    Article  Google Scholar 

  18. Minh, P., Hoai, V., Quoc, L.: WSDF: weighting of signed distance function for camera motion estimation in RGB-D data. Int. J. Adv. Res. Artif. Intell. 5(9), 27–32 (2016)

    Article  Google Scholar 

  19. Stefani, S.A., Nagarajah, C.R., Toncich, D.J.: Non-contact inspection for the detection of internal surface defects in hollow cylindrical work-pieces. Int. J. Adv. Technol. J. 11(2), 146–154 (1996)

    Google Scholar 

  20. Stavridis, J., Papacharalampopoulos, A., Stavropoulos, P.: Quality assessment in laser welding: a critical review. Int. J. Adv. Technol. J. 94(5–8), 1825–1847 (2018)

    Google Scholar 

  21. Wang, K., Zhang, G., Bao, H.: Robust 3d reconstruction with a RGB-D camera. IEEE Trans. Image Process. 23(11), 4893–4906 (2014)

    Article  MathSciNet  Google Scholar 

  22. Harooni, M., Carlson, B., Kovacevic, R.: Detection of defects in laser welding of AZ31B magnesium alloy in zero-gap lap joint configuration by a real-time spectroscopic analysis. Opt. Lasers Eng. 56, 54–66 (2014)

    Article  Google Scholar 

  23. Chang, C.C., Chiang, H.W.: Three-dimensional image reconstructions of complex objects by an abrasive computed tomography apparatus. Int. J. Adv. Technol. J. 22(9–10), 708–712 (2003)

    Google Scholar 

  24. Qin, Y., Brockett, A., Ma, Y., Razali, A., Zhao, J., Harrison, C., Pan, W., Dai, X., Loziak, D.: Micro-manufacturing: research, technology outcomes and development issues. Int. J. Adv. Technol. J. 47(9–12), 821–837 (2009)

    Google Scholar 

  25. Guo, K., Xu, F., Yu, T., Liu, X., Dai, Q., Liu, Y.: Real-time geometry, albedo, and motion reconstruction using a single RGB-D camera. ACM Trans. Graph., 36(3), 32 (2017)

  26. Zollhöfer, M., Dai, A., Innmann, M., Wu, C., Stamminger, M., Theobalt, C.: Shading-based refinement on volumetric signed distance functions. ACM Trans. Graph. 34(4), 1–14 (2015)

    Article  Google Scholar 

  27. Lin, J., Yao, Y., Ma, L., Wang, Y.: Detection of a casting defect tracked by deep convolution neural network. Int. J. Adv. Manuf. Technol. 97(1–4), 1–9 (2018)

    Google Scholar 

Download references

Acknowledgements

This project is supported by National High-tech R&D Program (Grant no. 2014AA7031010B) and Science-Technology Project of the thirteenth Five-Year Plan (Grant no. 2016345).

Author information

Authors and Affiliations

  1. Computer Application Technology, Changchun University of Technology, Changchun, 130000, China

    Jinhua Lin

  2. Mechatronic Engineering, Chinese Academy of Sciences University, Changchun, 130033, China

    Jinhua Lin

  3. FAW Foundry Co., Ltd, Changchun, 130000, China

    Lin Ma

Authors
  1. Jinhua Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinhua Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, J., Ma, L. Fast restoration of geometric details of automobile castings scanned by RGB-D sensor. J Real-Time Image Proc 17, 871–886 (2020). https://doi.org/10.1007/s11554-018-0835-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0835-3

Keywords

Search

Navigation