Skip to main content
SpringerLink
Log in

Registration of 3D scan data using image reprojection

  • Regular Paper
  • Published:
International Journal of Precision Engineering and Manufacturing Aims and scope Submit manuscript

Abstract

This paper proposes a method of registering point clouds using 2D images, 3D point clouds, and their correspondences in order to provide appropriate initial conditions for 3D fine registration algorithms such as Iterative Closest Point. Many commercially available optical 3D scanners capture both 3D point clouds and 2D images, and their correspondences can be obtained using camera calibration information. The proposed method registers 3D source data (moving) to 3D reference data (fixed) in an iterative manner, with each iteration consisting of three steps: (1) finding image correspondences in the source and reference images, (2) transforming the source data using the corresponding 3D points, (3) generating a virtual image of the source data in the transformed coordinates. The above steps are repeated until the source data approaches suitable initial conditions for fine registration. The proposed method has been tested on various objects, including mechanical parts, animals, and cultural items.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

A :

intrinsic camera matrix

I :

intensity value

P :

reference data

P 2d :

reference 2d point

P 3d :

reference 3d point

P I :

reference intensity value

Q :

source data

Q 2d :

source 2d point

Q 3d :

source 3d point

Q I :

source intensity value

h 2D :

arbitrary 2D point with an intensity of zero

r k :

the closest non-zero intensity 2D points

T :

transformation matrix

References

  1. Kim, H. T., Kim, S. T., and Cho, Y. J., “Quick and Efficient Light Control for Conventional Automatic Optical Inspection (AOI) Systems,” Int. J. Precis. Eng. Manuf., Vol. 16, No. 2, pp. 247–254, 2015.

    Article  Google Scholar 

  2. Hartley, R. I., “Triangulation,” Computer Vision and Image Understanding, Vol. 68, No. 2, pp. 146–157, 1997.

    Article  Google Scholar 

  3. Besl, P. J. and McKay, N. D., “A Method for Registration of 3-D Shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, pp. 239–256, 1992.

    Article  Google Scholar 

  4. Chen, Y. and Medioni, G., “Object Modelling by Registration of Multiple Range Images,” Image and Vision Computing, Vol. 10, No. 3, pp. 145–155, 1992.

    Article  Google Scholar 

  5. Zhang, Z., “Iterative Point Matching for Registration of Free-Form Curves and Surfaces,” International Journal of Computer Vision, Vol. 13, No. 2, pp. 119–152, 1994.

    Article  Google Scholar 

  6. Lee, K. W. and Bo, P., “Feature Curve Extraction from Point Clouds via Developable Strip Intersection,” Journal of Computational Design and Engineering, Vol. 3, No. 2, pp. 102–111, 2016.

    Article  Google Scholar 

  7. Hasan, S. M. A. and Ko, K., “Depth Edge Detection by Image-Based Smoothing and Morphological Operations,” Journal of Computational Design and Engineering, Vol. 3, No. 3, pp. 191–197, 2016.

    Article  Google Scholar 

  8. Gelfand, N., Mitra, N. J., Guibas, L. J., and Pottmann, H., “Robust Global Registration,” Proc. of Eurographics Symposium on Geometry Processing, 2005.

    Google Scholar 

  9. Mori, G., Belongie, S., and Malik, J., “Efficient Shape Matching Using Shape Contexts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 11, pp. 1832–1837, 2005.

    Article  MATH  Google Scholar 

  10. Huang, Q.-X., Flöry, S., Gelfand, N., Hofer, M., and Pottmann, H., “Reassembling Fractured Objects by Geometric Matching,” ACM Transactions on Graphics (TOG), Vol. 25, No. 3, pp. 569–578, 2006.

    Article  Google Scholar 

  11. Fadaifard, H. and Wolberg, G., “Multiscale 3d Feature Extraction and Matching,” Proc. of International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 228–235, 2011.

    Google Scholar 

  12. Aiger, D., Mitra, N.J., and Cohen-Or, D., “4-Points Congruent Sets for Robust Pairwise Surface Registration,” ACM Transactions on Graphics (TOG), Vol. 27, No. 3, Article No. 85, 2008.

  13. Mellado, N., Aiger, D., and Mitra, N. J., “Super 4pcs Fast Global Pointcloud Registration via Smart Indexing,” Computer Graphics Forum, Vol. 33, No. 5, pp. 205–215, 2014.

    Article  Google Scholar 

  14. Johnson, A. E. and Hebert, M., “Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, No. 5, pp. 433–449, 1999.

    Article  Google Scholar 

  15. Johnson, A. E., “Spin-Images: A Representation for 3-D Surface Matching,” Ph.D. Thesis, Carnegie Mellon University Pittsburgh, 1997.

    Google Scholar 

  16. Rusu, R. B., Blodow, N., and Beetz, M., “Fast Point Feature Histograms (FPFH) for 3D Registration,” Proc. of IEEE International Conference on Robotics and Automation, pp. 3212–3217, 2009.

    Google Scholar 

  17. Bendels, G. H., Degener, P., Wahl, R., Körtgen, M., and Klein, R., “Image-Based Registration of 3D-Range Data Using Feature Surface Elements,” Proc. of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, pp. 115–124, 2004.

    Google Scholar 

  18. Chu, J. and Nie, C.-m., “Multi-View Point Clouds Registration and Stitching Based on Sift Feature,” Proc. of 3rd International Conference on Computer Research and Development (ICCRD), pp. 274–278, 2011.

    Google Scholar 

  19. Moldovan, D., Yano, S., and Inoue, N., “Automatic Registration of Multiple Range Views Based on Feature Matching and Feature-Depth Correspondences,” Proc. of 5th International Conference on Intelligent Computer Communication and Processing, pp. 169–172, 2009.

    Google Scholar 

  20. Sehgal, A., Cernea, D., and Makaveeva, M., “Real-Time Scale Invariant 3D Range Point Cloud Registration,” in: Image Analysis and Recognition, Campilho, A., Kamel, M., (Eds.), Lecture Notes in Computer Science, Springer, pp. 220–229, 2010.

    Chapter  Google Scholar 

  21. Andreasson, H. and Lilienthal, A. J., “6D Scan Registration Using Depth-Interpolated Local Image Features,” Robotics and Autonomous Systems, Vol. 58, No. 2, pp. 157–165, 2010.

    Article  Google Scholar 

  22. Moussa, W., “Integration of Digital Photogrammetry and Terrestrial Laser Scanning for Cultural Heritage Data Recording,” Ph.D. Thesis, University of Stuttgart, 2014.

    Google Scholar 

  23. Lowe, D. G., “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, Vol. 60, No. 2, pp. 91–110, 2004.

    Article  Google Scholar 

  24. Bay, H., Tuytelaars, T., and Van Gool, L., “Surf: Speeded Up Robust Features,” in: Computer Vision-ECCV 2006, Leonardis, A., Bischof, H., Pinz, A., (Eds.), Lecture Notes in Computer Science, Springer, Vol.3951, pp. 404–417, 2006.

    Article  Google Scholar 

  25. McLaughlin, R. A., Hipwell, J., Penney, G. P., Rhode, K., Chung, A., et al., “Intensity-Based Registration Versus Feature-Based Registration for Neurointerventions,” Proc. of Medical Image Understanding and Analysis, pp. 69–72, 2001.

    Google Scholar 

  26. Li, W.-L., Xie, H., Yin, Z.-P., and Xiong, Y.-L., “A New Algorithm for Non-Rigid Shape Matching with Anisotropic-Scaling Transformation Parameters,” Int. J. Precis. Eng. Manuf., Vol. 16, No. 5, pp. 895–903, 2015.

    Article  Google Scholar 

  27. Zhang, Z., “A Flexible New Technique for Camera Calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, pp. 1330–1334, 2000.

    Article  Google Scholar 

  28. Shepard, D., “A Two-Dimensional Interpolation Function for Irregularly-Spaced Data,” Proc. of the 23rd ACM National Conference, pp. 517–524, 1968.

    Google Scholar 

  29. Kim, D., Khalil, H., Nam, J., and Park, K., “Image-Based Tracking System for Rotating Object Vibration Measurement Using Laser Scanning Vibrometer,” Int. J. Precis. Eng. Manuf., Vol. 16, No. 8, pp. 1717–1721, 2015.

    Article  Google Scholar 

  30. Fischler, M. A. and Bolles, R. C., “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Communications of the ACM, Vol. 24, No. 6, pp. 381–395, 1981.

    Article  MathSciNet  Google Scholar 

  31. Evangelidis, G. D. and Psarakis, E. Z., “Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 10, pp. 1858–1865, 2008.

    Article  Google Scholar 

  32. Im, S., Lee, Y., Kim, J., and Chang, M., “A Solution for Camera Occlusion Using a Repaired Pattern from a Projector,” Int. J. Precis. Eng. Manuf., Vol. 17, No. 11, pp. 1443–1450, 2016.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul, 02841, South Korea

    Sungho Byun, Keonhwa Jung, Sungbin Im & Minho Chang

Authors
  1. Sungho Byun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keonhwa Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sungbin Im
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Minho Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minho Chang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Byun, S., Jung, K., Im, S. et al. Registration of 3D scan data using image reprojection. Int. J. Precis. Eng. Manuf. 18, 1221–1229 (2017). https://doi.org/10.1007/s12541-017-0143-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12541-017-0143-z

Keywords

Search

Navigation