Skip to main content
SpringerLink
Log in

3D Point Cloud Analysis for Detection and Characterization of Defects on Airplane Exterior Surface

  • Published:
Journal of Nondestructive Evaluation Aims and scope Submit manuscript

Abstract

Three-dimensional surface defect inspection remains a challenging task. This paper describes a novel automatic vision-based inspection system that is capable of detecting and characterizing defects on an airplane exterior surface. By analyzing 3D data collected with a 3D scanner, our method aims to identify and extract the information about the undesired defects such as dents, protrusions or scratches based on local surface properties. Surface dents and protrusions are identified as the deviations from an ideal, smooth surface. Given an unorganized point cloud, we first smooth noisy data by using Moving Least Squares algorithm. The curvature and normal information are then estimated at every point in the input data. As a next step, Region Growing segmentation algorithm divides the point cloud into defective and non-defective regions using the local normal and curvature information. Further, the convex hull around each defective region is calculated in order to englobe the suspicious irregularity. Finally, we use our new technique to measure the dimension, depth, and orientation of the defects. We tested and validated our novel approach on real aircraft data obtained from an Airbus A320, for different types of defect. The accuracy of the system is evaluated by comparing the measurements of our approach with ground truth measurements obtained by a high-accuracy measuring device. The result shows that our work is robust, effective and promising for industrial applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28

Similar content being viewed by others

Abbreviations

\(P_{N} = \{p_{1}, p_{2},...,p_{N}\}\) :

A set of N points, \(p_{i}\) is the \(i^{th}\) data point

\(p_{i} = (x_{i}, y_{i}, z_{i})\) :

A point in three-dimensional

\(P^K= {p_{1}, p_{2},..., p_{K}}\) :

The set of points which are located in the k-neighborhood of a query point \(p_{i}\)

\(\overline{p}\) :

The centroid of the data e.g., given a set of points \(P_{N}\), we have: \(\overline{p} = \frac{1}{N}(\sum \limits _{i = 1}^{N}x_{i}, \sum \limits _{i = 1}^{N} y_{i}, \sum \limits _{i = 1}^{N} z_{i})\)

\(n_{i}\) :

A surface normal estimated at a point \(p_{i}\)

\(\cdot \) :

The dot product

\(\times \) :

The cross product

\(\Vert \circ \Vert \) :

The Euclidean norm of \( \circ \)

References

  1. Benhabiles, H., Lavoué, G., Vandeborre, J., Daoudi, M.: Learning boundary edges for 3d mesh segmentation. Comput. Gr. Forum 30, 2170–2182 (2011)

    Article  Google Scholar 

  2. Besl, P.J., Jain, R.C.: Segmentation through variable-order surface fitting. IEEE Trans. Pattern Anal. Mach. Intell. 10(2), 167–192 (1988)

    Article  Google Scholar 

  3. Bhanu, B., Lee, S., Ho, C., Henderson, T.: Range Data Processing: Representation of Surfaces by Edges. Department of Computer Science, University of Utah, Utah (1985)

    Google Scholar 

  4. Borsu, V., Yogeswaran, A., Payeur, P.: Automated surface deformations detection and marking on automotive body panels. In: 2010 IEEE Conference on Automation Science and Engineering (CASE), pp. 551–556. IEEE (2010)

  5. Itseez: Open source computer vision library (2015). https://github.com/itseez/opencv

  6. Carlbom, I., Paciorek, J.: Planar geometric projections and viewing transformations. ACM Comput. Surv. (CSUR) 10(4), 465–502 (1978)

    Article  MATH  Google Scholar 

  7. Deng, H., Zhang, W., Mortensen, E., Dietterich, T., Shapiro, L.: Principal curvature-based region detector for object recognition. In: CVPR’07. IEEE Conference on Computer Vision and Pattern Recognition, p. 18. IEEE (2007)

  8. Dey, T.K., Li, G., Sun, J.: Normal estimation for point clouds: a comparison study for a voronoi based method. In: Pointbased graphics, 2005. Eurographics/IEEE VGTC Symposium Proceedings, pp. 39–46. IEEE (2005)

  9. Dimitrov, D., Knauer, C., Kriegel, K., Rote, G.: On the bounding boxes obtained by principal component analysis. In: 22nd European Workshop on Computational Geometry, pp. 193–196. Citeseer (2006)

  10. Dyn, N., Hormann, K., Kim, S., Levin, D.: Optimizing 3d triangulations using discrete curvature analysis. Math. Methods Curves Surf. 28(5), 135–146 (2001)

    MathSciNet  MATH  Google Scholar 

  11. Filin, S.: Surface clustering from airborne laser scanning data. Int. Arch. Photogramme. Remote Sens. Spat. Inf. Sci. 34(3/A), 119–124 (2002)

    Google Scholar 

  12. Fua, P., Sander, P.: Segmenting unstructured 3d points into surfaces. In: Computer Vision—ECCV’92, pp. 676–680. Springer (1992)

  13. Golovinskiy, A., Funkhouser, T.: Randomized cuts for 3d mesh analysis. ACM Trans. Gr. (TOG) 27(5), 145 (2008)

    Google Scholar 

  14. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn, pp. 669–671. Prentice Hall, Upper Saddle River (2002)

    Google Scholar 

  15. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn, pp. 667–669. Prentice Hall, Upper Saddle River (2002)

    Google Scholar 

  16. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn, pp. 655–657. Prentice Hall, Upper Saddle rRiver (2002)

    Google Scholar 

  17. Gottschalk, S., Lin, M.C., Manocha, D.: Obbtree: a hierarchical structure for rapid interference detection. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 171–180. ACM (1996)

  18. Haddad, N.A.: From ground surveying to 3d laser scanner: a review of techniques used for spatial documentation of historic sites. J. King Saud Univ. Eng. Sci. 23(2), 109–118 (2011)

    Google Scholar 

  19. Hoover, A., JeanBaptiste, G., Jiang, X., Flynn, P.J., Bunke, H., Goldgof, D.B., Bowyer, K., Eggert, D.W., Fitzgibbon, A., Fisher, R.B.: An experimental comparison of range image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 18(7), 673–689 (1996)

    Article  Google Scholar 

  20. Jin, H., Yezzi, A.J., Soatto, S.: Region-based segmentation on evolving surfaces with application to 3d reconstruction of shape and piecewise constant radiance. In: Computer Vision—ECCV 2004, pp. 114–125. Springer (2004)

  21. Jolliffe, I.: Principal Component Analysis. Wiley Online Library, Hoboken (2002)

    MATH  Google Scholar 

  22. Katz, S., Tal, A.: Hierarchical Mesh Decomposition Using Fuzzy Clustering and Cuts, vol. 22. ACM, New York (2003)

    Google Scholar 

  23. Khan, W.: Image segmentation techniques: a survey. J. Image Gr. 1(4), 166–170 (2013)

    Google Scholar 

  24. Klasing, K., Althoff, D., Wollherr, D., Buss, M.: Comparison of surface normal estimation methods for range sensing applications. In: ICRA’09. IEEE International Conference on Robotics and Automation, pp. 3206–3211. IEEE (2009)

  25. Koenderink, J.J., van Doorn, A.J.: Surface shape and curvature scales. Image Vis. Comput. 10(8), 557–564 (1992)

    Article  Google Scholar 

  26. Köster, K., Spann, M.: Mir: an approach to robust clustering-application to range image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(5), 430–444 (2000)

    Article  Google Scholar 

  27. Lancaster, P., Salkauskas, K.: Surfaces generated by moving least squares methods. Math. Comput. 37(155), 141–158 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  28. Levin, D.: The approximation power of moving leasts-quares. Math. Comput. Am. Math. Soc. 67(224), 1517–1531 (1998)

    Article  MATH  Google Scholar 

  29. Levin, D.: Mesh-independent surface interpolation. In: Farin, G. (ed.) Geometric Modeling for Scientific Visualization, pp. 37–49. Springer, Berlin (2004)

    Chapter  Google Scholar 

  30. Marani, R., Roselli, G., Nitti, M., Cicirelli, G., D’Orazio, T., Stella, E.: A 3d vision system for high resolution surface reconstruction. In: 2013 Seventh International Conference on Sensing Technology (ICST), pp. 157–162. IEEE (2013)

  31. Mitra, N.J., Nguyen, A.: Estimating surface normals in noisy point cloud data. In: Proceedings of the Nineteenth Annual Symposium on Computational Geometry, pp. 322–328. ACM (2003)

  32. Mumtaz, R., Mumtaz, M., Mansoor, A.B., Masood, H.: Computer aided visual inspection of aircraft surfaces. Int. J. Image Process. (IJIP) 6(1), 38 (2012)

    Google Scholar 

  33. Nealen, A.: An as-short-as-possible introduction to the least squares, weighted least squares and moving least squares methods for scattered data approximation and interpolation. http://www.nealen.com/projects, pp. 130–150 (2004)

  34. Nguyen, A., Le, B.: 3d point cloud segmentation: a survey. In: 2013 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM), pp. 225–230. IEEE (2013)

  35. Nurunnabi, A., Belton, D., West, G.: Robust segmentation in laser scanning 3d point cloud data. In: 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA), p. 18. IEEE (2012)

  36. Nurunnabi, A., West, G., Belton, D.: Robust methods for feature extraction from mobile laser scanning 3D point clouds. In: Veenendaal, B., Kealy, A. (eds.) Research@Locate’15, pp. 109–120. Brisbane, Australia (2015)

  37. Pauling, F., Bosse, M., Zlot, R.: Automatic segmentation of 3d laser point clouds by ellipsoidal region growing. In: Australasian Conference on Robotics and Automation (ACRA) (2009)

  38. Pauly, M., Gross, M., Kobbelt, L.P.: Efficient simplification of point-sampled surfaces. In: Proceedings of the Conference on Visualization’02, pp. 163–170. IEEE Computer Society (2002)

  39. Peng, J., Li, Q., Kuo, C.J., Zhou, M.: Estimating gaussian curvatures from 3d meshes. In: International Society for Optics and Photonics on Electronic Imaging, pp. 270–280 (2003)

  40. Rabbani, T., Van Den Heuvel, F., Vosselmann, G.: Segmentation of point clouds using smoothness constraint. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 36(5), 248–253 (2006)

    Google Scholar 

  41. Rusu, R.B.: Semantic 3d object maps for everyday manipulation in human living environments. Ph.D. thesis, Technische Universität München (2009)

  42. Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). Shanghai (2011)

  43. Rusu, R.B., Marton, Z.C., Blodow, N., Dolha, M., Beetz, M.: Towards 3d point cloud based object maps for household environments. Robot. Auton. Syst. 56(11), 927–941 (2008)

    Article  Google Scholar 

  44. Sappa, A.D., Devy, M.: Fast range image segmentation by an edge detection strategy. In: Proceedings of Third International Conference on 3D Digital Imaging and Modeling, pp. 292–299. IEEE (2001)

  45. Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point cloud shape detection. Comput. Gr. Forum 26, 214–226 (2007)

    Article  Google Scholar 

  46. Seher, C., Siegel, M., Kaufman, W.M.: Automation tools for nondestructive inspection of aircraft: Promise of technology transfer from the civilian to the military sector. In: Fourth Annual IEEE DualUse Technologies and Applications Conference (1994)

  47. Shakarji, C.M., et al.: Leasts-quares fitting algorithms of the nist algorithm testing system. J. Res. Natl. Inst. Stand. Technol. 103, 633–641 (1998)

    Article  Google Scholar 

  48. Siegel, M., Gunatilake, P.: Remote inspection technologies for aircraft skin inspection. In: Proceedings of the 1997 IEEE Workshop on Emergent Technologies and Virtual Systems for Instrumentation and Measurement, Niagara Falls, Canada, pp. 79–78 (1997)

  49. Siegel, M., Gunatilake, P., Podnar, G.: Robotic assistants for aircraft inspectors. Ind. Robot 25(6), 389–400 (1998)

    Article  Google Scholar 

  50. Simari, P., Nowrouzezahrai, D., Kalogerakis, E., Singh, K.: Multi-objective shape segmentation and labeling. Comput. Gr. Forum 28, 1415–1425 (2009)

    Article  Google Scholar 

  51. Strom, J., Richardson, A., Olson, E.: Graph-based segmentation for colored 3d laser point clouds. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2131–2136. IEEE (2010)

  52. Tang, P., Akinci, B., Huber, D.: Characterization of three algorithms for detecting surface flatness defects from dense point clouds. In: IS&T/SPIE Electronic Imaging on International Society for Optics and Photonics, pp. 72,390N–72,390N (2009)

  53. Tóvári, D., Pfeifer, N.: Segmentation based robust interpolation—a new approach to laser data filtering. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 36(3/19), 79–84 (2005)

    Google Scholar 

  54. Wang, C., Wang, X., Zhou, X., Li, Z.: The aircraft skin crack inspection based on different-source sensors and support vector machines. J. Nondestruct. Eval. 35(3), 46 (2016). doi:10.1007/s10921-016-0359-3

    Article  Google Scholar 

  55. Wendland, H.: Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. Adv. Comput. Math. 4(1), 389–396 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  56. Wirjadi, O.: Survey of 3D Image Segmentation Methods, vol. 35. ITWM, kaiserslautern (2007)

    Google Scholar 

  57. Wong, B.S., Wang, X., Koh, C.M., Tui, C.G., Tan, C., Xu, J.: Crack detection using image processing techniques for radiographic inspection of aircraft wing spar. Insight NonDestructive Test. Cond. Monit. 53(10), 552–556 (2011)

  58. Yang, J., Gan, Z., Li, K., Hou, C.: Graph-based segmentation for rgb-d data using 3d geometry enhanced superpixels. IEEE Trans. Cybernet. 45(5), 927–940 (2015)

  59. Zhang, X., Li, H., Cheng, Z., Zhang, Y.: Robust curvature estimation and geometry analysis of 3d point cloud surfaces. J. Inf. Comput. Sci 6(5), 1983–1990 (2009)

    Google Scholar 

Download references

Acknowledgements

This work is part of the AIR-COBOT project (http://aircobot.akka.eu) approved by the Aerospace Valley world competitiveness cluster. The authors would like to thank the French Government for the financial support via the Single Inter-Ministry Fund (FUI). The partners of the AIR-COBOT project (AKKA TECHNOLOGIES, Airbus Group, ARMINES, 2MoRO Solutions, M3 SYSTEMS and STERELA) are also acknowledged for their support. Nicolas Simonot and Patrick Metayer from AIRBUS/NDT are also acknowledged for their help in providing dial gauge measurements.

Author information

Authors and Affiliations

  1. Institut Clément Ader (ICA), Université de Toulouse, CNRS, INSA, UPS, Mines Albi, ISAE, Campus Jarlard, 81013, Albi, France

    Igor Jovančević, Huy-Hieu Pham, Jean-José Orteu, Rémi Gilblas & Jacques Harvent

  2. KEONYS, 5 avenue de l’escadrille Normandie-Niemen, 31700, Blagnac, France

    Xavier Maurice & Ludovic Brèthes

Authors
  1. Igor Jovančević
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huy-Hieu Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-José Orteu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rémi Gilblas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacques Harvent
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xavier Maurice
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ludovic Brèthes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Jovančević.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jovančević, I., Pham, HH., Orteu, JJ. et al. 3D Point Cloud Analysis for Detection and Characterization of Defects on Airplane Exterior Surface. J Nondestruct Eval 36, 74 (2017). https://doi.org/10.1007/s10921-017-0453-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10921-017-0453-1

Keywords

Search

Navigation