Automatic Detection of Structural Deficiencies Using 4D Hue-Assisted Analysis of Color Point Clouds

Conference paper
Part of the Conference Proceedings of the Society for Experimental Mechanics Series book series (CPSEMS)


Recent developments in the fields of robotics and remote sensing technologies such as 3D laser scanners and photogrammetric approaches have provided an unprecedented opportunity to collect a massive amount of data from infrastructure systems in a contactless and nondestructive manner, which can potentially improve the structural health monitoring process. However, the complex nature of these geometrically accurate and high-resolution 3D models makes it inefficient and time-consuming to manually analyze and manipulate them and automating this process continue to pose a challenge. Thus, procedures that automate the data processing in order to detect a variety of damages are desired to make full use of these modern inspection technologies as a tool for infrastructure integrity assessment and asset management. The aim of this paper is to present a new algorithm to automatically identify and evaluate structural deficiencies in massive 3D point clouds of complex infrastructure systems. This approach takes advantage of both local geometry and color data properties associated with each point to improve the damage detection capabilities in a variety of scenarios. Linear and non-linear transformations from the RGB color space to non-RGB spaces were performed to increase separability between the damage and the structure and to achieve robustness to changes in illumination. Recently, a complex and large-scale gravity dam in Maryland, USA has served as a test bed for the developed methodology. In this experiment, a multi-scale photogrammetric computer vision approach was utilized to generate accurate and highly detailed 3D models of the targeted dam. In order to maximize the accessibility and to overcome geometric constraints, different multi-rotor Unmanned Aerial Vehicle (UAV) platforms with varied payload and maneuverability capabilities, each equipped with different optical sensors were used in this study. Experimental results demonstrate that the presented 4D point cloud analysis method can accurately detect and quantify a variety of anomalies from spalling to moisture infiltration in exposed concrete structures.


3D point cloud Unmanned Aerial Vehicle (UAV) Computer vision Structural health monitoring Damage detection 



This material is based upon the work supported by the National Science Foundation (NSF) under Grant No. CMMI-1433765. The authors would also like to acknowledge the support made by NVIDIA Corporation with the donation of a Tesla K40 GPU used in this research. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the NSF or NVIDIA.


  1. 1.
    Phares, B.M., Washer, G.A., Rolander, D.D., Graybeal, B.A., Moore, M.: Routine highway bridge inspection condition documentation accuracy and reliability. J. Bridg. Eng. 9, 403–413 (2004)CrossRefGoogle Scholar
  2. 2.
    Chang, P.C., Flatau, A., Liu, S.C.: Review paper: health monitoring of civil infrastructure. Struct. Health Monit. 2, 257–267 (2003)CrossRefGoogle Scholar
  3. 3.
    Koch, C., Georgieva, K., Kasireddy, V., Akinci, B., Fieguth, P.: A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv. Eng. Inform. 29, 196–210 (2015)CrossRefGoogle Scholar
  4. 4.
    Lattanzi, D., Miller, G.: Review of robotic infrastructure inspection systems. J. Infrastruct. Syst. 23, 04017004 (2017)CrossRefGoogle Scholar
  5. 5.
    Khaloo, A., Lattanzi, D.: Hierarchical dense structure-from-motion reconstructions for infrastructure condition assessment. J. Comput. Civ. Eng. 31, 04016047 (2016)CrossRefGoogle Scholar
  6. 6.
    McGuire, B., Atadero, R., Clevenger, C., Ozbek, M.: Bridge information modeling for inspection and evaluation. J. Bridg. Eng. 21, 04015076 (2016)CrossRefGoogle Scholar
  7. 7.
    Khaloo, A., Lattanzi, D., Cunningham, K., Dell’Andrea, R., Riley, M.: Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling. Struct. Infrastruct. Eng. 14(1), 124–136 (2018)CrossRefGoogle Scholar
  8. 8.
    Fathi, H., Dai, F., Lourakis, M.: Automated as-built 3D reconstruction of civil infrastructure using computer vision: achievements, opportunities, and challenges. Adv. Eng. Inform. 29, 149–167 (2015)CrossRefGoogle Scholar
  9. 9.
    Jafari, B., Khaloo, A., Lattanzi, D.: Deformation tracking in 3D point clouds via statistical sampling of direct cloud-to-cloud distances. J. Nondestruct. Eval. 36, 65 (2017)CrossRefGoogle Scholar
  10. 10.
    Pesci, A., Casula, G., Boschi, E.: Laser scanning the Garisenda and Asinelli towers in Bologna (Italy): detailed deformation patterns of two ancient leaning buildings. J. Cult. Herit. 12, 117–127 (2011)CrossRefGoogle Scholar
  11. 11.
    Kashani, A.G., Graettinger, A.J.: Cluster-based roof covering damage detection in ground-based lidar data. Autom. Constr. 58, 19–27 (2015)CrossRefGoogle Scholar
  12. 12.
    Gong, J., Maher, A.: Use of mobile lidar data to assess hurricane damage and visualize community vulnerability. Transp. Res. Rec. J. Transp. Res. Board. 2459, 119–126 (2014)CrossRefGoogle Scholar
  13. 13.
    Riveiro, B., González-Jorge, H., Varela, M., Jauregui, D.V.: Validation of terrestrial laser scanning and photogrammetry techniques for the measurement of vertical underclearance and beam geometry in structural inspection of bridges. Measurement. 46, 784–794 (2013)CrossRefGoogle Scholar
  14. 14.
    Liu, W., Chen, S., Hasuer, E.: Bridge clearance evaluation based on terrestrial LIDAR scan. J. Perform. Constr. Facil. 26, 469–477 (2012)CrossRefGoogle Scholar
  15. 15.
    Tang, P., Huber, D., Akinci, B.: Characterization of laser scanners and algorithms for detecting flatness defects on concrete surfaces. J. Comput. Civ. Eng. 25, 31–42 (2010)CrossRefGoogle Scholar
  16. 16.
    Armesto-González, J., Riveiro-Rodríguez, B., González-Aguilera, D., Rivas-Brea, M.T.: Terrestrial laser scanning intensity data applied to damage detection for historical buildings. J. Archaeol. Sci. 37, 3037–3047 (2010)CrossRefGoogle Scholar
  17. 17.
    Hou, T.-C., Liu, J.-W., Liu, Y.-W.: Algorithmic clustering of LiDAR point cloud data for textural damage identifications of structural elements. Measurement. 108, 77–90 (2017)CrossRefGoogle Scholar
  18. 18.
    Jiang, C., Tsai, Y.J.: Enhanced crack segmentation algorithm using 3D pavement data. J. Comput. Civ. Eng. 30, 04015050 (2016)CrossRefGoogle Scholar
  19. 19.
    Conde, B., Del Pozo, S., Riveiro, B., González-Aguilera, D.: Automatic mapping of moisture affectation in exposed concrete structures by fusing different wavelength remote sensors. Struct. Control Health Monit. 23, 923–937 (2016)CrossRefGoogle Scholar
  20. 20.
    Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. Presented at the Robotics-DL tentative (1992)Google Scholar
  21. 21.
    Toldo, R., Beinat, A., Crosilla, F.: Global registration of multiple point clouds embedding the Generalized Procrustes Analysis into an ICP framework. In: 3DPVT 2010 Conference (2010)Google Scholar
  22. 22.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP 1. 2, 2 (2009)Google Scholar
  23. 23.
    Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon, pp. 85–92. Moscow (2003)Google Scholar
  24. 24.
    Cheng, H.-D., Jiang, X.H., Sun, Y., Wang, J.: Color image segmentation: advances and prospects. Pattern Recogn. 34, 2259–2281 (2001)CrossRefGoogle Scholar
  25. 25.
    Son, H., Kim, C., Kim, C.: Automated color model–based concrete detection in construction-site images by using machine learning algorithms. J. Comput. Civ. Eng. 26, 421–433 (2012)CrossRefGoogle Scholar
  26. 26.
    Siegel, M., Gunatilake, P.: Remote enhanced visual inspection of aircraft by a mobile robot. In: Proc. of the 1998 IEEE Workshop on Emerging Technologies, Intelligent Measurement and Virtual Systems for Instrumentation and Measurement (ETIMVIS’98), pp. 49–58 (1998)Google Scholar
  27. 27.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, New Jersey (2008)Google Scholar
  28. 28.
    Szeliski, R.: Computer Vision. Springer London, London (2011)CrossRefGoogle Scholar
  29. 29.
    Westland, S., Ripamonti, C., Cheung, V.: Computational Colour Science Using MATLAB. Wiley, West Sussex (2012)CrossRefGoogle Scholar
  30. 30.
    Men, H., Pochiraju, K.: Hue-assisted automatic registration of color point clouds. J. Comput. Des. Eng. 1, 223–232 (2014)Google Scholar
  31. 31.
    Choi, K.Y., Kim, S.S.: Morphological analysis and classification of types of surface corrosion damage by digital image processing. Corros. Sci. 47, 1–15 (2005)CrossRefGoogle Scholar
  32. 32.
    Aligholi, S., Lashkaripour, G.R., Khajavi, R., Razmara, M.: Automatic mineral identification using color tracking. Pattern Recogn. 65, 164–174 (2017)CrossRefGoogle Scholar
  33. 33.
    Wesolkowski, S.: Color image edge detection and segmentation: a comparison of the vector angle and the euclidean distance color similarity measures. (1999)

Copyright information

© The Society for Experimental Mechanics, Inc. 2019

Authors and Affiliations

  1. 1.Department of Civil, Environmental, and Infrastructure EngineeringGeorge Mason UniversityFairfaxUSA

Personalised recommendations