Skip to main content
SpringerLink
Log in

UAV-Based Visual Remote Sensing for Automated Building Inspection

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13807))

Included in the following conference series:

  • 1665 Accesses

Abstract

Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component’s contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to automate these inspections through UAV-based image data collection and a software library for post-processing that helps in estimating the seismic structural parameters. The key parameters considered here are the distances between adjacent buildings, building plan-shape, building plan area, objects on the rooftop and rooftop layout. The accuracy of the proposed methodology in estimating the above-mentioned parameters is verified through field measurements taken using a distance measuring sensor and also from the data obtained through Google Earth. Additional details and code can be accessed from https://uvrsabi.github.io/.

K. Srivastava and D. Patel—denotes equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    UAV specification details can be found at the official DJI website: https://www.dji.com/mavic-mini.

  2. 2.

    The ToF sensor can be found at: https://www.terabee.com/shop/lidar-tof-range-finders/teraranger-evo-60m/.

References

  1. Spencer, B.F., Hoskere, V., Narazaki, Y.: Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering 5(2), 199–222 (2019). ISSN 2095-8099. https://doi.org/10.1016/j.eng.2018.11.030, https://www.sciencedirect.com/science/article/pii/S2095809918308130

  2. Abdel-Qader, I., Abudayyeh, O., Kelly, M.: Analysis of edge-detection techniques for crack identification in bridges. J. Comput. Civ. Eng. 17, 10 (2003). https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)

    Article  Google Scholar 

  3. Zhang, W., Zhang, Z., Qi, D., Liu, Y.: Automatic crack detection and classification method for subway tunnel safety monitoring. Sensors 14(10), 19307–19328 (2014). ISSN 1424-8220. https://doi.org/10.3390/s141019307, https://www.mdpi.com/1424-8220/14/10/19307

  4. Liu, Y.-F., Cho, S., Spencer, B., Fan, J.-S.: Concrete crack assessment using digital image processing and 3D scene reconstruction. J. Comput. Civ. Eng. 30, 04014124 (2014). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000446

  5. Adhikari, R.S., Moselhi, O., Bagchi, A.: A study of image-based element condition index for bridge inspection. In: ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, vol. 30, p. 1. IAARC Publications (2013)

    Google Scholar 

  6. Paal, S., Jeon, J.-S., Brilakis, I., Desroches, R.: Automated damage index estimation of reinforced concrete columns for post-earthquake evaluations. J. Struct. Eng. 141, 04014228 (2015). https://doi.org/10.1061/(ASCE)ST.1943-541X.0001200

    Article  Google Scholar 

  7. Yeum, C.M., Dyke, S.J.: Vision-based automated crack detection for bridge inspection. Comput.-Aided Civ. Infrastruct. Eng. 30(10), 759–770 (2015)

    Article  Google Scholar 

  8. Jahanshahi, M.R., Chen, F.-C., Joffe, C., Masri, S.F.: Vision-based quantitative assessment of microcracks on reactor internal components of nuclear power plants. Struct. Infrastruct. Eng. 13(8), 1013–1026 (2017)

    Article  Google Scholar 

  9. Son, H., Hwang, N., Kim, C., Kim, C.: Rapid and automated determination of rusted surface areas of a steel bridge for robotic maintenance systems. Autom. Constr. 42, 13–24 (2014)

    Article  Google Scholar 

  10. Shen, H.-K., Chen, P.-H., Chang, L.-M.: Automated steel bridge coating rust defect recognition method based on color and texture feature. Autom. Constr. 31, 338–356 (2013)

    Article  Google Scholar 

  11. Medeiros, F.N.S., Ramalho, G.L.B., Bento, M.P., Medeiros, L.C.L.: On the evaluation of texture and color features for nondestructive corrosion detection. EURASIP J. Adv. Signal Process. 2010, 1–7 (2010)

    Article  Google Scholar 

  12. Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)

    Article  Google Scholar 

  13. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3708–3712. IEEE (2016)

    Google Scholar 

  14. Atha, D.J., Jahanshahi, M.R.: Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct. Health Monit. 17(5), 1110–1128 (2018)

    Article  Google Scholar 

  15. Yeum, C.M., Dyke, S.J., Ramirez, J.: Visual data classification in post-event building reconnaissance. Eng. Struct. 155, 16–24 (2018)

    Article  Google Scholar 

  16. Cha, Y.-J., Choi, W., Suh, G., Mahmoudkhani, S., Büyüköztürk, O.: Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civ. Infrastruct. Eng. 33(9), 731–747 (2018)

    Article  Google Scholar 

  17. Zhang, A., et al.: Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)

    Article  Google Scholar 

  18. Hoskere, V., Narazaki, Y., Hoang, T., Spencer Jr., B.F.: Vision-based structural inspection using multiscale deep convolutional neural networks. arXiv preprint arXiv:1805.01055 (2018)

  19. Hoskere, V., Narazaki, Y., Hoang, T.A., Spencer Jr., B.F.: Towards automated post-earthquake inspections with deep learning-based condition-aware models. arXiv preprint arXiv:1809.09195 (2018)

  20. Zhu, Z., Brilakis, I.: Concrete column recognition in images and videos. J. Comput. Civ. Eng. 24(6), 478–487 (2010)

    Article  Google Scholar 

  21. Koch, C., Paal, S.G., Rashidi, A., Zhu, Z., König, M., Brilakis, I.: Achievements and challenges in machine vision-based inspection of large concrete structures. Adv. Struct. Eng. 17(3), 303–318 (2014)

    Article  Google Scholar 

  22. Xiong, X., Adan, A., Akinci, B., Huber, D.: Automatic creation of semantically rich 3d building models from laser scanner data. Autom. Constr. 31, 325–337 (2013)

    Article  Google Scholar 

  23. Armeni, I., et al.: 3D semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1534–1543 (2016)

    Google Scholar 

  24. Golparvar-Fard, M., Bohn, J., Teizer, J., Savarese, S., Peña-Mora, F.: Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques. Autom. Constr. 20(8), 1143–1155 (2011)

    Article  Google Scholar 

  25. Lu, R., Brilakis, I., Middleton, C.R.: Detection of structural components in point clouds of existing RC bridges. Comput.-Aided Civ. Infrastruct. Eng. 34(3), 191–212 (2019)

    Article  Google Scholar 

  26. Gao, Y., Mosalam, K.M.: Deep transfer learning for image-based structural damage recognition. Comput.-Aided Civ. Infrastruct. Eng. 33(9), 748–768 (2018)

    Article  Google Scholar 

  27. Liang, X.: Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization. Comput.-Aided Civ. Infrastruct. Eng. 34(5), 415–430 (2019)

    Article  MathSciNet  Google Scholar 

  28. Yeum, C.M., Choi, J., Dyke, S.J.: Automated region-of-interest localization and classification for vision-based visual assessment of civil infrastructure. Struct. Health Monit. 18(3), 675–689 (2019)

    Article  Google Scholar 

  29. Narazaki, Y., Hoskere, V., Hoang, T.A., Fujino, Y., Sakurai, A., Spencer Jr., B.F.: Vision-based automated bridge component recognition with high-level scene consistency. Comput.-Aided Civ. Infrastruct. Eng. 35(5), 465–482 (2020)

    Google Scholar 

  30. Dimitrov, A., Golparvar-Fard, M.: Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections. Adv. Eng. Inform. 28(1), 37–49 (2014)

    Article  Google Scholar 

  31. Golparvar-Fard, M., Pena-Mora, F., Savarese, S.: Automated progress monitoring using unordered daily construction photographs and IFC-based building information models. J. Comput. Civ. Eng. 29(1), 04014025 (2015)

    Article  Google Scholar 

  32. Hamledari, H., Davari, S., Azar, R., McCabe, B., Flager, F., Fischer, M.: UAV-enabled site-to-BIM automation: aerial robotic-and computer vision-based development of as-built/as-is BIMs and quality control. In: Construction research congress, pp. 336–346 (2017)

    Google Scholar 

  33. Pelizari, P.A., Geiß, C., Aguirre, P., María, H.S., Peña, Y.M., Taubenböck, H.: Automated building characterization for seismic risk assessment using street-level imagery and deep learning. ISPRS J. Photogram. Remote Sens. 180, 370–386 (2021). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2021.07.004, https://www.sciencedirect.com/science/article/pii/S0924271621001817

  34. Gonzalez, D., et al.: Automatic detection of building typology using deep learning methods on street level images. Build. Environ. 177, 106805 (2020). ISSN 0360-1323. https://doi.org/10.1016/j.buildenv.2020.106805, https://www.sciencedirect.com/science/article/pii/S0360132320301633

  35. Hackl, J., Adey, B., Woźniak, M., Schümperlin, O.: Use of unmanned aerial vehicle photogrammetry to obtain topographical information to improve bridge risk assessment. J. Infrastruct. Syst. 24, 04017041 (2018). https://doi.org/10.1061/(ASCE)IS.1943-555X.0000393

  36. Miari, M., Choong, K.K., Jankowski, R.: Seismic pounding between adjacent buildings: identification of parameters, soil interaction issues and mitigation measures. Soil Dyn. Earthq. Eng. 121, 135–150 (2019). ISSN 0267-7261. https://doi.org/10.1016/j.soildyn.2019.02.024, https://www.sciencedirect.com/science/article/pii/S0267726118313848

  37. Carboney, J.A., García, H.J., Ortega, R., Iglesias, J.: The Mexico earthquake of September 19, 1985 - statistics of damage and of retrofitting techniques in reinforced concrete buildings affected by the 1985 earthquake. Earthq. Spectra 5 (1989). https://doi.org/10.1193/1.1585516

  38. Kasai, K., Maison, B.F.: Building pounding damage during the 1989 Loma Prieta earthquake. Eng. Struct. 19(3): 195–207 (1997). ISSN 0141-0296. https://doi.org/10.1016/S0141-0296(96)00082-X, https://www.sciencedirect.com/science/article/pii/S014102969600082X

  39. Lin, J.-H., Weng, C.-C.: A study on seismic pounding probability of buildings in Taipei metropolitan area. J. Chin. Inst. Eng. 25(2), 123–135 (2002). https://doi.org/10.1080/02533839.2002.9670687, https://doi.org/10.1080/02533839.2002.9670687

  40. Kaushik, H.B., Da, K., Sahoo, D.R., Kharel, G.: Performance of structures during the Sikkim earthquake of 14 February 2006. Curr. Sci. 91, 449–455 (2006)

    Google Scholar 

  41. Bektaş, N., Kegyes-Brassai, O.: Conventional RVS methods for seismic risk assessment for estimating the current situation of existing buildings: a state-of-the-art review. Sustainability 14(5) (2022). ISSN 2071-1050. https://doi.org/10.3390/su14052583, https://www.mdpi.com/2071-1050/14/5/2583

  42. Ramancharla, P., et al.: A primer on rapid visual screening (RVS) consolidating earthquake safety assessment efforts in India (2020)

    Google Scholar 

  43. Vacca, G., Dessì, A., Sacco., A.: The use of nadir and oblique UAV images for building knowledge. ISPRS Int. J. Geo-Inf. 6, 393 (2017). https://doi.org/10.3390/ijgi6120393

  44. Arnold, C., Reitherman, R.: Building Configuration and Seismic Design. Wiley, Hoboken (1982)

    Google Scholar 

  45. Sahar, L., Muthukumar, S., French, S.P.: Using aerial imagery and GIS in automated building footprint extraction and shape recognition for earthquake risk assessment of urban inventories. IEEE Trans. Geosci. Remote Sens. 48(9), 3511–3520 (2010)

    Article  Google Scholar 

  46. Chen, D., Zhang, L., Li, J., Liu, R.: Urban building roof segmentation from airborne lidar point clouds. Int. J. Remote Sens. 33(20), 6497–6515 (2012). https://doi.org/10.1080/01431161.2012.690083

    Article  Google Scholar 

  47. Vo, A.-V., Truong-Hong, L., Laefer, D.F., Bertolotto, M.: Octree-based region growing for point cloud segmentation. ISPRS J. Photogram. Remote Sens. 104, 88–100 (2015). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2015.01.011, https://www.sciencedirect.com/science/article/pii/S0924271615000283

  48. Sampath, A., Shan, J.: Segmentation and reconstruction of polyhedral building roofs from aerial lidar point clouds. IEEE Trans. Geosci. Remote Sens. 48(3), 1554–1567 (2010). https://doi.org/10.1109/TGRS.2009.2030180

    Article  Google Scholar 

  49. Dong, Z., Yang, B., Hu, P., Scherer, S.: An efficient global energy optimization approach for robust 3D plane segmentation of point clouds. ISPRS J. Photogram. Remote Sens. 137, 112–133 (2018). ISSN 0924-2716. https://doi.org/10.1016/j.isprsjprs.2018.01.013, https://www.sciencedirect.com/science/article/pii/S0924271618300133

  50. Li, W., He, C., Fang, J., Zheng, J., Fu, H., Yu, L.: Semantic segmentation-based building footprint extraction using very high-resolution satellite images and multi-source GIS data. Remote Sens. 11(4) (2019). ISSN 2072-4292. https://doi.org/10.3390/rs11040403, https://www.mdpi.com/2072-4292/11/4/403

  51. Qin, Y., Wu, Y., Li, B., Gao, S., Liu, M., Zhan, Y.: Semantic segmentation of building roof in dense urban environment with deep convolutional neural network: a case study using GF2 VHR imagery in China. Sensors 19, 1164 (2019). https://doi.org/10.3390/s19051164

    Article  Google Scholar 

  52. Bhat, A.S., Shivaprakash, A.V., Prasad, N.S., Nagaraj, C.: Template matching technique for panoramic image stitching. In: 2013 7th Asia Modelling Symposium, pp. 111–115 (2013). https://doi.org/10.1109/AMS.2013.22

  53. Adwan, S., Alsaleh, I., Majed, R.: A new approach for image stitching technique using dynamic time warping (DTW) algorithm towards scoliosis X-ray diagnosis. Measurements 84, 32–46 (2016). https://doi.org/10.1016/j.measurement.2016.01.039

    Article  Google Scholar 

  54. Bonny, M., Uddin, M.: A technique for panorama-creation using multiple images. Int. J. Adv. Comput. Sci. Appl. 11 (2020). https://doi.org/10.14569/IJACSA.2020.0110293

  55. Alomran, M., Chai, D.: Feature-based panoramic image stitching. In: 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1–6 (2016). https://doi.org/10.1109/ICARCV.2016.7838721

  56. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004). ISSN 1573-1405. https://doi.org/10.1023/B:VISI.0000029664.99615.94

  57. Zhang, Y., Yang, L., Wang, Z.: Research on video image stitching technology based on surf. In: 2012 Fifth International Symposium on Computational Intelligence and Design, vol. 2, pp. 335–338 (2012). https://doi.org/10.1109/ISCID.2012.235

  58. Tsao, P., Ik, T.-U., Chen, G.-W., Peng, W.-C.: Stitching aerial images for vehicle positioning and tracking. In: 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 616–623. IEEE (2018)

    Google Scholar 

  59. Liu, Y., Xue, F., Huang, H.: UrbanScene3D: a large scale urban scene dataset and simulator (2021)

    Google Scholar 

  60. Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2

  61. Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31

    Chapter  Google Scholar 

  62. Wang, Y., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1860–1864 (2019). https://doi.org/10.1109/ICIP.2019.8803154

  63. Zhou, X., Girdhar, R., Joulin, A., Krähenbühl, P., Misra, I.: Detecting twenty-thousand classes using image-level supervision. arXiv preprint arXiv:2201.02605 (2021)

Download references

Acknowledgement

The authors acknowledge the financial support provided by IHUB, IIIT Hyderabad to carry out this research work under the project: IIIT-H/IHub/Project/Mobility/2021-22/M2-003.

Author information

Authors and Affiliations

  1. International Institute of Information Technology, Hyderabad, Hyderabad, India

    Kushagra Srivastava, Dhruv Patel, Ravi Kiran Sarvadevabhatla, Pradeep Kumar Ramancharla, Harikumar Kandath & K. Madhava Krishna

  2. Indian Institute of Technology, Kharagpur, Kharagpur, India

    Aditya Kumar Jha & Mohhit Kumar Jha

  3. University of Petroleum and Energy Studies, Dehradun, India

    Jaskirat Singh

Authors
  1. Kushagra Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dhruv Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aditya Kumar Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohhit Kumar Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jaskirat Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ravi Kiran Sarvadevabhatla
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pradeep Kumar Ramancharla
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Harikumar Kandath
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. K. Madhava Krishna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kushagra Srivastava .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Srivastava, K. et al. (2023). UAV-Based Visual Remote Sensing for Automated Building Inspection. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13807. Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25082-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25081-1

  • Online ISBN: 978-3-031-25082-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation