Abstract
The paper focuses on a review of augmented reality/mixed reality (AR/MR) technologies in the perspective of the application to structural health monitoring (SHM) for submerged floating tunnels (SFT). In particular, Microsoft HoloLens Devices (MHD) are precious tools for survey purposes, since they allow the superimposition of information about the surrounding environment, leading to its complete control with a reduction of errors. Preliminary, the most common SHM techniques in traditional waterway crossings are examined, for identifying those best fitting with SFTs. The infrared (IR) thermography for damage detection is deepen. The possibility of connecting IR Camera and MHD, with the aim at facilitating the acquisition of thermal data and enabling stereoscopic thermal overlays in the worker’s augmented view, is examined through three different experiments, namely steel fatigue crack detection on a steel plate, CFRP delamination detection and coating thickness detection on a steel members. To this purpose, a simulation of the combination of thermal image with HoloLens is performed showing the potentiality of how the view of the thermal camera in front of the user can enhance the perception of possible problems on the target structure earlier with respect to analyzing the data afterwards.
Similar content being viewed by others
Data availability
All data generated and analysed during this study are included in this published article and available from corresponding author on specific reasonable requests.
References
Rytter A (1993) Vibrational based inspection of civil engineering structures. Aalborg. Denmark, Publisher: Dept. of building technology and structural engineering. Series: fracture and dynamics number: 44, Vol. R9314. ISSN0902–7513.
Remseth S, Leira B-J, Rönnquist A, Udahl G (1999) Dynamic response and fluid/structure interaction of submerged floating tunnels. Computers & Structures. Ph.D.-Thesis defended publicly at the University of Aalborg, April 20, 1993.
Xiang Y, Yang Y (2016) Challenge in design and construction of submerged floating tunnel and state-of-art. Procedia Eng 166(2016):53–60. https://doi.org/10.1016/j.proeng.2016.11.562
Teknologidagene K (2018) [Online]. Available: https://www.vegvesen.no/_attachment/2487060/binary/1294864?fast_title=Research+center+for+smart+submerged+floating+tunnel+system.pdf.
Seo S-I, Sagong M, Son S-W (2015) Global response of submerged floating tunnel against underwater explosion. KSCE J Civ Eng 19:2029–2034
Costanzo FA (2011) Underwater explosion phenomena and shock physics. Structural dynamics. Springer, New York, NY, pp 917–938
Liang B, Jiang B (2016) Study on composition and simulation analysis of traffic loads in submerged floating tunnels. Procedia Eng 166:180–189. https://doi.org/10.1016/j.proeng.2016.11.581
Martinelli L (2018) Submerged floating tunnels under seismic and seaquake loadings. 9th maintenance, safety, risk, management and life-cycle performance of bridges (IABMAS), Melbourne, Australia. 1st Edition, 2018. Editors: Nigel Powers, Dan Frangopol, Riadh Al-Mahaidi, Colin Caprani. Publisher: CRC Press (ISBN: 9781315189390).
Giurgiutiu V (2014) Structural health monitoring with piezoelectric wafer active sensors—predictive modeling and simulation. Incas Bull 2(3):31–44. https://doi.org/10.13111/2066-8201.2010.2.3.4
Hansen E, Zilberstein S (2001) Monitoring and control of anytime algorithms: a dynamic programming approach. Artif Intel 126(1–2):139–157. https://doi.org/10.1016/S0004-3702(00)00068-0
Yun C-B, Bahng E-Y (2000) Substructural identification using neural networks. Comput Struct 77(1):41–52. https://doi.org/10.1016/S0045-7949(99)00199-6
Brownjohn J-M-W, Moyo P, Rizos C, Chuan T-S (2003) Practical issues in using novel sensors in SHM of civil infrastructure: problems and solutions in implementation of GPS and fibre optics. Proc., 4th Int. Workshop on SHM, Stanford Univ., Stanford, CA, 499–506.
Setec - Regular inspection Tarascon - Beaucaira bridge https://www.setec.fr/en/realisations/inspection-dtaille-priodique-pont-de-tarascon-beaucaire
Boller C (2006) Structural health monitoring – a design and integration issue. 1st Asia-Pacific workshop on structural health monitoring, December 4-6, 2006, Yokohama, Japan. Vol. 1, 647–657.
Oliveira F-G-M, Morais P-G, Freitas A (2011) 400 mm base extensometer for measurement of structural deformations. Experimental vibration analysis for civil engineering structures (EVACES 2011), Italy, Verona. Editors: Zhishen Wu, Tomonori Nagayama, Ji Dang, Rodrigo Astroza. Publisher: EDP Sciences (ISBN: 9781510814578).
Goanta V (2019) Extensometer for determining strains on a tensile and torsion simultaneous load. Sensors (Basel) 20(2):385. https://doi.org/10.3390/s20020385
Gutkowski R, Robinson G, Peterson M, Tran A (1999) Field load testing of open-deck timber trestle railroad bridges. In Proc., RILEM symposium on timber engineering, Sept. 13–15, Stockholm, Sweden.
Epsilontech [Online]. Available: https://www.epsilontech.com/products/bolt-elongation-extensometer-model-3565/.
Althensensors [Online]. Available: https://www.althensensors.com/sensors/fiber-optical-sensors/fiber-optic-extensometers/4947/oep-c-fiber-optic-extensometer/
Oliveira S-L (2014). Monitoramento e ensaio de pontes. VII Congresso Brasileiro de Pontes e Estruturas, Rio de Janeiro, 21–23 May, 2014. Editor: Starrylink.
Hbk Company, 2020 [Online]. Available: https://www.hbm.com/en/0236/displacement-sensors-and-displacement-transducers/?product_type_no=Displacement%20Sensor.
Fieldsr [Online]. Available: https://www.fieldsrl.it/en/wms/inclinometer-section/
Domaneschi M, Sigurdardottir D, Glišic B (2017) Damage detection on output-only monitoring of dynamic curvature in composite decks. Struct Monit Maint 4(1):1–15. https://doi.org/10.12989/smm.2017.4.1.001
Casciati S, Domaneschi M (2005) Damage assessment from SOFO dynamic measurements. Proceedings of SPIE - 17th international conference on optical fibre sensors. Marc Voet, Reinhardt Willsch, Wolfgang Ecke, Julian Jones, Brian Culshaw, eds. Vol. 5855. DOI: https://doi.org/10.1117/12.623656.
Domaneschi M, Niccolini G, Lacidogna G, Cimellaro G-P (2020) Non destructive monitoring techniques for crack detection and localization in RC elements. Appl Sci 10:3248. https://doi.org/10.3390/app10093248
Morgese M, Domaneschi M, Ansari F, Cimellaro G-P, Inaudi D (2021) Improving distributed fiber-optic sensor measures by digital image correlation: two-stage structural health monitoring. ACI Struct J. https://doi.org/10.14359/51732994
Trimble 2018 [Online]. Available: https://www.trimble-italia.com/prodotti/software/trimble-access.
Golaski L, Gebsky P, Ono K (2002) Diagnostics of reinforced concrete bridges by acoustic emission. Journal of acoustic emission 20 (1), 83-98, 2002. Acoustic Emission Group (ISSN: 0730-0050).
Wickert J, Beyerle G, Hajj G, Schwieger V, Reigber C (2002) GPS radio occultation with champ: atmospheric profiling utilizing the space-based single difference technique. Geophys Res Lett. https://doi.org/10.1029/2001GL013982
Jelalian A (1980) Laser radar systems. EASCON '80; electronics and aerospace systems conference, Arlington, VA, September 29-October 1, 1980.
Meng X, Gogoi N (2011) Using multi-constellation GNSS and EGNOS for bridge deformation monitoring. Proceedings of the joint international symposium on deformation monitoring. Hong Kong, China, p. 2–4.
Chen S-E (2012) Laser scanning technology for bridge monitoring. In: Apolinar Munoz J (ed) Rodriguez laser scanner technology. InTech, Janeza Trdine, Rijeka, Croatia (ISBN: 978-953-51-0280-9).
Harding G, Kosanetzky J-M (1989) Scattered X-ray beam nondestructive testing. Nucl Instrum Methods Phys Res Sect A 280(2–3):517–528
Novatest, 2019. [Online]. Available: https://www.novatest.it/2019/11/15/la-radiografia-digitale-come-tecnica-di-ispezione-delle-infrastrutture/.
Park B, Yeum C, Thanh CT (2013) Laser ultrasonic imaging and damage detection for a rotating structure. Struct Health Monit 12(5–6):494–506. https://doi.org/10.1177/1475921713507100
Pan M, He Y, Tian G, Chen D, Luo F (2012) Defect characterisation using pulsed eddy current thermography under transmission mode and NDT applications. NDT E Int 52:28–36. https://doi.org/10.1016/j.ndteint.2012.08.007
Rosen-Group. (2016, March 22nd). Measurement technology - Eddy current. Retrieved from https://www.rosen-group.com/global/company/explore/we-can/technologies/measurement/eddy-current.html
Libby H (1971) Introduction to electromagnetic nondestructive test methods. Krieger Pub Co, Oregon
Merten A (2010) Non-destructive test methods for hollow-core composite insulators. 2010 International conference on high voltage engineering and application. New Orleans, LA, USA. DOI: https://doi.org/10.1109/ICHVE.2010.5640709.
Zhao Q, Zhao Q, Dan X, Sun F, Wang Y, Wu S, Yang L (2018) Digital shearography for NDT: phase measurement technique and recent developments. Appl Sci 8(12):2662
Maldague X (2002) Introduction to NDT by active infrared thermography. Materials Evaluation 60 (9), 1060–1073.
Liu P, Lim HJ, Park B (2019) Noncontact nonlinear ultrasonic wave modulation for fatigue crack and delamination detection. NDT E Int 66:106–116. https://doi.org/10.1016/j.ndteint.2014.06.002
I. D. C. IDC, IDC (2019) Expects augmented and virtual reality spending [Online]. Available: https://www.idc.com/getdoc.jsp?containerId=prEUR247725821.
Steve M, Furness T, Yuan Y, Iorio J, Wang Z (2018) All reality: virtual, augmented, mixed (X), mediated (X,Y), and multimediated reality. ArXiv:1804.08386 [Cs]. Retrieved from http://arxiv.org/abs/1804.08386
Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Transa Inform Syst 77(12):1321–1329
Palmarini R (2017) A systematic review of augmented reality applications in maintenance. Robot Comput-Integr Manuf 49:215–228
Silva A-D (2009) Hybrid reality and location-based gaming: redefining mobility and game spaces in urban environments. Simul Gaming 40(3):404–424. https://doi.org/10.1177/1046878108314643
Corporation I-D (2019) IDC expects augmented and virtual reality spending. Retrieved from https://www.idc.com/getdoc.jsp?containerId=prAP44957819.
Trimble Inc. (2019). Trimble XR10 with HoloLens. https://fieldtech.trimble.com/en/products/mixed-reality/trimble-xr10-with-hololens-2. Accessed 01 oct 2021.
Gervautz M, Schmalstieg D (2012) Anywhere interfaces using handheld augmented reality. Computer 45(7):26–31. https://doi.org/10.1109/MC.2012.72
Pluraview (2020) [Online]. Available: https://www.3d-pluraview.com/en/
UploadVR (2017) [Online]. Available: https://uploadvr.com/hololamp-projector-ar-holograms/
Google glass surgeon (2013) Retrieved from http://www.googleglasssurgeon.com/surgery
Microsoft (2017). Windows mixed reality—gestures. Retrieved from https://developer.microsoft.com/en-us/windows/mixed-reality/gestures
Rubino D (2016) Microsoft HoloLen–here are the full processor, storage. Retrieved from https://www.windowscentral.com/microsoft-hololens-processor-storage-and-ram
Taylor A (2017) Develop microsoft HoloLens app now. Apress, Oregon, USA. https://doi.org/10.1007/978-1-4842-2202-7 (ISBN: 978-1-4842-2201-0)
Harsha S, Bhavya G (2018) Google glass. International Journal of Advance Research, Ideas and Innovations in Technology. 4(3). IJARIIT.
Jasoren [Online]. Available: https://jasoren.com/augmented-reality-military/
Taylor A (2019). Microsoft dynamics_365. Retrieved from https://blogs.microsoft.com/blog/2018/09/18/announcing-new-ai-and-mixed-reality-business-applications-for-microsoft-dynamics/
Lee G-A, Duenser A (2012) CityViewAR: a mobile outdoor AR application for city visualization. IEEE international symposium on mixed and augmented reality–Arts, Media, and Humanities (ISMAR-AMH), Atlanta, USA, 2012. Editor: University of Canterbury. Human Interface Technology Laboratory
OutSource2India (2018) Retrieved from https://www.outsource2india.com/eso/articles/augmented-reality-can-help-architects-engineers.asp#
Engadget (2017) Retrieved from https://www.engadget.com/2017/01/25/hololens-is-helping-engineers-visualize-building-blueprints/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAABU9DZXLrQRfua4TE0yOH7WyqcTFxq263mmNr7FFL5weqgyQrEHLybYJQSsu1yRlJT6BgDumers7f
GA smart building (2018) Retrieved from https://www.ga.fr/en/
Bentley_systems (2017) Retrieved from https://communities.bentley.com/other/old_site_member_blogs/bentley_employees/b/stephanecotes_blog/posts/using-the-hololens-for-accurate-subsurface-utility-pipes-augmented-reality
Wu Z, Mei G (2017) Dynamic response analysis of cable of submerged floating tunnel under hydrodynamic force and earthquake. Shock Vib 2017(1):1–14. https://doi.org/10.1155/2017/3670769
ITA. (2011). Monitoring and control in tunnel construction. WG2 Research.
Ki J-M, Lee J, Sohn H (2018) Detection of tension force reduction in a post-tensioning tendon using pulsed-eddy-current measurement. Struct Eng Mec 65(2):129–139. https://doi.org/10.12989/SEM.2018.65.2.129
Sheppard R, Spong R, Sauls M, Hosch P, Danaczko M, Puskar F, Leverette S (2019) Integrity management process of tension leg platforms. Offshore technology conference, Houston, Texas, May 2019. Paper num.: OTC-29661-MS. https://doi.org/10.4043/29661-MS.
Hong K-Y, Lee G-H (2014) Collision analysis of submerged floating tunnel by underwater navigating vessel. J Comput Struct Eng Inst Korea 27(5):369–377
Lee Y, Kim D-M, Han S-H, Park W-S (2013) Impact analysis of submerged floating tunnel for external collision. Proceedings of the 7th international conference on Asian and Pacific coasts (APAC 2013) Bali, Indonesia, September 24–26, 2013, p. 214–219. Editors: Dadang A. Suriamihardja, T. Harianto, M. A. Abdurrahman, T. Rachman. ISBN: 978–979–530–125–7.
Faggiano B, Iovane G, Mazzolani F-M, Landolfo R (2021) Preliminary study on the behaviour of the SFT Qiandao prototype against explosions and impacts . Proceedings of the 14th international conference on vibration problems (ICOVP 2019), 1–4september, Crete, Greece, E. J. Sapountzakis et al. (eds.), Lecture Notes in Mechanical Engineering, SPRINGER NATURE Singapore Pte Ltd. 2021 https://doi.org/10.1007/978-981-15-8049-9_50.
Almir D, Chang H (2012) Underwater wireless sensor networks. Oceans 2012, Hampton Roads, VA, USA, 14-19 October 2012, 1–5. IEEE Xplore, Piscataway. https://doi.org/10.1109/oceans.2012.6405141 (ISSN: 0197-7385)
Maldague X. (2001). Theory and Practice of Infrared Technology for Nondestructive Testing. John Wiley & Son, New York, 2001 (ISBN: 978-0-471-18190-3).
Kayes RJ (1977) Optical and infrared detectors. Appl Phys. https://doi.org/10.1002/piuz.19790100308
Steen W-M, Mazumder J (2010) Laser material processing. Springer Science & Business Media, Arrington, UK
Hwang S (2016) Development of a line laser thermography system for structural damage detection under dynamic conditions. Advisors: Sohn, Hoon; Son Hoon; Korea Advanced institute of science and technology: department of construction and environmental engineering, Publisher: Korea advanced institute of science and technology. Thesis (Master) - Korea advanced institute of science and technology: department of construction and environmental engineering, 2016.8, [p. num. 47 ]
Ma J, Niu X, Liu X, Wang Y, Wen T, Zhang J (2019) Thermal infrared imagery integrated with terrestrial laser scanning and particle tracking velocimetry for characterization of landslide model failure. Sensors (Basel). 20(1):219. https://doi.org/10.3390/s20010219
Castanedo C-I, Genest M, Piau J, Guibert S, Bendada A, Maldague X (2007) Active infrared thermography techniques for the nondestructive testing of materials. Ultrasonic and Advanced Methods for Nondestructive Testing and Material Characterization. World Scientific Publishing Company; 1st edition: Chapter X, 325-348, 2007 (ISBN-10: 9812704094).
Kaplan H (1999) Practical applications of infrared thermal sensing and imaging equipmen, 3rd edn. SPIE Publications, Bellingham, Washington USA. https://doi.org/10.1117/3.725072 (ISBN: 9780819467232)
Martin L, Zhao H, Ganippa L (2013) Gas phase thermometry of hot turbulent jets using laser induced phosphorescence. Opt Express 21(10):12260–12281. https://doi.org/10.1364/OE.21.012260
Ciampa F, Mahmoodi P, Pinto F, Meo M (2018) Recent advances in active infrared thermography for non-destructive testing of aerospace components. Sensors 18(2):609. https://doi.org/10.3390/s18020609
Tenek L-H, Henneke E-G (1991) Flaw dynamics and vibro-thermographic thermoelastic NDE of advanced composite materials. In: Baird GS (ed) Thermosense XIII, vol 1467. SPIE, Bellingham, Washington USA, 252–263. https://doi.org/10.1117/12.46440
Ambrozinski L, Piwakowski B, Pieczonka L, Uhl T (2014) Pitch-catch air-coupled ultrasonic technique for detection of barely visible impact damages in composite laminates. EWSHM–7th European workshop on structural health monitoring, IFFSTTAR, Inria, Université de Nantes, Jul 2014, Nantes, France. ffhal-01022043f.
Ghosh K-K, Karbhari V-M (2011) Use of infrared thermography for quantitative non-destructive evaluation in FRP strengthened bridge systems. Mater Struct/Materiaux et Constr 44:169–185. https://doi.org/10.1617/s11527-010-9617-5. (ISSN: 1359-5997)
Hiasa S, Catbas N, Matsumoto M, Mitani K (2016) Monitoring concrete bridge decks using infrared thermography with high speed vehicles. Struct Monit Maint Int J 3:277–296. https://doi.org/10.12989/smm.2016.3.3.277
Matsumoto M, Mitani K, Masanobu S, Kazuaki H, Rick M (2012) Innovative bridge assessment methods using image processing and infrared thermography technology. IABSE Congress Report. 18. https://doi.org/10.2749/222137912805111636.
MATLAB - Code compatibility analyzer R2022a, 2022. MathWorks. End-User License Agreement (EULA).
IRBIS3 Pro - Infrared thermographic software by INFRATEC, 2008. https://www.infratec.eu/thermography/thermographic-software/#c47500
Hwang S, An Y-K, Kim J-M, Sohn H (2019) Monitoring and instantaneous evaluation of fatigue crack using integrated passive and active laser thermography. Opt Lasers Eng 119:9–17. https://doi.org/10.1016/j.optlaseng.2019.02.001
Hwang S, An Y-K, Sohn H (2017) Continuous line laser thermography for damage imaging of rotating wind turbine blades. Procedia Eng 188(2017):225–232
Hong X-G, Liu H (2014) Roberts edge detection algorithm based on GPU. J Chem Pharm Res 6(7):1308–1314
Gonzales R-C, Woods R-E, Eddins S-L (2014) Digital image processing using MATLAB, 2nd edn. McGraw Hill Education, USA (ISBN: 0982085400)
Hough Transform (1972). Hough transform. Retrieved from https://homepages.inf.ed.ac.uk/rbf/HIPR2/hough.htm
Ericksonn A, Kim K, Ryan S, Gerd B, Greg W (2019) Is it cold in here or is it just me? Analysis of augmented reality temperature visualization for computer-mediated thermoception. 2019 IEEE international symposium on mixed and augmented reality (ISMAR) - Beijing, China (2019.10.14–2019.10.18).
Mascareñas D, Harden T, Morales J, Boardman B, Sosebee E, Blackhart C, Cattaneo A, Krebs M, Tockstein J, Green A, Dasari S, Bleck B, Katko B, Moreu F, Maharjan D, Aguero M, Fernandez R, Trujillo J, Wysong A (2019) Augmented reality for enabling smart nuclear infrastructure. Front Built Environ. https://doi.org/10.3389/fbuil.2019.00082
Aguero M, Maharjan D, Rodriguez M-P, Mascarenas D-D, Moreu F (2020) Design and implementation of a connection between augmented reality and sensors. Robotics 9(1):3. https://doi.org/10.3390/robotics9010003
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Palma, V., Iovane, G., Hwang, S. et al. Innovative technologies for structural health monitoring of SFTs: proposal of combination of infrared thermography with mixed reality. J Civil Struct Health Monit 13, 1653–1681 (2023). https://doi.org/10.1007/s13349-023-00698-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13349-023-00698-1