The application of the thermal and infrared technology in different areas of research is considerably increasing. These applications involve nondestructive testing, medical analysis (computer aid diagnosis/detection—CAD), and arts and archeology, among many others. In the arts and archeology field, infrared technology provides significant contributions in terms of finding defects of possible impaired regions. This has been done through a wide range of different thermographic experiments and infrared methods. The proposed approach here focuses on application of some known factor analysis methods such as standard nonnegative matrix factorization (NMF) optimized by gradient-descent-based multiplicative rules (SNMF1) and standard NMF optimized by nonnegative least squares active-set algorithm (SNMF2) and eigen-decomposition approaches such as principal component analysis (PCA) in thermography, and candid covariance-free incremental principal component analysis in thermography to obtain the thermal features. On the one hand, these methods are usually applied as preprocessing before clustering for the purpose of segmentation of possible defects. On the other hand, a wavelet-based data fusion combines the data of each method with PCA to increase the accuracy of the algorithm. The quantitative assessment of these approaches indicates considerable segmentation along with the reasonable computational complexity. It shows the promising performance and demonstrated a confirmation for the outlined properties. In particular, a polychromatic wooden statue, a fresco, a painting on canvas, and a building were analyzed using the above-mentioned methods, and the accuracy of defect (or targeted) region segmentation up to 71.98%, 57.10%, 49.27%, and 68.53% was obtained, respectively.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
The computational load is processing time measured while the computer system performs (CPU time).
Scudieri F, Mercuri F, Volterri R. Solar loading thermography: time-lapsed thermographic survey and advanced thermographic signal processing for the inspection of civil engineering and cultural heritage structures. J Therm Anal Calorim. 2001;66(1):307.
Mercuri F, Zammit U, Orazi N, Paoloni S, Marinelli M, Scudieri F. Active infrared thermography applied to the investigation of art and historic artefacts. J Therm Anal Calorim. 2011;104(2):475.
Šulcová P, Šesták J, Menyhárd A, Liptay G. Some historical aspects of thermal analysis on the mid-European territory. J Therm Anal Calorim. 2015;120(1):239.
Davin T, Serio B, Guida G, Pina V. Spatial resolution optimization of a cooling-down thermal imaging method to reveal hidden academic frescoes. Int J Therm Sci. 2017;112:188.
Yao Y, Sfarra S, Ibarra-Castanedo C, You R, Maldague XP. The multi-dimensional ensemble empirical mode decomposition (MEEMD). J Therm Anal Calorim. 2017;128:1841–58.
Nolesini T, Frodella W, Bianchini S, Casagli N. Detecting slope and urban potential unstable areas by means of multi-platform remote sensing techniques: the Volterra (Italy) case study. Remote Sens. 2016;8(9):746.
Becherini F, Bernardi A, Di Tuccio MC, Vivarelli A, Pockelè L, De Grandi S, Fortuna S, Quendolo A. Microclimatic monitoring for the investigation of the different state of conservation of the stucco statues of the Longobard Temple in Cividale del Friuli (Udine, Italy). J Cult Herit. 2016;18:375.
Tang Z, Dai SB. Wavelet analysis applied to thermographic data for the detection of sub-superficial flaws in mosaics. Stud Conserv. 2016;61(sup1):37.
Cadelano G, Bison P, Bortolin A, Ferrarini G, Peron F, Girotto M, Volinia M. Monitoring of historical frescoes by timed infrared imaging analysis. Opto Electron Rev. 2015;23(1):102.
Sfarra S, Ibarra-Castanedo C, Paoletti D, Maldague X. Infrared vision inspection of cultural heritage objects from the city of L’Aquila, Italy and its surroundings. Mater Eval. 2013;71(5):561.
Yousefi B, Sfarra S, Maldague XP. Quantitative assessment in thermal image segmentation for artistic objects. In: Optics for arts, architecture and archaeology (SPIE, 2017). pp. 10,331–7.
Sfarra S, Ibarra-Castanedo C, Ridolfi S, Cerichelli G, Ambrosini D, Paoletti D, Maldague X. Holographic interferometry (HI), infrared vision and X-ray fluorescence (XRF) spectroscopy for the assessment of painted wooden statues: a new integrated approach. Appl Phys A Mater Sci Process. 2014;115(3):1041.
Ibarra-Castanedo C, Sfarra S, Paoletti D, Bendada A, Maldague XP. Nondestructive testing of externally reinforced structures for seismic retrofitting using flax fiber reinforced polymer (FFRP) composites. In: Proceedings of SPIE. vol. 8705, pp. 87,050U–1.
Thakur VK, Thakur MK, Kessler MR. Handbook of composites from renewable materials, biodegradable materials, vol. 5. Hoboken: Wiley; 2017.
Rockinger O. Image sequence fusion using a shift-invariant wavelet transform. In: Proceedings of international conference on image processing. IEEE; 1997. vol. 3, pp. 288–91.
Rajic N. Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures. Compos Struct. 2002;58(4):521.
Zhang H, Robitaille F, Grosse CU, Ibarra-Castanedo C, Martins JO, Sfarra S, Maldague XP. Optical excitation thermography for twill/plain weaves and stitched fabric dry carbon fibre preform inspection. Compos Part A Appl Sci Manuf. 2018;107:282–93.
Zhang H, Hassler U, Genest M, Fernandes H, Robitaille F, Ibarra-Castanedo C, Joncas S, Maldague X. Comparative study on submillimeter flaws in stitched T-joint carbon fiber reinforced polymer by infrared thermography, microcomputed tomography, ultrasonic c-scan and microscopic inspection. Opt Eng. 2015;54(10):104109.
Zhang H, Sfarra S, Sarasini F, Ibarra-Castanedo C, Perilli S, Fernandes H, Duan Y, Peeters J, Avdelidis NP, Maldague X. Optical and mechanical excitation thermography for impact response in basalt–carbon hybrid fiber-reinforced composite laminates. IEEE Trans Ind Inf. 2018;14(2):514–22.
Ibarra-Castanedo C, Piau JM, Guilbert S, Avdelidis NP, Genest M, Bendada A, Maldague XP. Comparative study of active thermography techniques for the nondestructive evaluation of honeycomb structures. Res Nondestruct Eval. 2009;20(1):1.
Lee DD, Seung HS. Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems; 2001. pp. 556–562.
Yousefi B, Fleuret J, Zhang H, Maldague XP, Watt R, Klein M. Automated assessment and tracking of human body thermal variations using unsupervised clustering. Appl Opt. 2016;55(34):D162.
Avdelidis N, Moropoulou A. Emissivity considerations in building thermography. Energy Build. 2003;35(7):663.
Yousefi B, Sfarra S, Ibarra-Castanedo C, Maldague XP. Thermal NDT applying candid covariance-free incremental principal component thermography (CCIPCT). In: Thermosense 2017 (SPIE, 2017). pp. 10,214–58.
Yousefi B, Sfarra S, Ibarra-Castanedo C, Maldague XP. Comparative analysis on thermal non-destructive testing imagery applying candid covariance-free incremental principal component thermography (CCIPCT). Infrared Phys Technol. INFPHY-2017-146. 2017.
Sfarra S, Marcucci E, Ambrosini D, Paoletti D. Infrared exploration of the architectural heritage: from passive infrared thermography to hybrid infrared thermography (HIRT) approach. Materiales de Construcción. 2016;66(323):094.
Mallat SG. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell. 1989;11(7):674.
Stollnitz EJ, DeRose AD, Salesin DH. Wavelets for computer graphics: a primer. 1. IEEE Comput Graph Appl. 1995;15(3):76.
Li H, Manjunath B, Mitra SK. Multi-sensor image fusion using the wavelet transform. In: Proceedings of IEEE international conference on image processing, 1994, ICIP-94. IEEE; 1994. vol. 1, pp. 51–55.
MacQueen J et al. Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley symposium on mathematical statistics and probability, Oakland, CA, USA; 1967. vol. 1, pp. 281–297.
Tillmann AM, Pfetsch ME. The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans Inf Theory. 2014;60(2):1248.
The authors would thank anonymous reviewers and Journal of Thermal Analysis and Calorimetry’s editor for their constructive comments. We would like to thank Geltrude Di Matteo (director) and Alessandro Verrocchia (chief restorer), of the Musè (Il Museo delle Paludi di Celano) Italy, for their kind cooperation in this work. The authors also want to acknowledge the support of the Multipolar Infrared Vision Canada Research Chair (MIVIM), tier 1. A special thank to Mr. Giovanni Pasqualoni, the University of L’Aquila (L’Aquila, Italy), for the technical support during the thermographic acquisitions.
About this article
Cite this article
Yousefi, B., Sfarra, S., Ibarra-Castanedo, C. et al. Thermography data fusion and nonnegative matrix factorization for the evaluation of cultural heritage objects and buildings. J Therm Anal Calorim 136, 943–955 (2019). https://doi.org/10.1007/s10973-018-7644-6
- Thermal image segmentation
- Negative matrix factorization analysis
- Gradient-descent-based multiplicative rules
- Nonnegative least squares (NNLS) active-set algorithm
- Wavelet data fusion