1 Introduction

Imaging spectroscopy techniques and instrumentation have been developed in the last two decades, in both the visible and infrared spectral range, to study cultural heritage material composition and degradation mechanisms. These techniques are becoming increasingly popular because they are non-contact and non-invasive. For diffuse reflectance hyperspectral imaging, the visible (VIS, 400–750 nm) to near-infrared (NIR, 750 − 2500 nm), and specifically the short-wave infrared (SWIR, 1000–2500 nm) spectral range, is probably the most exploited at present, and reflectance applications have been found effective for pigment mapping since 2010 [1]. Although remote sensing solutions have been recently published [2], the most common configuration for the macro-scanning of cultural heritage surfaces still requires that the object is located at a short distance from the instrument [3]. Applications include the study of plastic objects and inks [4]. The most recent prototype instruments have enabled the analysis of negative and positive films [5], and tapestries [6]. For easel paintings and illuminated manuscripts, instruments have been tested for the identification of artists’ materials and technique [7] and the documentation of degradation features [8]. To study artists’ materials, the potentialities of the VIS and NIR range have also been combined [1, 9], sometimes with additional Fourier transform (FT-HIS or HS FTS) for the hyperspectral imaging of pigments [10].

Recently, the potential of the mid-infrared (MIR, 2500–15,000 nm) range for macro-imaging spectroscopy has been investigated on painted and metal artworks [11, 12]. Developing new solutions in the MIR is of particular interest in cultural heritage, as this spectral region has high specificity in the identification of chemical compounds commonly found in artworks [13].

The potential of the NIR region has been previously explored for the identification of resins and binders [14], although signals in this range are usually weak overtone and combination bands. The MIR region is still considered the most informative to study cultural heritage materials. However, bands in the 3000–2800 cm−1 are only partially informative about organic or (a few) inorganic components, and they need to be combined with signals in the fingerprint region (1800–600 cm−1) for correct identification. As the bandwidth of this spectral region is still limited by the currently available IR sources and detectors, this significant constraint could be mitigated by the implementation of post-processing strategies, e.g. the statistical processing of the data cube for mapping purposes, which could offer a means to discriminate different constituent materials and visualize their distribution using false colour imaging.

The three-dimensional data cube from an imaging spectrometer consists of two spatial dimensions and one spectral dimension (one spectrum/interferogram per pixel). Based on the way this ‘image cube’ is collected, a classification has been proposed for spectral imaging methods [15]. It has been stated that most of the systems used in the technical examination of cultural heritage employ line scanning sensors [5], which allow the simultaneous acquisition of a 2D dataset, which consists of one spatial dimension and one spectral dimension. The 3D data cube is then obtained by scanning the second spatial direction, which is orthogonal to the first one.

Here we introduce a novel portable system originally conceived for transmission measurements [16] and then adapted to cultural heritage applications in reflection mode [17], within the frame of the PISTACHIO project (Photonic Imaging Strategies for Technical Art History and Conservation). Innovative features of the system are the way light, and data are collected. Most of the sensors used in cultural heritage studies cover the VIS-SWIR spectral range, employing CCD/CMOS, InGaAs, MCT, InSb detectors, some of which require cooling. The PISTACHIO imager is equipped with an internal interferometer and a commercially available microbolometer camera, which operates within the MIR using an uncooled vanadium oxide focal plane array (VOX FPA) detector. This configuration means that, to be scanned, the object is moved while the camera is stationary. This reduces the overall cost of the equipment by a factor of 3 (approximately GBP 25,000) when compared to benchtop FTIR imaging systems. Unlike the majority of sensors presently used in cultural heritage investigations, the PISTACHIO imager can be classified as a ‘snapshot’ system, the (2D) spatial and spectral information being collected with one exposure. This feature reduces the time of acquisition for one tile to 4.5 min at a 4 cm−1 spectral resolution. Band-pass non-uniform sampling has been applied in this system to achieve a spatio-spectral acquisition rate (SSAR) considerably higher [16] than in any recently published reflection FTS images and in a slightly wider spectral range than the most recent instruments [11, 18]. The analysis in the full MIR range is usually necessary for the study of artists’ materials, because in this spectral range are the main spectral signals of both inorganic (pigments and fillers) or organic (dyes, binders and varnishes) compounds [11]. As in our system the available bandwidth is cut down to 1400–700 cm−1, we aimed at implementing post-processing clustering strategies in the MATLAB programming environment, to overcome this system limitation and contribute to the mapping of materials used in cultural heritage.

The use of a commercially available camera, which comes with a defined field of view, limits the spatial sampling to an area of 12 mm × 12 mm. For this reason, we further aimed at the application of post-processing merging strategies, to apply and test the clustering on a composite data cube, i.e. on a larger area of the painting.

2 Materials and methods

2.1 The hyperspectral imaging system

The hyperspectral imaging system used in this study is an implementation of the hyperspectral Fourier transform spectrometer (HS FTS) illustrated in [17]. It was originally configured in transmission mode. The instrument has since been adapted for reflection measurements (Fig. 1) to perform analysis of painted surfaces. It is intended as a portable device due to its dimensions (L90 × W60 × H37 cm).

Fig. 1
figure 1

PISTACHIO imager: a schematics; b top view of optics and microbolometer camera

Full size image

The light source is a 1200 K Thorlabs silicon nitride globar lamp, which emits IR radiation (1400–700 cm−1). The spectrum of the incoherent light source is modulated by the Michelson interferometer. The beam is then incident on a focusing lens and a long-pass optical filter (to mitigate heating of the painted surface) before reflecting on the painting.

The field of view of a single acquisition is limited to 12 mm × 12 mm due to the physical constraints of the instrument—mainly resulting from using a light source that is incoherent and off-the-shelf optical components no larger than 50.8 mm. As such, it is not possible to image a full painting in a single acquisition, so the painting is moved relative to the instrument for analysis of a larger composite image. To this end, the painting is held on a two-axis translation stage. The motorized stage (ASI Imaging MS-2000) is software controlled with 0.01 µm resolution, with a maximum x–y movement of 72 mm in the x direction and 86 mm in the y direction. The z-axis position of the painting is manually set by a linear stage to allow the positioning of the focal point of the camera at the surface of the painting prior to the analysis, usually in the centre of the area to be scanned.

The IR radiation reflected from the chosen region is then collected by the long-wave IR (LWIR) microbolometer camera (FLIR A65sc), which has a spectral range of 1400–700 cm−1, an effective field of view of 512 × 640 pixels, a 30 Hz frame rate and 17 μm pixel pitch. The LWIR camera is positioned at a distance of approximately 85 mm from the painting, with the angle between incident and reflected light of approximately 20° (Fig. 1a).

For a single acquisition, LWIR camera frames are recorded for a set number of optical path difference (OPD) positions which are obtained by moving one of the FTS mirrors on a motorized translation stage. The stage can be moved to sample all camera pixel interferograms by regular subsampling, at regular displacement intervals, or non-uniform subsampling, with positions determined by a probability distribution for possible reduction in acquisition time, as discussed by Charsley and co-authors [16]. The acquired stack of camera frames sampling each pixel’s interferogram is known as an interferometric data cube. Fourier transform analysis of individual pixel interferograms provides spectral profiles of light reflected from the sample for each pixel in the image—the hyperspectral (HS) data cube. The HS data cube spectral resolution (≥ 0.5 cm−1) is limited by the total displacement of the mirror translation stage (≤ 10 mm).

The data presented in this paper were obtained via regular subsampling of interferograms every 1.2 µm through an optical path difference of 625 µm to give a spectral resolution of 16 cm−1. The acquisition took 1.5 min per interferometric data cube which is constrained by the 30 Hz LWIR camera frame rate. The 12 mm × 12 mm field of view for a single acquisition corresponds to a 100 × 100-pixel image on the LWIR camera which is cropped from the full 512 × 640-pixel image.

A background reference is required to isolate the reflection features from the hyperspectral image by computing the ratio of the sample spectra with the reference spectrum. An aluminium-coated glass slide is placed at the image plane and a single acquisition is run and processed to provide this background spectrum. In this system, the focusing lens placed between the interferometer and the background reference (or sample) increases the signal-to-noise (SNR) of the captured microbolometer camera images. However, from an imaging perspective, the lens images the light source filament and filament collimator onto the camera plane. The resultant intensity image from the aluminium-coated glass slide is non-uniform as it has a concentric-ring-like high-intensity feature around the centre of the incoherent light due to imaging the filament collimator. This has the effect of varying the SNR of the reconstructed spectra across the hyperspectral image.

During the experiments, videos are taken with a thermal camera (Seek Thermal Compact iOS) to monitor the temperature at the surface of the object.

2.2 The graphical user interface (GUI)

Analysis of the hyperspectral image may make it possible to distinguish separately mixed materials, either through slice analysis or by computational analysis of the entire hyperspectral image from the use of a clustering algorithm, in a MATLAB-built GUI. Slice analysis shows single image stacks of the HS data cube at a chosen single wavenumber. The clustering algorithm implemented here and described by Charsley and co-authors [16] fits a Gaussian mixture model (GMM) to pixels with spatially correlated labels [19]. Several parameters are user-defined in the clustering algorithm: the number of clusters, the number of iterations, the number of principle component (PC) dimensions to run the algorithm with to mitigate computational cost, robustness as to whether to assign clusters to outliers and the choice of 4 or 8 nearest neighbours for the spatial labels.

The results can be illustrated by a clustering map and by a clustered 2D or 3D principal component analysis (PCA) plot. The clustering map displays the spatial distribution of the different cluster pixels across the image by allocating a distinct false colour to each cluster [12]. The cluster assignments can possibly correspond to distinct pigment/pigment mixtures. The cluster assignment shown in a PCA plot is used to assess if spectral signatures tend to aggregate [20]. Finally, the average spectrum and standard deviation associated with each cluster can be studied for chemical interpretation.

2.3 The case study

The oil painting ‘Uplands in Lorne’ (Fig. 2) is a landscape painting of the Lorne area of the highlands (Argyll and Bute), Scotland.

Fig. 2
figure 2

Painting ‘Uplands in Lorne’ by D.Y. Cameron

Full size image

The Glasgow Boy painter David Young Cameron (1865–1945) painted it on boxwood circa 1900. It is in collection of the Hunterian, University of Glasgow, UK (Acc. No.: GLAHA43427). The painting size is 141 mm × 182 mm. It is currently being examined at the Kelvin Centre for Conservation and Cultural Heritage Research, Glasgow University, as part of ongoing research into Cameron’s materials and methods, which included portable XRF measurements to help identify the pigments used by the artist. ‘Uplands in Lorne’ is an ideal case study for testing the HS FTS as it can be easily held on the stage because of its small dimensions. Additionally, it contains compositional details that test the performance of the clustering strategy, i.e. neighbouring areas that imply the use of pigments with different spectral signature.

2.4 The reconstruction

A reconstruction panel was made to validate the HS FTS on known pigments or pigment mixtures. A boxwood panel was painted with pigments in oil. Commercially available traditional oil paints were chosen to reproduce D.Y. Cameron’s palette. They are: cobalt (CoB in Fig. 3) and cerulean blue (CeB), vermillion (V), yellow ochre (YO) and ivory black (IB). Different white pigments were also included, specifically titanium (TW), lead (LW) and zinc (ZnW) white. Because the painting ‘Uplands in Lorne’ was found not to have a ground layer, the first row of squares in the panel are pure pigment on a rabbit skin glue size. The other rows represent mixtures of the pure pigments with a white, where the white pigment is in the first square of each row. To increase the level of complexity of the reconstruction, the last row has a mixture of lead and zinc white added to each pigment.

Fig. 3
figure 3

Reconstruction panel used to reproduce D.Y. Cameron’s palette and validate the HS FTS

Full size image

Titanium white is included as reference in the reconstruction, but it is not mixed with any other pigment, as this type of white was not found in Cameron’s palette. We included titanium white as this is a frequently used pigment since the 1920s as substitute for lead white. The test and validation experiments were also used as an opportunity to build a spectral library (now published as open-access database in the University of Glasgow repository) for future research.

2.5 Analytical conditions

A representative area on the top right side of the painting ‘Uplands in Lorne’ was scanned. The following parameters were used: 1 scan, 16 cm−1, 600 nm stage step size. A total of 121 data cubes (tiles) were collected and processed. Frequency calibration for each pixel was performed, and each interferogram was high-pass-filtered and apodized with the Norton–Beer strong lineshape. Finally, a composite HS data cube merging all 121 tiles was created from registration of adjacent tiles in the x and y directions, respectively, by performing a two-dimensional cross-correlation. Single and composite data cubes were processed for clustering and image stack (or slice) analysis.

Each square of the reconstruction panel was scanned. The following parameters were used: 5 averaged scans, 16 cm−1, 600 nm step size. Spectra from single pixels were analysed individually or compared after background subtraction. The result of this processing step is that spectra from the imager represent the total radiance collected by the imaging camera, subtracted by the thermal radiance reflected from the background, shown as intensity as a function of the wavenumber. The shape of intensity signals is usually opposite to that of pseudo-absorbance signals from a conventional benchtop µ-FTIR spectrometer, used in this study to cross-validate hyperspectral imaging data.

The benchtop µ-FTIR spectrometer is a Thermo Scientific Nicolet iN™ 10 MX IR imaging microscope, equipped with MCT/A detector. Spectra were collected in external reflection mode, at 4 cm−1 resolution in the 4000–700 cm−1 spectral range. The reference spectra are the average of 64 individual spectra, computed using Norton–Beer medium/strong apodization.

3 Results and discussion

3.1 Validation of the HS FTS

One of the main concerns on surface analysis of real paintings is to minimize the exposure to the IR radiation, as it might cause excessive heating leading to damage. The first validation step monitored the temperature variation on the reconstruction panel, to avoid excessive heating. It was found that the long-pass optical filter needed to be included in the HS FTS to keep the heating of the painting surface at acceptable levels. With the filter present, the exposure to IR irradiation, which lasted for about 9 min during a prolonged double-run at 4 and 8 cm−1, caused a maximum increase in temperature of 2 °C (depending on the pigment, see Table 1) with subsequent cooling down to room temperature in approximately 10 s.

Table 1 Temperature variation at the surface of the reconstruction panel due to exposure to IR irradiation
Full size table

This result shows excellent improvement with respect to the most recent prototypes in the MIR range [11], which recorded a 4–5 °C temperature increase for the fastest application in CH studies (9 × 9 cm2, 32 scans in 80 s). It also proves that the temperature can be controlled with no need of cooling systems, as proposed by Gabrieli et al. [18].

The instrument was also tested for measurement repeatability, empirically. After the subtraction of a background data cube from the aluminium-coated glass slide (see Sect. 2.1), several spectra collected from the same area of the reconstruction were compared. They showed that similar signals were given when spectra from different pixels of the same painted area were compared—within the same data cube and with data cubes taken at different times. Figure 4 illustrates the HS FTS repeatability for ivory black, where the identifiable compound is hydroxyapatite, Ca5(OH)(PO4)3. The main vibrational modes of the phosphate group [21] can be recognized in three different spectra from the same data cube (black—solid, dashed and dotted—in Fig. 4a): the most intense ν3 (PO4)3− at 1069 with the minor shoulder at 1086 cm−1, and the ν1 (PO4)3− at 874 and 964 cm−1, the latter being slightly visible. These three spectra have good agreement with data from the benchtop µ-FTIR (grey spectrum).

Fig. 4
figure 4

Comparison of spectra from different areas of the reconstruction, grey—collected by µ-FTIR, black—collected by HS FTS: a ivory black, the 3 HS FTS spectra (solid, dashed and dotted) come from 3 different pixels of the same data cube; b cerulean blue, dotted and continuous black line correspond to spectra extracted from the same area after different measurements

Full size image

Agreement in spectral features was also found within the same area after a second acquisition, meaning that the instrument has good repeatability, and it is still possible to identify pigments that produce a response in the MIR. For example, spectral features from a sulphate-based compound were recorded either in the first or second run (Fig. 4b), probably added as filler in the commercial pigment. It shows the characteristic asymmetric (ν3) and symmetric (ν1) stretching modes [22] at 1193, 1128, 1085 and 985 cm−1, in good agreement with the µ-FTIR results. The identification was also possible because spectral distortions, usually occurring in reflection mode for inorganic and organic compounds [23], were consistent between the imager and the benchtop instrument.

The clustering was tested on the full dataset from the reconstruction, where each region of interest (ROI) included 4 different squares of the panel. The clustering was performed on each tile after the subtraction of the background collected at the beginning of the experiment as described in Sect. 2.1. Figure 5 illustrates the clustering efficiency in the discrimination of several pigments for the ROI that includes two pure pigments, a mixture with zinc white and the size (Fig. 5a). For this clustering task, the algorithm was set to provide 4 clusters with spectrally different features, plus a fifth cluster associated with low-signal spectra. This makes the clustering more robust as low-signal spectra tend to make the identification of classes more challenging. The algorithm ran for 700 iterations, with a PC dimension reduced to 4, and using a 4-neighbour structure. The blue cluster in the PCA map (Fig. 5b) corresponds to the rabbit skin glue size, the red is cobalt blue, and the green is likely to represent zinc white, which is also present in the bottom right square, but in mixture with cobalt blue. The black area is interpreted as a low-signal cluster.

Fig. 5
figure 5

a Raw data cube collected from the area of the reconstruction with the size (top left), cobalt blue (top right), zinc white (bottom left) and the mixture of cobalt blue and zinc white (bottom right); b PC map and c score plot of the ROI

Full size image

In this example, it is interesting to note the separation of the sized area. Further experiments might explore the potentiality of the HS FTS system to discriminate different organic media in the same artwork. The tiles containing cobalt and cobalt with zinc white are not separated, highlighting the difficulty in the distinction of pigment mixtures, especially if they show weak features in the investigated spectral range. Alternative clustering strategies, like uniform manifold approximation and projection or self-organizing maps, might be tested in the future for a better discrimination of pigment mixtures.

3.2 Testing the HS FTS on the case study

A 5.4 × 5.4 cm2 area of the painting ‘Uplands in Lorne’ was scanned by the HS FTS system in 3 h 40 min. The acquisition time was longer than required for a step equal to the ROI (1.2 cm) because scan areas were partially overlapped and averaged by setting a step smaller than the ROI (0.4 cm). The full acquisition took less than half the time reported for one of the latest imaging applications of a single point MIR-FTIR spectrometer on a nineteenth-century icon (paint on wood), measuring 10.2 × 17.2 cm2, in 30 h [13].

Data cubes were first analysed individually. Then a composite data cube was built, mosaicking together the 121 tiles from the scanned area. In this case, ratioing of the background spectrum was carried out using an average spectrum of a normalized HS data cube from the aluminium-coated glass slide, rather than pixel-wise ratioing. This is to prevent strong artefacts in the background image structure altering the ratioed painting image structure due to different gain settings used for the LWIR camera between background and signal images.

3.2.1 Slice analysis

In slice analysis, monochromatic spectral images can be extracted from a data cube to investigate the painting at specific wavenumbers. This type of analysis was carried out either on single tiles of the scanned area or on the composite data cube, where it highlighted the distribution of the lighter brush strokes.

Figure 6 shows the slice taken at 1041 cm−1 from several tiles of the scanned area (a–b) and for the composite data cube (c). The bright regions of the slice possibly show the distribution of lead white, as a ν1-associated band, produced by the symmetric stretching vibration of the CO32−, is usually observed at this wavenumber for cerussite, PbCO3 [24], the main component in lead white. Indeed, XRF (not shown) confirmed that a lead-based pigment has been used by Cameron to paint the cloud and the brightest parts of the mountains.

Fig. 6
figure 6

Slices taken at 1040 cm−1 from several tiles of the scanned area, superimposed to a visible-light picture of the painting ‘Uplands in Lorne’: a, b single slice; c composite data cube

Full size image

3.2.2 Cluster analysis

The MATLAB environment was used to test the PCA and identify meaningful clusters that could represent the distribution of different pigments or mixtures of them in the case study. When the PCA reduced the dimensionality of the dataset to 5 in 600 iterations with 8 neighbouring pixels, 4 clusters (plus 1 for robustness) were found to produce a good representation of the top left corner of the scanned area (Fig. 7).

Fig. 7
figure 7

Principal component map (a) and score plot (b) of the top left tile of the scanned area, superimposed to a visible-light picture of the painting ‘Uplands in Lorne’: five-dimensional PC space, 600 iterations, 8 neighbouring pixels, 4 clusters (1–4 in a) plus one for robustness

Full size image

Based on the distribution of these clusters on the tile and with the aid of XRF data, these clusters can be interpreted as follows: cluster 1 (blue in Fig. 7a) represents the distribution of a dark blue mixture, possibly corresponding to cobalt blue and some iron oxides (earth pigments); cluster 2 (red) shows the distribution of lead white, used for the cloud; cluster 3 (green) represents the light blue pigment used for the sky, possibly in mixture with zinc white, while cluster 4 (black) is likely to come from a different compound used to tone the cloud. Even though each cluster could be uniquely assigned to specific details of the composition, the subtraction of the non-uniform background (discussed in Sect. 2.1) over the whole scanned area reduced the ability in identifying the spectral features typical of each cluster. The resultant reflection spectral data cube also has non-uniform SNR across the image, which non-uniformly scales noise contributions from each pixel when normalizing for input into the clustering algorithm. Increasing the camera sensitivity or increasing the optical power of the light source would eliminate the requirement of a focusing lens and therefore remove the imaging of the filament collimator. For this reason, the composition of the pigment/pigment mixture corresponding to the single cluster could not be confirmed, nor could the potential spectral distortion typical of reflection measurements be determined within the bandwidth of analysis. Hence, the interpretation of these clusters was mainly based on visual observation and XRF results.

Similarly, in the composite data cube (ten-dimensional PC space, 800 iterations, 8 neighbouring pixels, 4 clusters + 1 for robustness) the clustering (Fig. 8) led to a good representation of the composition, meaning that it is sensitive to pigment distribution, even with some loss of detail compared to the single tile.

Fig. 8
figure 8

HS FTS results for the full scanned area of ‘Uplands in Lorne’: a raw composite data cube, b PC map with spectral signature of each cluster (c–f), average spectrum in solid line, standard deviation in dotted lines and (g) corresponding VIS-light photograph

Full size image

The plot of each cluster spectrum, calculated as deviation from the mean spectrum of the four cluster spectra (Fig. 8c–f) was analysed for chemical interpretation. While the black cluster (Fig. 8f) is interpreted as a low-signal cluster, the rest of the plots are characterized by different spectral signatures: the blue (Fig. 8d) and red (Fig. 8e) show a broad band in the range 1100–800 cm−1, which might be attributed to the strong Si–O antisymmetric stretching modes of silicate minerals [25]. Silicates are typically found in earth pigments, used both for the blue (with cerulean blue) and brown shades of the mountains. Specifically, the broad band centred at about 1030 cm−1 might come from clay minerals like montmorillonite [26], which are naturally occurring impurities in ochres. The latter contribute to the lighter hue of the ochre used for the brightest areas of the mountain and cloud (blue cluster). This band is less intense in the red cluster, possibly because of a minor contribution of clay minerals in the dark brown paint. The grey (Fig. 8c) has features of lead white, i.e. the band at about 1040 cm−1, already used for interpretation in slice analysis.

4 Conclusions

We have demonstrated that fast hyperspectral imaging on pigments in external reflection mode is possible in the MIR at reasonable acquisition times and cost. Results on the paint reconstruction show that the HS FTS offers good repeatability for pigments that have an infrared spectral signature and that the heating at the surface of the painting can be kept within a safe range for the entire duration of the measurement. Spectral features from single pigments are also compatible with data obtained from a conventional micro-FTIR system.

For the case study painting ‘Uplands in Lorne’, the HS FTS led to the acquisition of multiple data cubes, which were processed both for slice analysis and clustering purposes. The first type of data processing is suggested for the study of the distribution of specific pigments with good response in the MIR. The clustering was found able to overcome the limitations of spectral interpretation in such a narrow bandwidth. On the case study painting the clustering showed that the PISTACHIO imager is capable of discrimination between details of the composition, for example the cloud, mainly painted with lead white, and the sky, painted in cerulean blue and a different white pigment (zinc white). Future data processing strategies would include t-distributed stochastic neighbour embedding (t-SNE) or uniform manifold approximation and projection (UMAP), as potentially the most effective mapping of pigment and binders on polychrome surfaces, and for a firm identification. Further device adaptations could include the complete automation of multiple data cube collection and the application of compressive strategies.