1 Introduction

Information provided by structural health monitoring (SHM) systems in building management systems (BMS) can reduce losses due to natural, accidental, and man-made disasters on buildings and increase the social resilience because they improve the decision support system (DSS) that allows suitable decisions to be made regarding the maintenance of the building, as well as rapid reaction to dangerous situations [1]. Indeed, following a disaster, rapid and reliable assessment of damage is necessary for the management and coordination of interventions to secure and/or evacuate buildings of potentially injured people. Permanent monitoring of structural response and damage-related parameters, integrated with a decision-making and alerting system, can be highly useful in such circumstances. The SHM system allows the near-real-time identification of the most badly damaged structures and the definition of a hierarchy to be followed in the management of emergency interventions. Moreover, in the case of earthquakes, for instance, damage can likely accumulate because of the effects of multiple events and/or because the building is unrepaired between sequential events [2]. Therefore, the precise type of failure is extremely hard to recognize, since damage scenarios can easily change over time due to seismic swarm and cumulative effects of damage, as may have happened, for example, at several sites in central Italy during the 2016–2017 seismic sequence [3].

In situ observations after seismic events have shown that non-structural component failures are a source of many earthquake-related losses [4], and can put human lives at risk. For instance, infill walls and masonry partitions, commonly used as elements to enclose or separate internal spaces in reinforced concrete (RC) framed buildings, are typically designed to guarantee sound and thermal insulation. From a structural point of view, they are considered non-structural components due to their low bearing capacity, even if they are able to interact with the structural frame, influencing the dynamic behavior and seismic response of a building [5, 6]. Nevertheless, the assessment of their vulnerability, especially out-of-plane (OOP), is currently an important topic in the field of seismic engineering because their failure can cause serious economic damage (e.g., limiting immediate occupancy after the earthquake event) and can aggravate the risk to human life due to falling debris. Moreover, it has been proven that OOP collapse depends on the type of wall and the efficiency of its connection to the RC frame [7].

Given all this, monitoring seems to be a highly effective tool for controlling the state of existing buildings, considering that people’s safety must be protected, on the one hand, though on the other the costs of intervention are not compatible with widespread and generalized immediate intervention. Moreover, it is clear that in new constructions monitoring systems able to guarantee effective management of the built environment in the future are extremely important.

Recent literature has highlighted that dynamic in situ tests based mainly on operational modal analysis (OMA) techniques are excellent tools with which to derive the modal parameters under service conditions [8,9,10] and calibrate reliable numerical models while taking into account the actual boundary conditions [11,12,13]. Since damage produces variations in the mechanical and physical properties of the structure, which can change its natural frequencies and mode shapes, analyses of the dynamic properties of the structure can be used for damage detection. On the other hand, artificial intelligence (AI) methodology based on semi-supervised methods with a data-driven approach has been shown to be reliable for detecting damage to structural members [14].

Since the aforementioned concepts could also be applied to non-structural components, this paper shows the outcomes of output-only vibration experiments carried out on an infill partition, as part of the PON research project CADS—“Creating a Safe Home Environment”.

Damage scenarios were studied with experimental data recorded on the actual damaged wall and then simulated through a numerical model. The possibility of creating damage scenarios does not come about often in real structures due to the costs, which are not negligible. The authors exploited the opportunity provided by restoration work that was being done in the building, and obtained permission from the owner to damage the wall. The model was preliminarily updated with data taken from the healthy component to evaluate its ability to introduce various damage conditions.

It should be noted that using a set of accelerometers directly installed on the infill is not a standard practice to assess the health state of a partition wall. The proposed methodology is designed to use the damage of partitions to evaluate the consequences of seismic events that do not cause significant damage to structures but instead increase a building’s vulnerability and introduce risks to the people’s safety through damage to non-structural components (infill walls or partitions), which in many post-earthquake data collections are related to the highest percentage of losses. Furthermore, the relationship of these components with the structure can reduce seismic capacity in aftershocks, and in damage detection it may, therefore, be more useful to have a warning message in a domotic integrated system, rather than waiting for the outcome of an in situ inspection. Finally, the damage to partitions and infill walls can also give a measure of the drifts that have occurred in the structure. The aim of this paper is to show that for real applications, if an innovative algorithm is used instead of traditional techniques, it is possible to use a single sensor on the individual panel when the objective of the monitoring system is limited to rapid damage identification and not location. Clearly, an optimization of the monitoring system on the building requires the choice of the most critical and/or significance elements to be monitored; that, however, falls outside the scope of this paper.

2 Research significance

The rapid assessment of the health status of “non-structural components”, i.e., elements that do not contribute to the bearing capacity of the structure but are characterized instead by their own structural behavior, is a key issue in improving the resilience of buildings after a catastrophic event. The issue is particularly significant when the main structure maintains its bearing capacity but loses its operationality and functionality due to severe damage to partitions, plants (e.g., firefighting or air conditioning systems), windows and so on. Furthermore, it is well known that most of a building’s cost is related to its non-structural elements. Therefore, the prioritization of their repair allows one to optimize the costs and improve the resilience of the building’s functionality, while also protecting the safety of the occupants, who may be threatened by the failure of these elements.

Given all this, the focus of this paper is the design of a continuous dynamic monitoring system for non-structural elements that is able to detect damage through a data-driven approach. Without losing generality, the procedure is applied to the case study of an internal partition.

The study has been developed by comparing the results of damage detection procedures based on the interpretation of acceleration, by means of: (i) established techniques such as operational modal analyses (OMA); and (ii) innovative machine learning techniques such as artificial neural networks (ANN). The techniques are applied to the acceleration recorded on undamaged and damaged elements.

Finally, also worthy of note is the comparison between the performance of convolutional and fully connected neural networks used in the semi-supervised variational autoencoders (VAEs).

3 Dynamic identification and model updating

The masonry internal partition wall selected for the in situ dynamic tests is located on the first floor of an RC-framed building in a seismic area of the south of Italy (Fig. 1). The wall is b = 500 cm × h = 350 cm, with a total thickness ttot = 37 cm, obtained by coupling two hollow clay brick independent walls with t = 10 cm, with an internal gap of 14 cm, and a 1.5 cm thick plaster layer on both the external sides (see Fig. 2). The wall is oriented in the transverse direction of the building, as shown in the floor plan (red wall in Fig. 2a). In addition, the masonry partition is connected by means of a layer of mortar to the floor, ceiling, and lateral reinforced concrete columns.

Fig. 1
figure 1

Building hosting the masonry partition

Full size image
Fig. 2
figure 2

Wall cross section

Full size image

3.1 Ambient vibration test

The first ambient vibration test was carried out to obtain the natural frequencies and the mode shapes of the masonry partition wall under operating conditions. As stated above, the partition is constituted by two leaves that are not connected, and thus independent. Therefore, the investigations were performed exclusively on one leaf. The excitation was environmental ambient noise; though not measured, it was assumed to be broadband random. The best sensor location was planned according to the results of a modal analysis carried out on a preliminary FE model, as described in the following section. A regular and dense measurement grid of 21 points was defined to obtain accurate mode shapes through out-of-plane accelerometers, numbered as shown in Fig. 3.

Fig. 3
figure 3

Layout of the sensors placement

Full size image

The available eleven accelerometers were enough to cover the entire extension of the member with two setups. Figure 3 shows one red dot representing the accelerometer used as reference to merge the data obtained by the two acquisitions and ten blue/green dots representing the location of the accelerometers for the setups S1/S2, respectively. The accelerometers were placed on the wall through mechanical connection (Fig. 4).

Fig. 4
figure 4

Pictures of the in situ dynamic test: a setup S1; b setup S2

Full size image

The equipment was composed of uniaxial Integrated Circuit Piezoelectric (ICP) accelerometers with a bandwidth ranging from 0.04 to 400 Hz and a sensitivity of 10 V/g. The noise was less than 10−6 g. These accelerometers were connected via cables to a twelve-channel data acquisition system with a 24-bit ADC, provided with anti-alias filters. The parameters set for the dynamic tests were a sampling frequency of 200 Hz and 3600 s time duration for each test. These assumptions assure that frequencies from 1 to 80 Hz would be properly measured.

3.2 Data processing for the dynamic identification

The data processing to extract the modal properties was carried out with the software Artemis Modal Pro [15], working both in frequency and time domains. In the frequency domain, the power spectral density matrices were obtained using the frequency domain decomposition (FDD) technique [16], which decomposes the spectral density matrix of the system response into a set of single degree of freedom (SDOF) systems using the singular value decomposition (SVD). The singular values estimate the spectral density of the SDOF systems, and the singular vectors estimate the mode shapes. In the time domain, the SSI-UPC merged test setups algorithm, available in the software, and based on the data-driven stochastic subspace identification (SSI) method [17], was used. In this procedure, the data are merged and normalized prior to the identification step. Table 1 lists frequencies and damping factors of the four modes identified through these methods (FDD and SSI). The methods were in excellent agreement, since they were characterized by a frequency difference of ε = fexp,FDD / fexp,SSI lower than 1% and modal assurance criteria (MAC) values greater than 0.9. Figure 5 shows the modal displacements for the identified mode shapes.

Table 1 Identified frequencies and damping factors
Full size table
Fig. 5
figure 5

Identified modal shapes

Full size image

It is worth underlining that the identification of the first natural frequency from the frequency-domain plot of singular values of spectral densities vs frequency was not unique, in this case. Figure 6a shows the presence of three peaks (18.16 Hz; 19.04 Hz; 20.22 Hz), and the analysis of the AutoMAC, depicted in Fig. 6b, did not provide information useful for defining which of these was right. Therefore, the frequency to which the low complexity value was associated was selected (19.04 Hz) and considered in the following for the model updating.

Fig. 6
figure 6

Identification of the first natural frequency: a singular values of spectral densities; b AutoMAC for the first three identified peaks

Full size image

3.3 Numerical model

A 2D linear elastic model of one leaf of the wall was developed using the software SAP2000 [18]. Homogeneous and orthotropic material was adopted, since the masonry wall is constituted of horizontally hollowed clay units, with mass per unit volume ρ = 700 kg/m3. The hollow clay brick wall was modeled using 10 cm × 10 cm shell elements with the thickness 12.5 cm (i.e., the brick and the plaster layer). This mesh made it possible also to have joints at the same position as the sensors used during the experimental test. The elastic modulus E1 = 900 MPa and E2 = 1800 MPa of the hollowed clay brick infill in the two directions parallel and orthogonal to the holes, respectively, were assumed, according to the technical literature for masonry infill, such as case study [19]. Moreover, the model of the wall was restrained along the perimeter by linear springs that had to be calibrated using the experimental results. At this initial stage, very high values of stiffness were assigned to the springs to reproduce a fully restrained condition along the four sides.

Generally, the frequencies and mode shapes obtained from in situ testing do not coincide with those of the numerical model, and therefore model updating was required to modify the numerical model by using experimental measurements [20]. A modal analysis was then performed with this model to obtain the numerical modal properties. The correlation between the dynamic experimental behavior and the numerical behavior was developed using the frequency difference ε, in percentage, along with the MAC, as suggested in [21]. For each i-th mode, Fig. 7 lists the values of the experimental and numerical frequencies, fi, and MACi values.

Fig. 7
figure 7

Frequencies (in Hz) provided by numerical model and experimental identification, and MAC values before the model updating

Full size image

The numerical frequencies are lower than the experimental frequencies for the first two modes and are higher for the other modes, with differences ε = (fnum—fexp) / fexp up to 13%. The MAC values show good correlation for all the modes except the last (with a value of 0.76). Moreover, the third numerical mode was not detected experimentally. Hence, a calibration of the model was necessary to improve the correlation with the experimental results. By updating the model, the differences between the numerical and experimental modal properties could be minimized; specifically, by changing some uncertainty parameters, such as material properties and boundary conditions.

3.4 Modal tuning

To update the model, the Young’s modulus was manually adjusted in two directions (E1 parallel and E2 orthogonal to the holes), while the stiffness of the translational springs was adjusted along the vertical y-axis (Ky), and the rotational springs were adjusted around the horizontal in-plane x-axis (Krx) for the two horizontal sides (side 1 at the top and 2 at the bottom), and the horizontal z-axis (Krz) for the two vertical sides (side 3 on the left hand and side 4 on the right hand). Table 2 lists the parameters before and after the tuning: the degree of constraint of the horizontal sides (1, 2) is lower than the vertical sides (3, 4). The updated parameters were obtained minimizing the differences between fi,exp and fi,num, which are the experimental and numerical values of the i-th frequency, and maximizing the MAC via the following convergence criteria:

$$\frac{{\sum }_{i=1}^{N}\frac{1}{i}\left|\frac{{f}_{i,exp}-{f}_{i,num}}{{f}_{i,exp}}\right|}{{\sum }_{i=1}^{N}1/i}\le {\delta }_{f}$$
(1)

and

$$1-\frac{{\sum }_{i=1}^{N}\frac{{MAC}_{i}}{i}}{{\sum }_{i=1}^{N}1/i}\le {\delta }_{s},$$
(2)

where MACi is the scalar indicator relevant to the i-th mode and N is the number of vibration modes on which the updating is based (in this case, N = 4 since only four modes were experimentally detected). As suggested in [22], δf = 5% and δs = 25% were assumed admissible values.

Table 2 Values of the updated parameters
Full size table

Figure 8 shows that frequencies’ differences and the MAC values for each mode together with the weighted percentage errors according to Eqs. (1) and (2) highlighting a good correlation.

Fig. 8
figure 8

Frequencies (in Hz) provided by numerical model and experimental identification, and MAC values after the model updating

Full size image

4 Damage scenarios

Paper [23] shows typical in-plane damage to an infill masonry panel (with either diagonal or sliding cracking) after an earthquake, including detachment of the panel from the surrounding RC frame, also because this lateral connection is a weakness of the handmade construction. This type of defect/damage is particularly interesting because it can reduce the capacity of the component for out-of-plane seismic actions (the horizontal arch mechanism cannot occur), causing failure and harm to people even in earthquakes of low magnitude, which do not damage the structures.

To check the ability of the OMA technique to detect damage via an engineering approach based on the updating of a numerical model, and comparing this with the application of AI algorithms to dynamic measures, three damage scenarios were implemented on the partition wall with increasing levels of severity. To assess the reliability of post processing of monitoring measures with respect to simulation of numerical damage, the importance of actually executing the damage is evident.

Damage scenario 1 involves the detachment of the plaster layer and partial demolition of the mortar along the upper half of one vertical side, because the damage in this kind of partition often starts in the corner zone of the wall that is typically more stressed in the interaction with the surrounding bearing structure, and built by hand, and, thus, vulnerable. Scenario 2 represents a more serious level of the previous condition, because the same detachment of the plaster layer and mortar demolition has extended along the entire vertical side. Finally, scenario 3 involves the most significant damage, because the infill leaf has been completely detached from the column, removing the mortar vertical joint.

The three damage scenarios were implemented (see Fig. 9), and further dynamic tests were performed to investigate whether the operation modal analysis could detect the damage. One acquisition of 3600 s with a sampling frequency of 200 Hz was carried out for each damage scenario. Figure 10 shows the sensors layout used for the damage scenarios.

Fig. 9
figure 9

Real and simulated damage scenarios

Full size image
Fig. 10
figure 10

Sensors layout for damage scenarios

Full size image

Moreover, the model updated with data taken from the healthy condition was used to define and analyse the damage scenarios that were actually implemented on the wall. Damage scenario 1 was introduced to the numerical model, reducing the thickness of the shell elements of the last column of the mesh grid to 10 cm (to simulate the removal of the plaster layer), while the stiffness of the springs inserted along the half of the right vertical was halved to simulate the partial removal of the layer of mortar between the partition and the RC column. The same changes—though along the entire vertical side—were introduced to the numerical model to simulate damage scenario 2. Damage scenario 3 was numerically simulated by assigning a very low value for the stiffness of the springs along the entire vertical side (a stiffness four orders of magnitude lower than for no damage). This value was set to take into account both the disconnection and friction phenomena. Indeed, the friction between the panel and the column could give a stiff connection at the interface under micro-vibration conditions. Figure 9 also shows the drawing of the FE model for the three cases, with an indication of the length of removed plaster represented by a thin black line and the entire side that had been detached represented by a thick black line. Note that the models with damage were made to obtain feedback from a numerical modal analysis. The proposed methodology requires neither model updating nor damage simulation.

5 Damage detection by experimental OMA and numerical model

It is worth noting that numerical models were used to complete the framework of the possible approaches in the analysis of the data from monitoring systems, but the aim of this work is to assess damage detection procedures based only on experimental data. Therefore, for this aim, experimental data that varied due to damage were post-processed with traditional techniques to obtain an anomaly index based on the dynamic properties of the element (i.e., variation of frequencies and mode shapes). The raw data were then elaborated by AI techniques to obtain alternative anomaly indexes.

First, it is worth underlining that identification of the frequency associated with the first mode was difficult for the cases of damage scenario as well. Figure 11 shows very close peaks in the plots of the singular values of spectral densities. As for the undamaged scenario, in this case too, the AutoMAC analysis confirms that they are the same modes. Accordingly, for the comparison all the detected values were considered.

Fig. 11
figure 11

Magnification of singular values of spectral densities: a damage scenario 1; b damage scenario 2; c damage scenario 3

Full size image

In Fig. 12, the experimental and numerical results obtained for the damage scenarios are compared with the results of the undamaged scenario. The variation of frequency for increasing damage from scenario 1 to scenario 3 is lower than 7% for all modes identified with the experimental data. The MAC values are close to one for all the modal shapes identified for the first two damage scenarios and lower (0.88, 0.66 and 0.16) for the third damage scenario. Low values of the MAC that are much lower than one indicate a change in the modal shapes. Therefore, the experimental results show that the OMA technique could not detect the damage of scenarios 1 and 2, while damage detection for scenario 3 was enabled by considering the change of frequency (about 5% in average for modes 1 and 2) and, overall, the change of modal shape (MAC lower than 0.9 and 0.7 for modes 1 and 2, respectively). On the other hand, damage simulation by the numerical model provides results that are trivial when compared to those obtained by experimental methods: (i) the frequency always decreases when the damage increases for all the modes considered; and (ii) scenario 3 is clearly individuated with variations of 30% in the second and third modes.

Fig. 12
figure 12

Dynamic effect of damage scenarios (experimental and numerical results)

Full size image

In contrast to the numerical simulations, the trend of the experimental frequency obtained for the damaged walls underlines that the OMA experimental procedure can provide an anomaly detection but may fail for an identification of the damage type and localization. Note that this result is obtained by comparison with a numerical model calibrated on the healthy structure.

5.1 Damage detection by AI algorithms

The limited ability of OMA demonstrated by the previous analyses leads us to consider the ability of a data-driven approach that can arrive at the same anomaly detection, avoiding both the over-complicated analysis of the experimental results and the developing of numerical model. For the application of the AI to automatically detect damage in the infills, the methodology proposed by the authors of [14] is applied, which aims at identifying the presence of damage regardless of its severity, with outcomes that can be interpreted in terms of a binary classification response. Basically, the methodology depends on performing a training stage that is based only on normal data, thus avoiding analysis of structure deterioration through numerical or analytical modeling.

6 Brief outline of the algorithm

Autoencoders correspond to neural networks that are logically divided into two components: an encoder and a decoder. The encoder projects the input into internal representation, while the decoder reconstructs the input from the internal representation. The internal representations form the latent encoding space. In particular, variational autoencoders (VAEs) are generative models where the latent encoding space is defined by a mixture of distributions instead of a fixed functional mapping. In the proposed approach, VAE is trained only on undamaged data, so that the network learns the natural structure behavior (undamaged conditions). The damage-sensitive features are the errors made by the network in the signal reconstruction. In this case, authors used the mean squared error (MSE), which measures the reconstruction error between the input acceleration signals and their reconstruction and the original-to-reconstructed-signal ratio (ORSR), which represents the ratio in decibels between the magnitudes of the original signal and its reconstruction [14]. Data are first fed into the VAE. Then, using original and reconstructed signals, after a feature extraction stage, data are fed into a one-class support vector machine (OC-SVM) for classification as being either damaged or undamaged. For this application, acceleration generated by the updated finite element (FE) numerical model was used, to test the feasibility of the methodology that would be tested on the damage scenarios.

7 Data pre-processing and arrangement

Data were downsampled to emphasize the most important informative content of the signals, thus reducing the data dimensionality. The downsampling was set at 100 Hz, based on the expected dynamical properties of the instrumented element. Data were then divided into frames of one second each. A random 10% of the undamaged samples were extracted to evaluate the performance of the model when the structure was in an undamaged state. Finally, a random 10% of the resulting training set was extracted to validate the generalization capability of the VAE during the training stage. Finally, for each sensor s, data were normalized using a Z-score as follows:

$${{\varvec{X}}}_{{\varvec{s}}}\left({\varvec{t}}\right)=\frac{{{\varvec{X}}}_{{\varvec{s}}}-{{\varvec{\mu}}}_{{\varvec{s}}}}{{{\varvec{\sigma}}}_{{\varvec{s}}}},$$

where \({X}_{s}\) represents data related to the sensor \(s\), \({\mu }_{s}\) represents their mean, and \({\sigma }_{s}\) represents their standard deviation computed on the training set.

Following [14], a first attempt to model a VAE to process the experimental cases in this study was made using a multi-layer perceptron VAE (MLP-VAE). Throughout the model selection process, exploration of the MLP-VAE architecture encompassed a broad range of configurations, exhaustively investigated through grid search, while constraining the decoder to be symmetric to the encoder. In particular, architectures with 1–3 hidden layers were evaluated, with each layer containing 2–256 neurons in powers of two. The dimensions of the latent space were explored among 32, 64, and 128 neurons. Every configuration was evaluated with respect to the Sigmoid, Tanh, and ReLU activation functions.

However, after the model selection stage, the MLP-VAE was unable to correctly distinguish between damaged and undamaged cases. Figure 13 shows the graphical representation of a random signal sampled from a sensor (blue line) and its reconstruction (orange line) obtained through the MLP-VAE. The x-axis has been labeled with ‘time [s]’ to indicate that it represents the time axis in seconds. In particular, the plots show 1 s of signal. Also, the signal is normalized, and is, thus, dimensionless. For this reason, the y-axis indicates the data from which the signal was randomly sampled for the plot (in this case, X5 since it belongs to sensor 5). The best model barely reconstructed the input training signals, since it tended to approximate input signals to values close to the average trend, i.e., high error of signals reconstruction. For this reason, the choice of the model was oriented toward Convolutional VAE (CVAE) to leverage their capability in successfully capturing spatial and temporal dependencies in data, thus allowing us to obtain a reliable signal reconstruction and internal modeling of undamaged prior distribution. The acquisitions of all the sensors were arranged through the channel dimension, thus resulting in samples that had 11 input channels and 100 features. A summary of the dataset is reported in Table 3.

Fig. 13
figure 13

Graphical representation of the signal reconstructed by VAE with the feed forward neural network

Full size image
Table 3 Data dimensionality for each partition
Full size table

8 Training configuration and CVAE architecture

An RMSProp optimizer was used in the training stage, with a starting learning rate of 0.0001, a weight decay factor of 0.0003, and momentum factor of 0.7. A learning rate scheduler was adopted to reduce the learning rate by a factor of 30% when it plateaued for 20 consecutive epochs. Batch size was fixed to 256, and early stopping was used as convergence criteria, with 100 epochs of patience. A maximum number of epochs was fixed at 15,000. Training data were augmented by adding random Gaussian noise at each epoch and flipping data through normalized amplitude and time.

Since the CVAE processes one-dimensional input signals, all the convolutional layers were implemented as one-dimensional convolution.

The encoder network was based on the following convolutional block:

Convolutional layer → Batch normalization → Tanh → Max pooling layer.

This block was repeated three times. In each block, kernel sizes of the convolutional layer and the max pooling were, respectively, 5 and 2. The number of filters was doubled for each building block, starting from 32 filters. Moreover, same-padding configuration was used for each convolutional layer to ensure the same dimensionality of the data after the convolutional operation. The output of the last convolutional block, which had a shape of (128, 12), was flattened to a vector of length 1536, and given as input to the resampling process, which was performed by mapping data in two dense layers representing, respectively, the mean and the variance of the latent isotropic unit, with a Gaussian distribution of 70 dimensions.

In the decoder network, latent representations were first given as input to a dense layer of 1536 neurons and Tanh as activation function. This was followed by a dropout layer with a 20% of drop probability. Next, data were reshaped to recover the dimensionality of the output of the last encoding block, i.e., (128, 12). The following convolutional block was used in the decoder network:

Transposed convolution → Convolutional layer → Batch normalization → Tanh.

This block was repeated three times to mirror the encoder network, except for the last block where the batch normalization layer and Tanh activation function were omitted. Same-padding configuration was used for each convolutional layer for the decoder network as well. Transposed convolutions were used to mirror the functionality that max pooling layers have in the encoder network, i.e., expanding the input data. Thus, the number of input and output filters are equal, the kernel size is equal to 2, and convolution is made using a stride of 2. The kernel size of the convolutional layers is instead fixed at 5, as in the encoder network, and the number of filters is halved after each block. In the last block, the number of output filters of the convolutional layer is equal to the number of sensors. A graphical representation of the CVAE architecture is shown in Fig. 14.

Fig. 14
figure 14

Graphical representation of the encoder and decoder network of the CVAE

Full size image

After the extraction of damage-sensitive features, differently from [14], both for MSE and ORSR, data within 2 standard deviations were selected from the mean to remove outliers that might compromise the classification made with the OC-SVM (note that, as in [14], the OC-SVM is trained with the goal of including all the undamaged data within the decision hypersphere). Leveraging Chebyshev’s theorem [24], this assumption guarantees at least 75% of the data points for each of the distributions, whatever the data distribution, up to 95% for a standard distribution. Finally, data were normalized using Min–Max normalization before the training stage of the OC-SVM, which was fitted considering radial basis function as kernel function and v = 0.001 [14].

9 Damage detection on simulated data

The first application of this CVAE was carried out on the data gathered through the FE model (i.e., undamaged and damaged partitions).

Figure 15 shows the graphical representation of the same random signal sampled from sensor X5 shown in Fig. 13 (blue line) and its reconstruction (orange line) obtained through the CVAE. By comparing the representations, it is easy for the reader to see that when using the CVAE the signal reconstruction is significantly better than when using the MLP-VAE.

Fig. 15
figure 15

Graphical representation of the signal reconstructed by VAE with the convolutional neural network

Full size image

Figure 16 shows the reconstruction of a random sample for each partition (training and validation are referred to the undamaged partition). The normalized input signal (in blue) and its reconstruction obtained through CVAE (in orange) are reported. For each case, signals related to each sensor (A1–A11) were concatenated to allow the simultaneous graphical representation of all the sensors data belonging to a random input signal. The vertical lines guide the reader in identifying the signal segments attributed to each sensor. It is worth noting that, for instance, the reconstruction of the signal associated with sensor A5 in Test Damage 2 exhibits significant inaccuracies, suggesting a potential damage. Figure 17a,b show the damage-sensitive feature extraction MSE and ORSR. Each distribution is represented through boxplot (top) and histogram (bottom). The box represents the interquartile range, and the orange line represents the median value. The two lines emanating from the box are used to detect potential outliers (represented by circles) within the data distribution.

Fig. 16
figure 16

Graphical representation of a random input signal (blue) and its reconstruction obtained through CVAE (orange). Vertical lines separate signal segments attributed to each sensor

Full size image
Fig. 17
figure 17

MSE and ORSR distribution—whole dataset (a,b) and without outliers (c,d)

Full size image

The damage-sensitive feature extraction MSE and ORSR are characterized by several outliers. This leads the OC-SVM to learn a decision hypersphere that covers portion of the hyperspace where undamaged data are not actually located. Extracting only data within 2 standard deviations from the mean value (Chebyshev’s inequality theorem), 3% of data were excluded and considered as outliers. Figure 17c, d show the features distributions after the outlier removal.

The method performance evaluation was obtained by the probability of damage (PoD) computed as.

PoD = c / n,

where c is the number of samples classified as damaged by the OC-SVM and n is the total number of samples. Table 4 shows that the PoD values reflect the a priori known damage conditions of the structure: damage probability is low for the undamaged case, while it is high for the remaining cases (i.e., damaged cases). Sensors that could better identify damage were trained with a strong computational effort. The last row shows that when the network is trained by using all the sensors together, the performance of the ANN increases, with scenarios 2 and 3 identified perfectly. In the authors’ opinion this result is similar to that obtained by using OMA: with the operational analyses the frequency alone is not a damage feature, while modal shapes are more significant. Here, using an ANN, the data processed together can exploit information about the different vibration shapes of the wall in the various scenarios.

Table 4 PoD values, in percentage, for undamaged and damage scenarios—simulated data (standard deviation in brackets)
Full size table

By way of example, Fig. 18 shows the graphical representation of the OC-SVM for the undamaged partition and three damage scenarios, based on the data recorded by all sensors. Input samples are represented through red and blue dots. The red area represents the hypersphere learned by the OC-SVM. An input sample falling into the red area is classified as undamaged (red) by the OC-SVM. On the other hand, an input sample falling outside the red area is classified as damaged (blue) by the OC-SVM. Data classification is represented for the undamaged case and three damage scenarios. For damage scenario 3 the scale of representation greatly changes, to allow the representation of all the data.

Fig. 18
figure 18

Damage detection: graphical representation of the OC-SVM for the undamaged partition and three damage scenarios (numerical data)

Full size image

10 Damage detection on experimental data

In this section, the AI algorithm is applied to the experimental data. Table 5 shows the PoD values obtained for the undamaged scenario and the three damaged scenarios. The machine learning algorithm identified three damage scenarios realized with increasing intensity on the wall, with the index PoD equal to 23%, 51% and 94% on a scale ranging from 0 to 100, where 100 represents the maximum score of anomaly detection. Even if there is a great improvement when the ANN is trained with the data recorded by all the sensors, it is worth noting that for the approach used here, which is based on acceleration analysis, the sensors most sensitive to damage are those closest to the damaged area (i.e., sensors 4, 5, and 9), and in particular sensor 9 for damage scenario 1 and sensor 5 for damage scenarios 2 and 3, which only differ according to intensity. This result was not obtained when the CVAE was carried out on the data gathered through the FE model.

Table 5 PoDavg values, in percentage, for undamaged and damage scenarios—experimental data (standard deviation in brackets)
Full size table

It is important to note that the proposed AI approach was first applied to simulated data to assess its plausibility and effectiveness (see Table 4 and Fig. 17), and then validated on real data (see Table 5 and Fig. 18). The tests on real data aim to verify the validity of the proposed AI approach in real-life scenarios. Including noisy data in the training set allows us to enhance the generalization capabilities of the approach, a practice commonly used in various machine learning methods.

Figure 19 shows the graphical representation of the OC-SVM for the three damage scenarios, based on the data recorded by all sensors. Although in the authors’ opinion the quality of the result would improve greatly if the quantity of experimental data available increased, the figure shows that the network can identify even slight damage in the scenarios.

Fig. 19
figure 19

Damage detection: graphical representation of the OC-SVM for the undamaged partition and three damage scenarios (experimental data)

Full size image

To provide some preliminary information on the possibility of reducing the number of sensors necessary to detect the three damage scenarios, Table 6 lists:

Table 6 Comparisons between techniques and sensor layout
Full size table

- the values of frequency variation, ε = (fdam—fundamn) / fundamn, which can be obtained with OMA applied to acceleration recorded by the central sensor only;

- the differences between modal forms, μ = (1—MAC), which can be obtained with OMA using data recorded from five sensors (one in a central position and four on the edges of the wall);

- the values of ρ = PoD × 100, which can be obtained with AI for each of the five sensors.

As mentioned above, it is confirmed that OMA succeeds in providing damage indication only for scenario 3, and such that it is necessary to also consider the parameter on MAC (μ = 12%), because |ε|= 7% is trivial for the purpose of damage detection. Therefore, the damage detection is possible only if five sensors are used, at least.

In contrast, using AI algorithms, the damage can be identified with only one sensor. For instance, a single sensor in the center (i.e., A1) would enable damage identification for all three scenarios. The best outcome is obtained with the sensor that is closest to the location of damage, namely sensor 5. Sensor 5 consistently exhibits the highest PoD value in both the numerical and experimental scenarios, indicating that results belonging to single sensors could provide valuable insights about the damage location. To reduce the costs of real application, using the configuration with four sensors in the centers of each of the four sides of the partition (A2, A5, A8, A11) is proven to enable damage localization. An individual moving sensor could also be used for damage localization, but only if this innovative algorithm is used, because OMA would be not effective with one sensor at a time.

11 Conclusions

The use of artificial neural networks (ANN) as a machine learning tool is shown to be highly promising for monitoring the condition of non-structural elements. Indeed, the proposed artificial intelligence (AI) methodology is based on a machine learning approach, which belongs to the class of ‘data-driven’ methods, with ANN that are trained exclusively on actual measures recorded by sensors. Hence, the approach is model-free; i.e., it can be applied without creating a model of the structure, since it is based on the data recorded by sensors.

This study, which has aimed to identify which features are important to consider for use in a structural context, may be of use when searching for the correct design of a network of sensors to serve a neural network pilot. The focus on the VAE framework revealed limitations of MLP architectures for the objective of this work, leading to the implementation of a CNN-based VAE (CVAE).

The application of the methodology to data recorded from a real infill wall allowed us to demonstrate the effectiveness of a monitoring system that issued a warning about infill damage. Indeed, the machine learning algorithm identified three damage scenarios that had been realized with increasing intensity on the wall.

The algorithm recognized some signal alteration due to damage located at the corner of the wall, or damage that was spread along the interface between the wall and the building’s column/structural elements. It did so better than traditional techniques, and was also able to perform with a reduced set of accelerometers. The results obtained with the AI algorithm are encouraging because the algorithm was capable of detecting damage in all three damage scenarios, whereas the traditional OMA succeeded in only one case. Moreover, the algorithm was demonstrated to be able to identify anomalies even when a single sensor was used.

Exogenous environmental factors—such as temperature, wind, humidity, etc.—which may alter measurements and generate false positives, were not considered in these experiments, because their effects on internal ambient are lower than in the case of external structures. Nevertheless, they may also be considered in the algorithm training of the proposed method, so as to improve its forecast robustness.

Future developments will include: (i) optimizing the number and placement of sensors by introducing a criterion into the machine learning algorithm to identify an automated improvement in sensor layout; and (ii) using this methodology for dynamic identification tests with known input (i.e., an instrumented hammer).