On the denoising of structural vibration response records from low-cost sensors: a critical comparison and assessment

Abstract

Due to the increasing quest of adopting low-cost sensors in structural health monitoring (SHM) processes, which may lead to detecting signals contaminated by significant levels of noise, the need to devise appropriate and effective denoising strategies, at the post-processing stage, is becoming more and more essential. Among several approaches proposed in the literature, it has been demonstrated that the employment of discrete wavelet transform (DWT) as a multi-rate filter bank, as well as the use of singular value decomposition (SVD), may result to be quite effective in signal denoising within various research fields, as biological, acoustic and mechanical. Here, DWT- and SVD-based denoising techniques are first independently reconsidered and reimplemented, aiming at exploring their optimal calibration in purifying noise-corrupted vibration response signals encountered in civil engineering applications. Then, a systematic performance evaluation is provided within a comparative framework, developed at an increasing level of noise affecting the measurements, in terms of noise-to-signal (N/S) ratio. In the study, two specific classes of synthetic response signals are first considered, namely earthquake and ambient vibration signals, since they may be assumed as representative of more general non-stationary and stationary signal typologies, respectively. To achieve a complete description of the clarified signal, strengths and weaknesses of the two denoising approaches are explored, in both time and frequency domains. The results prove the effectiveness of the analyzed implementations, especially in purifying seismic response signals, while some limitations may arise concerning the treatment of ambient vibration signals, in particular for the DWT-based denoising technique. Finally, a real case study is analyzed, where both denoising approaches are adapted and employed for clarifying acceleration signals detected on a modern short-span railway bridge, with rather satisfactory results, for both techniques.

1 Introduction

Structural Health Monitoring (SHM) refers to the process of a continuous assessment of the current health conditions of a structural system, aiming at improving its integrity, by detecting potential manifestations of damage, before this may reach a critical state, possibly detrimental for structural safety. Dealing with the structural engineering context in general terms, virtuous examples in which SHM procedures have been successfully implemented, on different typologies of structures, may be found in the literature (e.g. Capellari et al. [10], Chatzi and Smyth [12], Ferrari et al. [26,27,28,29], Koo et al. [41], Lee et al. [42], Roberts and Dodson [63], and cited references). It is clear that the acquisition stage of signals, from which the current structural conditions may be detected, constitutes a crucial phase for the global success of such an analysis.

Referring to the civil engineering field, in particular, the most common typologies of sensors employed for the detection of response signals may be considered as accelerometers and displacement sensors, as the kind of response data that can be recorded may be good enough to describe the current structural health conditions. However, the raising need to adopt a low-cost and easy-to-use monitoring instrumentation, toward recording the structural dynamic response under live external loads, may often lead to measurements affected by sensible amounts of noise. This spurious signal contamination, superposed upon the useful structural response signal, may lead to spoil the monitoring information content and alter the prediction of the structural response, leading to unreliable SHM implementations. Consequently, noise reduction from acquired response signals constitutes a crucial issue in an efficient SHM process of civil structures. This may hold true, specifically, for low-cost sensors, especially when adopting automated monitoring equipment and systems, markedly after long-term operation, in terms of potential implications about a higher endemic presence of spurious noise, to be cleaned by appropriate signal processing approaches, as enquired in the present investigation.

The procedure by which it becomes possible to reconstruct a certain source signal, starting from a recorded, noise-corrupted one, removing its noisy part, without losing its useful information, may be referred to as denoising Buades and Coll [9], Chen and Bui [15], Portilla and Strela [56]. Two main issues have to be accounted for in dealing with a denoising procedure, namely: (a) how the initial data may be affected by the added noise; (b) how the reconstructed signal, out of the denoising procedure, shall be able to correctly reproduce the original (truthful) signal, preserving its fundamental features. Thus, the crucial matter of each denoising process lies in removing most of the unwanted noise, without losing the useful part of the signal containing the true monitoring information.

Several computational techniques have been proposed in the literature to address the denoising of signals, some of which are currently in development. The most basic and traditional way to remove the noise affecting the signals is represented by the employment of band-pass filters with cut-off frequencies, i.e. high-pass or low-pass filters, which allow to remove all the frequency contents greater or lower than a certain frequency value, respectively. Moving average filters, as well as Gaussian filters, can be considered as typical examples belonging to such a category. However, although the application of such filters may be useful when the noise is located within a precise frequency band, which is different from the frequency band in which the signal shall lay, in most cases, when the noise displays a similar frequency content as that of the signal to be analyzed, they cannot be effective, since even much of the useful signal may be lost.

This also constitutes a main drawback in the use of the Fourier Transform toward denoising [22]. In other words, such methods act in a global sense, since they process the signal regardless of whether the noise displays a uniform frequency distribution or not. However, in real cases, the noise distribution may be far from being uniform and it may be desirable to apply a “localized” form of denoising. In the past decades, this has led to searching for different methodological approaches and alternative filtering techniques. Arezki and Berkani [5] and Kam et al. [36] proposed the application of adaptive filters to remove Gaussian white noise and impulse noise from signals. Within their works, an iterative procedure was set up for minimizing in real time the error between the original signal and the denoised signal [45]. To suppress impulse noise, Veerakumar et al. [72] introduced a new algorithmic approach based on the combination of fuzzy logic and an asymmetric trimmed median filter, whereas in Premchaiswadi et al. [57] a kFill algorithm was combined with a median filter for reducing the impulse noise that could occur on images (denoising methods may also be applied on images, as an image may be interpreted as a two-dimensional signal, Ergen [22]).

A further powerful methodology for separating noise out of corrupted data involves the application of a discrete wavelet transform (DWT). In particular, in Dohono [20], a first DWT-based approach for denoising one-dimensional signals was provided. Afterwards, Chang et al. [11] introduced an innovative adaptive Wavelet thresholding for image denoising and compression, called BaeyShrink method. Furthermore, the use of singular value decomposition (SVD) for denoising purposes has also attracted considerable interest, as demonstrated e.g. in Konstantinides et al. [39] and Konstantinides and Yao [40], where a new filtering and noise estimation technique was introduced, known as block-based singular value decomposition (BSVD) filtering. Finally, a possible alternative approach in enhancing the quality of noise-affected signals may concern the application of a Kalman filter (KF) [13, 14], Ravizza et al. [59], and works cited therein). For instance, in Ravizza et al. [59], a KF has been successfully employed within a Heterogeneous Data Fusion procedure, to correct noisy displacement measurements, by enhancing them through a few reliable acceleration signals, toward cleaning structural acquisition extraction and modal dynamic identification.

In this paper, two of the above-mentioned approaches are systematically reconsidered and developed, in tackling first a controlled denoising problem set on reference structural response signals. In particular, the development of a DWT procedure, as a multi-rate filter bank, as well as the implementation of a SVD technique are extensively inspected, in their employment toward denoising structural response signals. Here, the two approaches are implemented and assessed on noise-corrupted structural vibration response signals that may be typical of practical applications belonging to the civil engineering context.

In signal processing, DWT is commonly applied for many other purposes, in addition to signal denoising. For instance, its employment shall be rather useful in detecting trends, breakdown points, discontinuities in higher derivatives and self-similarity of signals [69]. Moreover, as a denoising technique, it has been performed on signals of a different nature, i.e. gravity and magnetic signals [24], biological signals [3, 6], such as electroencephalograms (EEG) or electrocardiograms (ECG), but also on acoustic signals [25] and pressure signals [68]. However, its application in denoising structural signals of the civil engineering field does not seem to have been deeply inspected yet. Similarly, also SVD has found applications in digital signal processing as a technique for noise reduction. In particular, its effectiveness has already been proven dealing with audio signals (e.g. in Baravdish et al. [7], where it is combined with a non-linear PDE method), or biomedical signals [67], or even for radar target recognition of electromagnetic signals [43], but also for improving the quality of images [33]. Moreover, in the civil engineering domain, it has been exploited for many purposes, including those of modal dynamic identification (e.g. in Pioldi et al. [49,50,51]) and of structural damage detection at an early stage of development [64].

In the present investigation, the performances of DWT- and SVD-based denoising techniques is first assessed on earthquake and ambient vibration synthetic response signals, and a critical comparison based on the effectiveness of such methods is provided, at variable added noise. The selection of these two types of response signals is also motivated by the fact that they can be considered as being well representative of two great families of signals. In particular, the earthquake excitation input is known as a typical non-stationary signal, whereas the ambient vibration signal is instead considered as a common stationary signal, since it may almost be constant in time and frequency. To inspect advantages and possible limitations of the mentioned denoising techniques, in relation to the typology of the processed signal, constitutes an important goal of this study. To synthetically generate numerical response signals, the seismic and ambient vibration input signals are separately examined and applied on a reference structure, namely a one-bay ten-story shear-type frame, used as a benchmark mockup. In particular, the first- and last-floor acceleration responses are monitored, in the cleaning of the response signals (accelerations). Furthermore, to simulate the effect of the errors that may occur on the measurements during a real signal acquisition stage, within the analysis, several noise-to-signal (N/S) ratios are considered, for a superimposed noise signal affecting the data.

It is worth noting that the present investigation is first based on synthetic signals, namely pseudo-experimental signals numerically generated prior to the denoising analysis, for a crucial, necessary condition validation of the denoising procedures. Then, in the final part of the work, a real case study involving real signals, directly acquired “in situ” by appropriate sensors placed on a specific structure (a railway RC bridge), are eventually presented. Preliminary denoising simulations of the present efforts were presented in conference papers [60, 61].

It may be stressed out that, while the present focus is placed on dynamic measurements, the enquired techniques shall effectively be applied to static and quasi-static data as well. In fact, for data with such attributes, the denoising process shall become, in principle, more effective, than for dynamic measurements, where the noise sources may be propagated in time, through the associated structural dynamics, forming a more challenging denoising scenario and quest.

The main goals that this study aims to pursue are the following:

• to explore which is the optimal calibration of a DWT-based denoising technique, for dealing with both seismic and ambient vibration response signals;

• to examine the possibility to successfully exploit a SVD-based implementation, toward the clarification of the above-mentioned response signal typologies;

• to provide a critical comparison of strengths and weaknesses of each denoising method, for exploring their effectiveness, at a controlled, increasing level of noise inserted on the source signals;

• to recover the original signal in the time domain, with the target also of preserving its spectrum in the frequency domain, to achieve a comprehensive and more reliable reconstruction of the response signal, to be considered as rather truthful toward real monitoring purposes;

• to inspect if the two explored denoising techniques may also become effective in real applications, for clarifying real experimentally recorded vibration response signals.

The paper is organized as follows. In Sect. 2, a brief theoretical framework on the adoption of DWT and SVD techniques for denoising purposes is provided, together with the needed strategies for their adaptation in the handling of the present civil engineering scenario. Section 3 presents the analysis procedure, through the description of the benchmark dynamical system, the process of generation of the simulated noise-affected signals and the obtained results. Here, particular importance is placed on the preliminary calibration of the denoising technique based on DWT, through the search for the optimal configuration of its characteristic parameters; additionally, the criterion through which a SVD approach may effectively be exploited toward denoising purposes is also explored. Within the same section, a performance assessment of the two studied approaches is presented, and a critical comparison is provided. Section 4 covers a real case study, in which both DWT- and SVD-based denoising methods are reinterpreted, in clarifying real (noise-affected) acceleration signals acquired from a railway bridge. Conclusions and global remarks are finally outlined in Sect. 5, as well as some possible future developments of the present study.

2 Theoretical framework

2.1 Problem statement

A typical denoising problem pertains at decomposing a noise-corrupted signal y[k], $k=0,1,\dots ,N-1$, sampled at a rate $F_s$ (Hz), as

$$\begin{aligned} y[k]=x[k]+n[k] \end{aligned},$$

(1)

where x[k] corresponds to the noise-free counterpart and n[k] is a zero-mean, stationary, Gaussian white noise process of variance $\sigma _{nn}^2$ [1, 47]. Writing Eq. 1 in vector form [4],

$$\begin{aligned} {\mathbf {y}}={\mathbf {x}}+{\mathbf {n}} \end{aligned},$$

(2)

where

$$\begin{aligned} {\mathbf {y}}&= \begin{bmatrix}y[0]&y[1]&\dots&y[N-1]\end{bmatrix}^T,\end{aligned}$$

(3a)

$$\begin{aligned} {\mathbf {x}}&= \begin{bmatrix}x[0]&x[1]&\dots&x[N-1]\end{bmatrix}^T, \end{aligned}$$

(3b)

$$\begin{aligned} {\mathbf {n}}&= \begin{bmatrix}n[0]&n[1]&\dots&n[N-1]\end{bmatrix}^T, \end{aligned}$$

(3c)

one is thus interested in extracting an “optimal” approximation of ${\mathbf {x}}$, $\hat{{\mathbf {x}}}$, as well as an estimate of the noise contamination variance, ${\hat{\sigma }}_{nn}^2$.

Signal y[k] is treated as a realization of a stochastic process that might be either stationary, or non-stationary. Under this setting, in the following the DWT- and SVD-based denoising approaches are first outlined, and then an extensive benchmark study is performed, using artificial and real-world structural vibration response data.

2.2 Discrete wavelet transform-based denoising

Wavelet Transform (WT) may be seen as the process through which a certain signal can be decomposed into its low and high frequency components, via the introduction of a set of orthonormal wavelet functions, constituting a wavelet basis, which originate from the mother wavelet by scaling and shifting operations through two parameters known as scale parameter j and shift parameter k. In particular, when scale parameter j is chosen as a power of two, a dyadic orthonormal wavelet transform is obtained [74].

The measure of frequency content similarity between the signal that has to be denoised and the selected wavelet function is expressed by the so-called wavelet coefficients, determined as a convolution of the signal and the scaled wavelet function, which may be considered as an expanded band-pass filter [62]. In particular, approximation coefficients are associated with low frequency components, whereas detail coefficients refer to high frequency components. In this way, WT is implemented with a reconstruction filter bank using an orthogonal wavelet family. Such an employment of WT as a filter bank for signal denoising purposes is also known as DWT [22].

A DWT denoising procedure may be conceptually summarized as follows:

• (i) Decomposition of the original signal into the wavelet domain: digital signal y[k], $k=0,...,N-1$, is transformed into the wavelet domain by [46]

  $$\begin{aligned} Y_{j,k}=\sum \limits _{i=0}^{N-1} y_i W_{i,(j,k)} \end{aligned},$$

  (4)

  where N is a power of two and $W_{i,(j,k)}$ is the forward transform operator, defined as

  $$\begin{aligned} W_{i,(j,k)}=2^{-j/2}\psi (2^{-j}i-k) \end{aligned}.$$

  (5)

  Equation 5 indicates that the orthonormal basis functions are obtained through translations and dilations of a certain mother wavelet $\psi$ by means of shift (or translation) parameter $k\in {\mathbb {N}}$ and scale (or dilation) parameter $j\in {\mathbb {N}}$, sampled along the dyadic sequence [8, 79]. Mother wavelet $\psi$ is an oscillating function that may be selected among a set of pre-defined ones, including Symlet, Coiflet, Daubechies, Biorthogonal, Reverse Biorthogonal and Discrete Meyer, among others.

  Applying the DWT to Eq. 1 implies

  $$\begin{aligned} Y_{j,k}=X_{j,k}+N_{j,k} \end{aligned}.$$

  (6)

  Accordingly, by introducing denoising operator $D:{\mathbb {R}}^+\rightarrow [0,1]$, the DWT of x[k] can be recovered by

  $$\begin{aligned} {\widehat{X}}_{j,k}=DY_{j,k} \end{aligned},$$

  (7)

  and the denoising problem aims at designing D for minimizing the Mean Squared Error (MSE) between $X_{j,k}$and ${\widehat{X}}_{j,k}$

  $$\begin{aligned} r=E\{(X_{j,k}-DY_{j,k})^2\}=\frac{1}{N}\sum \limits _{i=1}^{N} (X_{j,k}-DY_{j,k})^2 \end{aligned},$$

  (8)

  where $E\{\cdot \}$ denotes expected value.

• (ii) Thresholding of the DWT coefficients: introducing a certain non-dimensional threshold value $\lambda$, which may be a function of DWT decomposition level l (level-dependent threshold) or not (level-independent threshold) [19], the noise affecting the signals may be considerably reduced by scaling the wavelet coefficients smaller than $\lambda$, so that only the coefficients greater than $\lambda$ are considered to be associated to reliable data, whereas the ones below $\lambda$ are set equal to zero.

  The idea is that the noise in the transformed domain tends to disperse over all the wavelet coefficients, so that a strategy in which values below a proper threshold are set to zero removes most of the noise, while preserving the information. In fact, switching into the wavelet domain, transformed signal $Y_{j,k}$ will have many more non-zero coefficients than $X_{j,k}$, due to the noise contribution. Thus, by selecting an appropriate threshold $\lambda$, which allows to switch from wavelet coefficients $Y_{j,k}$ to thresholded coefficients $Y_{j,k}(D,\lambda )$, it may be possible to alleviate the noise contamination.

  Two different types of thresholding may be performed, i.e., hard thresholding and soft thresholding [18]. The former zeros out all the wavelet coefficients (in absolute value) smaller than $\lambda$, by preserving the others unaltered:

  $$\begin{aligned} Y_{j,k}(D,\lambda )=\left\{ \begin{array}{lcl} Y_{j,k} &{}if &{}\left| Y_{j,k}\right| \ge \lambda \\ 0 &{}if &{}\left| Y_{j,k}\right| <\lambda \\ \end{array} \right. \end{aligned}.$$

  (9)

  In contrast, the latter reduces all wavelet coefficients (in absolute value) greater than $\lambda$ by a quantity equal to the threshold itself:

  $$\begin{aligned} Y_{j,k}(D,\lambda )=\left\{ \begin{array}{lcl} \left| Y_{j,k}\right| -\lambda &{}if &{}\left| Y_{j,k}\right| \ge \lambda \\ 0 &{}if &{}\left| Y_{j,k}\right| <\lambda \\ \end{array} \right. \end{aligned}.$$

  (10)

  Since typically the noise mainly affects the high frequency components, it is common to apply the thresholding to the so-called detail coefficients, i.e. to the low-frequency band that usually contains the most important components of the signal.

  Both thresholding types are amenable to some drawbacks. Soft thresholding (also referred to as wavelet shrinkage), is characterized by bias when the preserved coefficients are large, while hard thresholding is “sensitive” to small changes in the signal and, consequently, suffers from stability issues. To overcome these problems, new hybrid solutions have been provided [30, 31]. In specific, Eqs. 9-10 indicate that the choice of an appropriate threshold $\lambda$ value is crucial. Indeed, adopting a “small” threshold might lead to a reconstructed signal that will still be considerably affected by noise, whereas a “large” threshold might lead to the suppression of useful signal information. Although for a generic signal of length N the Universal Threshold $\lambda \propto \sqrt{2lnN}$ has been proposed, a fair amount of studies examines other level-dependent or block-dependent approaches [18, 19, 37].

  In this study, thresholding rules that belong to the Donoho-Johnson class are considered, namely:

  • Visu shrink (Sqtwolog) [18]. This rule provides $\lambda$ regardless of the DWT decomposition level in accordance to

    $$\begin{aligned} \lambda =\sigma \sqrt{2lnN} \end{aligned},$$

    (11)

    where $\sigma$ is the standard deviation of the Gaussian white noise added to the clean signal.

  • SURE shrink (Rigorous SURE) [19]. In this case, a threshold level is assigned to each decomposition level of the Wavelet Transform, based on the principle of minimizing the Stein Unbiased Estimate of Risk (SURE)

    $$\begin{aligned} {\text{SURE}}(\lambda ,\mathbf{y })=N+\sum \limits _{i=0}^{N-1}[\text {min}(\left| Y_{j,k}\right| ,\lambda )]^2 -2(i:\left| Y_{j,k}\right| <\lambda ) \end{aligned}.$$

    (12)

    The underlying procedure is very versatile and suitable for denoising a wide range of functions.

  • Heuresure (Heuristic SURE) [37]. This rule is a combination of the two previous ones. If $\lambda _1$ and $\lambda _2$ are the threshold values obtained from Universal and Rigorous SURE methods, respectively, the Heuristic SURE rule assigns a threshold $\lambda$ as

    $$\begin{aligned} \lambda =\left\{ \begin{array}{lcl} \lambda _1 &{}if &{}A<B\\ \text {min}(\lambda _1,\lambda _2) &{}if &{}A\ge B\\ \end{array} \right. \end{aligned},$$

    (13)

    where

    $$\begin{aligned} A=\frac{s-N}{N}, \qquad B=\sqrt{N(\text{log}_2N)^3} \end{aligned},$$

    (14)

    and $s=\sum \limits _{i=1}^{N} Y_{j,k}^2$ is the sum of the squared wavelet coefficients [73].

  • Minimax [21, 37]. This is a level-independent method based on a Minimax statistical principle. The denoised signal may be assimilated to the estimator of the unknown regression function, so that Minimax realizes the minimum of the maximum MSE obtained for a given set of functions and, consequently, optimal threshold $\lambda$ is determined by

    $$\begin{aligned} \lambda =\left\{ \begin{array}{lcl} 0.3936+0.1829\cdot log_2N &{}if &{}N>32 \\ 0 &{}if &{}N\le 32\\ \end{array} \right. \end{aligned}.$$

    (15)

• (iii) Signal reconstruction: finally, the decomposed signal ${\widehat{x}}[k]$ is reconstructed by applying the inverse Wavelet Transform

  $$\begin{aligned} {\widehat{x}}_i=\sum \limits _{j=1}^{J}\sum \limits _{k=1}^{K}{\widehat{X}}_{j,k} W^*_{i,(j,k)}=\sum \limits _{j=1}^{J}\sum \limits _{k=1}^{K} DY_{j,k} W^*_{i,(j,k)} \end{aligned},$$

  (16)

  where $W^*_{i,(j,k)}$ is the inverse wavelet transform operator, defined as

  $$\begin{aligned} W^*_{i,(j,k)}=2^{j/2}\psi (2^{j}i-k) \end{aligned}.$$

  (17)

2.3 Singular value decomposition-based denoising

The SVD is a numerical method that factorizes a matrix ${\mathbf {Y}}\in {\mathbb {R}}^{m \times n},$ of rank r as [76]

$$\begin{aligned} {\mathbf {Y}}= {\mathbf {U}}{\mathbf {S}}{\mathbf {V}}^T \end{aligned},$$

(18)

in which ${\mathbf {U}}\in {\mathbb {R}}^{m \times m},$ and ${\mathbf {V}} \in {\mathbb {R}}^{n \times n},$ are orthogonal matrices, and ${\mathbf {S}} \in {\mathbb {R}}^{m \times n},$ is a matrix of the form

$$\begin{aligned} {\mathbf {S}}= \begin{bmatrix} {\mathbf {S}}_r &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {0}} \end{bmatrix} \end{aligned},$$

(19)

where

$$\begin{aligned} {\mathbf {S}}_r=diag\{s_1,s_2,\dots ,s_r\} \end{aligned},$$

(20)

and $s_1>s_2>\dots ,s_r>0$ are the singular values. The SVD has been extensively applied in inverse engineering methods, as, indicatively, subspace identification [38, 70, 75] and spectral estimation [49,50,51,52,53,54,55]. In implementing it as a denoising tool, notice that Eq. 18 can be expanded as

$$\begin{aligned} {\mathbf {Y}}= \begin{bmatrix} {\mathbf {U}}_r&{\mathbf {U}}_{m-r} \end{bmatrix} \begin{bmatrix} {\mathbf {S}}_r &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {0}} \end{bmatrix} \begin{bmatrix} {\mathbf {V}}_r^T \\ {\mathbf {U}}_{n-r}^T \end{bmatrix}={\mathbf {U}}_r{\mathbf {S}}_r{\mathbf {V}}_r^T \end{aligned},$$

(21)

admitting thus a truncated, reduced representation. In view of Eq. 21, a denoising procedure can then be drafted as follows [78]:

(i):

Construction of a Hankel matrix: The noise-corrupted data is assembled into a Hankel matrix as [34]

$$\begin{aligned} {\mathbf {Y}}_{m,n} = \begin{bmatrix} y_1 &{} y_2 &{} \ldots &{} y_n \\ y_2 &{} y_3 &{} \ldots &{} y_{n+1} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ y_m &{} y_{m+1} &{} \ldots &{} y_N \\ \end{bmatrix} \end{aligned},$$

(22)

where m, n are user parameters [17]. Then, from Eqs. 1-2

$$\begin{aligned} {\mathbf {Y}}_{m,n} = {\mathbf {X}}_{m,n}+{\mathbf {N}}_{m,n} \end{aligned},$$

(23)

with matrices ${\mathbf {X}}_{m,n}$ and ${\mathbf {N}}_{m,n}$ retaining a similar Hankel structure and defining the signal and noise subspaces, respectively.

(ii):

Data matrix decomposition: The application of SVD to data matrix ${\mathbf {Y}}_{m,n}$ yields

$$\begin{aligned} {\mathbf {Y}}={\mathbf {U}}{\mathbf {S}}{\mathbf {V}}^T=\sum _{i=1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T \end{aligned},$$

(24)

expressing ${\mathbf {Y}}_{m,n}$ as the superposition of m outer product sub-matrices $\mathbf{Y }_i=s_i \mathbf{u }_i \mathbf{v }^T_i$, with each one of them corresponding to a singular component of the noise-corrupted signal.

(iii):

Signal denoising: in extracting noise-free counterpart ${\mathbf {X}}_{m,n}$, one seeks a pattern of the form [58, 77]

$$\begin{aligned} s_1>s_2>\dots >s_\ell \gg s_{\ell +1}=s_{\ell +2}=\dots =s_m\approx 0 \end{aligned},$$

(25)

in the singular values. Accordingly, from Eq. 24

$$\begin{aligned} {\mathbf {Y}}=\sum _{i=1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T=\sum _{i=1}^\ell s_i{\mathbf {u}}_i{\mathbf {v}}_i^T+\sum _{i=\ell +1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T={\mathbf {X}}_{m,n}+{\mathbf {N}}_{m,n} \end{aligned},$$

(26)

e.g. the noise-free information is approximated by the superposition of the largest $\ell$ singular components of ${\mathbf {Y}}_{m,n}$.

(iv):

Signal reconstruction: Noise-free data $\hat{{\mathbf {x}}}$ is recovered by applying diagonal averaging to matrix ${\mathbf {X}}_{m,n}$ [66].

Fig. 1

Sketch of the 10-DOF shear frame

Table 1 Structural parameters of the shear frame

Table 2 Modal parameters of the shear frame

The aforementioned steps render the SVD as a particularly attractive denoising method, due to the robustness of the underlying numerical procedure and the small number of user-defined parameters (m and $\ell$ for a given N). In fact, the crucial step pertains in estimating $\ell$ for successfully isolating the noise-free part of the signal. In this regard, many different approaches have been reported in the literature and the reader is referred to Chen and Zhang [16], Jing et al. [35] and Zhao and Ye [76] for further details.

3 Numerical assessment

3.1 Benchmark structure and simulation

The denoising methods outlined in the previous section are initially assessed and compared via a numerical study that considers the single-bay, ten-story shear frame of Fig. 1, with structural parameters listed in Table 1. Table 2 further displays the vibration modes of the frame, in terms of undamped natural frequencies and damping ratios (uncoupled modal damping is assumed).

Simulation is succeeded by first transforming the structural equations of motion into state-space and accordingly discretizing the system using the zero-order hold scheme, at a sampling rate $F_s=200$ Hz. Two types of base-excitation with diverse frequency content are implemented. The first, shown in Fig. 2, is a non-stationary input that pertains to the 1986 Kalamata earthquake (0.24 g peak ground acceleration, 5.9 magnitude, 22 km focal depth [44]), while the second, shown in Fig. 3, is a stationary, zero-mean Gaussian white noise of variance 0.01 g, corresponding to ambient vibration [23]. For the subsequent denoising tasks, the availability of structural vibration response at the first and the last story of the frame is initially assumed, in the form of absolute acceleration. Indicatively, Figs. 4-5 display the response signals, noise-corrupted at 25% N/S ratio, for the excitations of Figs. 2-3, respectively.

Fig. 2

Non-stationary excitation: 1986 Kalamata earthquake

Fig. 3

Stationary excitation: zero-mean Gaussian white noise

Fig. 4

Vibration acceleration responses for the 1986 Kalamata earthquake, noise-corrupted at 25% N/S ratio

Fig. 5

Vibration acceleration responses for the ambient excitation, noise-corrupted at 25% N/S ratio

In evaluating the performance of the denoising techniques, the percentage root mean square difference (PrmsD) [37, 65], defined as

$$\begin{aligned} \text {PrmsD}=100\sqrt{\frac{\sum \limits _{k=1}^{N}\big (x[k]-{\widehat{x}}[k]\big )^2}{\sum \limits _{k=1}^{N} \big (x[k]-\mu _x\big )^2}} \end{aligned},$$

(27)

where x[k] is the simulated structural vibration response, of mean value $\mu _x$, ${\hat{x}}[k]$ is the denoised vibration response and N the length of the data, is used as a measure of effectiveness, among other quality measures, such as the cross-correlation value [2], the mean square error (MSE) [65] and the signal-to-noise ratio (SNR) [32].

3.2 DWT-based denoising

Table 3 Percentage PrmsD index between the noise-free and the denoised signal, for different mother wavelets and thresholding rules ($10\%$ N/S ratio, decomposition level 3)

The effectiveness of the DWT-based denoising depends on several pre-defined parameters (user choices), including the mother wavelet typology, the thresholding rule, the type of thresholding and the wavelet decomposition level. In assessing these, for both the non-stationary and the stationary excitation cases, the N/S ratio is set equal to 10% and the wavelet decomposition level is set at 3. The Symlet, Coiflet, Daubechies, Biorthogonal, Reverse Biorthogonal and Discrete Meyer mother wavelets are considered, combined with the thresholding rules of Sect. 2.2, that is, the Heuristic SURE, the Sqtwolog, the Minimax and the Rigorous SURE ones. In addition, different numbers of oscillations are considered (indicated with a number next to the wavelet mother name) for each mother wavelet. As the performed analysis does not reveal substantial differences between the structural responses of the first and the last story, only results concerning the latter signal are reported for the rest of the benchmark study.

Table 4 Percentage PrmsD index between the noise-free and the denoised signal for different decomposition levels and thresholding types ($10\%$ N/S ratio)

The PrmsD index for the explored combinations of mother wavelets and thresholding rules is illustrated in Table 3. It is observed that in the seismic excitation case the results do not seem to be significantly affected by the number of oscillations characterizing the considered mother wavelet. On the contrary, for the ambient excitation a considerable dispersion of the index occurs. For the non-stationary signal, the lowest PrmsD value is attained for the Smylet-2 mother wavelet with the Heursure thresholding rule, whereas for the stationary signal the lowest value of the index is produced by the combination of Coiflet-4 mother wavelet with the Minimax rule.

By fixing these parameters, the decomposition level minimizing the PrmsD index is then examined, for both soft and hard thresholding, and the results are printed in Table 4. In the non-stationary case and for $10\%$ N/S ratio, a hard thresholding at level 2 is suggested, while for the stationary case and the same N/S ratio the optimal setting is provided by a hard thresholding at level 3. In light of the derived results, it clearly emerges how the choice of the mother wavelet type as well as the selection of the decomposition level, play a crucial role for the efficacy of the DWT-based denoising approach. The mother wavelet should be chosen such that it can be able to well characterize the processed signal and this may be established according to the correlation between the wavelet and the signal itself. Additionally, if the noise floor is hardly detectable, more decomposition levels might be required, in order to reconstruct even the finest details of the signal; consequently, in some cases, it may be useful to work with multiple decomposition levels. Since the optimal decomposition level of the wavelet may be different, depending on the level of noise affecting the data, within the present analysis, the optimal decomposition level will be each time re-adjusted, according to the considered N/S ratio, to get the best denoising estimates.

Figure 6 shows the denoised seismic response signal for $25\%$ N/S ratio, for the Smylet2 mother wavelet and the Heursure thresholding at level 2. The benefits of the denoising process are quite tangible and demonstrate that the noise is largely successfully removed and the original data set is almost completely reconstructed. On the contrary, the same results are not drawn for the processing of ambient vibration response signals: as illustrated in Fig. 7, for denoising via the Coiflet4 wavelet with Minimax thresholding at level 3, the error in terms of difference between the original clean signal and the denoised one is clearly higher than for the non-stationary case. These findings indicate that the effectiveness of the DWT-based denoising depends on the nature of the source signal. Indeed, denoising of non-stationary response signals can be very effective, whereas the same procedure exhibits visible limitations when applied to stationary records.

Fig. 6

DWT-based denoising of $\ddot{x}_{10}[k]$ under seismic input ($25\%$ N/S ratio)

Fig. 7

DWT-based denoising of $\ddot{x}_{10}[k]$ under ambient vibration ($25\%$ N/S ratio)

3.3 SVD-based denoising

Turning our attention to the SVD, the critical parameters of the method pertain to the size of the Hankel matrix (parameter m), as well as to the truncation of the sum of Eq. 26 (parameter $\ell$). Table 5 displays the percentage PrmsD index for the denoising of a 10% noise-corrupted acceleration signal, for various choices of the size and truncation parameters, where the latter is calculated as the subset of singular values that are lower than a certain percentage with respect to the largest one. It can be deduced that both m and $\ell$ affect the quality of the denoising process, while the lowest index values are succeeded for $m=300$ and $\ell =5\%$. Adopting these parameter values, Figs. 8 and 9 illustrate the denoised structural responses for the non-stationary and stationary cases, respectively, for the 25% noise-corrupted signals. The behaviour seems to be comparable to the previously treated DWT-based denoising, exhibiting a slightly better performance in the non-stationary case, where the error is somewhat smaller than for the DWT one.

Table 5 Percentage PrmsD index between the noise-free and the denoised signal for different values of m and $\ell$ (SVD, $10\%$ N/S ratio)

Fig. 8

SVD-based denoising of $\ddot{x}_{10}[k]$ under seismic input ($25\%$ N/S ratio)

Fig. 9

SVD-based denoising of $\ddot{x}_{10}[k]$ under ambient vibration ($25\%$ N/S ratio)

3.4 Time and frequency domain comparison

Table 6 and Fig. 10 show the performance of the two denoising methods for various N/S ratios, in terms of the PrmsD index, for both the non-stationary and stationary excitations. Focusing first on the seismic excitation, the DWT-based approach seems to outperform the SVD one for N/S ratios lower than 20%, values that are more often encountered in real applications. However, for higher N/S ratios, the SVD appears to be more robust, since it is not so affected by the level of noise considered on the measurements, and the PrmsD index “converges” asymptotically to 0.60-0.70%. On the contrary, for the ambient excitation case, the SVD-based approach is clearly more effective.

Table 6 Percentage PrmsD index for different signal types and N/S ratios

Fig. 10

Effectiveness evaluation of DWT- and SVD-based denoising techniques for increasing N/S ratios and different signal typologies: (left) earthquake excitation; (right) ambient vibration. Indicated trends come from a polynomial fit of degree 2

Proceeding further with the assessment in the time domain, the effects of denoising on the signal’s main peak value are examined, since this value plays oftentimes a key role in the design, or assessment phase. To this end, the vibration acceleration peak values resulting from the two denoising methods, as well as their percentage alteration with respect to the noise-corrupted peak ($\Delta _{na}$), and the peak value ($\Delta _{nd}$) of the original signal are summarized in Table 7. From the obtained results, it can be deduced that the denoising process always leads to a peak reduction; this feature is attributed to the fact that the artificial noise acts additively. In the non-stationary case, the SVD-based denoising seems to lead to a flattening effect on the main signal peak, in comparison to the DWT-based denoising, while the stationary case is generally associated with greater percentage variation values $\Delta _{nd}$. However, this main peak analysis is more pertinent to the non-stationary signals, for which $\Delta _{nd}$ presents low values, at least for common N/S ratios. In addition to having a greater practical relevance for vibration assessment purposes, the analysis conducted on non-stationary response signals turns out to be more significant than that performed on stationary signals, since the main acceleration peak clearly emerges and it is immediate to be detected, configuring itself as a characteristic feature describing this kind of signal.

Table 7 Peak acceleration values of the DWT and SVD denoised signals, and their variation with respect to noise-affected signal peak $\Delta _{na}$ and to numerically determined signal peak $\Delta _{nd}$, for different typologies of signals (i.e. non-stationary (seismic) signal and stationary (white noise) signal) and N/S ratios

Fig. 11

Power spectrum of the original and the denoised signals (seismic excitation, 10% N/S ratio)

Table 8 Original and identified natural frequencies (peak-picking on Welch’s spectral estimates)

Figures 11, 12, 13 and 14 display the power spectrum (Welch’s method, $N_{_{FFT}}=1024$, 50% overlap), of the original and the denoised records for two N/S ratios, e.g. 10% and 25%, respectively. For both the excitation cases, the quality of the denoising methods within the effective band ([0, 10] Hz) is quite comparable; a small discrepancy is observed in the SVD-based spectra, in a small frequency band before 10 Hz. This suggests that the 5% threshold in the execution of the SVD causes distortion of important information. On the contrary, the DWT-based denoising seems to increase the energy of the signal after approximately 10 Hz.

Fig. 12

Power spectrum of the original and the denoised signals (ambient excitation, 10% N/S ratio)

Fig. 13

Power spectrum of the original and the denoised signals (seismic excitation, 25% N/S ratio)

Fig. 14

Power spectrum of the original and the denoised signals (ambient excitation, 25% N/S ratio)

Finally, Table 8 illustrates the “denoised”, undamped natural frequencies, calculated by applying the peak-picking method to the Welch spectral estimates. The estimates ${\hat{f}}_n$ in the third column of the table correspond to nonparametric estimates from the power spectrum of the noise--free vibration acceleration signal. Expectedly, the results for the ambient excitation case are better, succeeding in identifying more frequencies than in the non-stationary case, for both N/S ratios. Rather similar results apply to the SVD-based denoising, showing that it is more the signal typology that may affect the estimates, rather than the employed denoising method.

Fig. 15

The UHPFRC railway bridge in Sempachersee, Switzerland

4 Operational assessment

The examined methods are now applied to the denoising problem of real vibration acceleration records, obtained during a long-term monitoring campaign of an Ultra-High Performance Fiber Reinforced Concrete (UHPFRC) railway bridge in Sempachersee, Switzerland (canton Lucerne). The bridge (Fig. 15a), which is located in “Linie 500”, at km 78.330 along the Sursee-Sempach route, consists of two UHPFRC plates with a ribbed profile and a 6 meter span length. The height of the ribs varies between 0.45 m at the supports and 0.5 m at midspan. These plates were then connected on-site by a mortar. Since there is no structure of similar traits to compare its performance to, a monitoring campaign, covering a period from December 2018 up to September 2020, is initiated to measure the long-term behaviour of the structure (and material), which would help for future constructions using the same material. An additional benefit comes from the continuous tracking of several performance indicators which could help identifying any unforeseen changes and thus aid to plan inspection and intervention programmes.

Fig. 16

Selected vibration acceleration response records from the railway bridge (position A4, respectively)

Fig. 17

Spectrograms of the records of Fig. 16

The complete instrumentation network is shown in Fig. 15b and consists of low-cost MEMS accelerometers (Ax), laser distometers (not visible), tiltmeters (Tx), strain gauges (SGx), as well as a temperature and a humidity sensor (not visible) for tracking the environmental effects on the bridge. The associated data acquisition process operates in triggered mode, every time a train crosses the bridge, and stores 22 s (including 2 s of pre-triggered samples) of bridge vibration response at a sampling rate $F_s=2000$ Hz. Two representative such records are displayed in Fig. 16 and herein used for assessing the two denoising methods. Both records correspond to crossings of typical revenue trains with five wagons. It is noted that, due to confidentiality reasons, axes scales are suppressed. Figure 17 illustrates the spectrograms of the associated time series (Welch’s method with $N_{_{FFT}}=512$ and 50% overlap). Five frequency zones are observed, corresponding to the passage of the associated wagons, confirming the transient nature of the vibration records.

Fig. 18

DWT-based denoising of the records of Fig. 16

In applying the DWT for the denoising of the signals of Fig. 16, all parameters from the numerical case study remain unaltered, except from the number of oscillations in the mother wavelet. In more detail, the Smylet mother wavelet is herein adopted, combined with the Heursure thresholding and a decomposition level 2, whereas the number of oscillations is increased from 2 to 20, aiming at achieving a better clarification of the real acceleration signal in the time domain and, at the same time, for obtaining a response spectrum coherent with the frequency content of the original signal. The results are shown in Fig. 18, in a time window during the crossing of the first and second wagons, respectively, for $y_1[k]$ and $y_2[k]$. All high frequency content of the signals, usually associated with complex vehicle-rail interaction effects for this class of problems, apart from sensor and instrumentation noise, has been greatly suppressed, rendering the denoised signals ${\hat{x}}_1[k]$ and ${\hat{x}}_2[k]$ representative of the structural dynamics of the infrastructure (rail, ballast and bridge).

Fig. 19

SVD-based denoising of the records of Fig. 16

Fig. 20

DWT- vs. SVD-based denoising of the records of Fig. 16

Similar results are attained for the SVD-based denoising, as it can be inspected in Fig. 19, for $m=300$ and $\ell =30\%$. The truncation of the signals using such a high value for $\ell$ is deemed necessary, due to the presence of considerable “local” high-frequency dynamics in the original records, as explained before. Indeed, lower values of $\ell$ maintain these local oscillating phenomena to a great extend. In appreciating the effects of SVD-based denoising, Fig. 20 compares the outcomes of the denoising process for both methods. Although a good agreement emerges between the two treated techniques in denoising the examined real acceleration signals, the SVD-based approach seems to produce higher overall values, compared to the DWT-based one, for the first record, whereas the opposite holds for the second record. This visual evidence is confirmed by calculating the percentage RMS and peak values reduction, which are presented in Table 9. Indeed, while for $y_1[k]$ the figures between DWT and SVD are comparable, the reduction after the denoising of $y_2[k]$ is almost double for the SVD. Taking into account the fact that the two records have been acquired from similar operational conditions, e.g. crossing of similar trains (wagon number) at comparable velocities (the spectral peaks in Fig. 17 are located at the same areas in both spectrograms), the different results in the denoising process are rather attributed to an alteration of the instrumentation noise. This, in turn, might be explained by many factors, with the environmental effects being the most prominent. In any case, this typical performance of long-term monitoring installations in infrastructures emphasizes on the necessity of adopting appropriate denoising measures before any post-processing and estimation operations.

Table 9 Percentage reduction of overall values (RMS, Peak) for the denoised bridge records

Fig. 21

Spectrograms of the bridge’s denoised records

Finally, Fig. 21 shows the spectrograms of the denoised signals (Welch’s method with $N_{_{FFT}}=512$ and 50% overlap). Compared to Fig. 16, the spectrograms of the acquired signals, the reduction of the signals’ energy above approximately 200 Hz is evident; the denoising process leaves only frequency zones of small energy. Interestingly, the effective frequency band of the SVD-based denoised signals is narrower, with respect to the one of the DWT-based records. This is an indication that threshold parameter $\ell$ might be overestimated. In any case, the frequency band of the denoised acceleration records is now concentrated in regions where the structural dynamics of the infrastructure usually dominate.

As final considerations, the following additional remarks may be supplied, on behalf of the interested reader, about the investigated operational case study:

• concerning the unscaled depictions in Figs. 16, 18, 19 and 20, the acquired data were subject to certain restrictions posed from the infrastructure manager. That being noted, first, the adopted representations aim to focus on the appreciation of the signal features, and the achieved denoising effects. It is believed that the interpretation of the results is then accomplished in a way that indeed quantifies the efficacy of the assessed denoising methods.

• the denoising process has been applied to all the data acquired from the various deployed acceleration sensors. Two most representing and characteristic signals have been selected, as reference samples, and for relative comparison. Indeed, it was decided not to include further data, since: (i) the results turn out very similar, due to the similarity of the sensors and the recorded data; and (ii) to avoid lengthening the presentation, while likely distracting from the main message, of achieved effective denoising, as already condensed and revealed, for the two adopted signals.

5 Conclusions

In this paper, the DWT and the SVD as denoising methods for civil engineering applications have been investigated and assessed, in both a numerical and an operational setting.

In the former case, some critical remarks can be summarized as follows:

• In qualifying the DWT parameters, it is found that in the non-stationary (seismic) excitation case, the combination of Smylet having two oscillations in its mother wavelet and Heursure hard thresholding type at decomposition level 2, renders the smallest PrmsD value. For ambient records, an optimal configuration is succeeded by the combination of Coiflet having four oscillations in its mother wavelet and Minimax hard thresholding type at decomposition level 3.

• Although the tuning of the SVD requires only two parameters, the size of the Hankel matrix and the truncation of the singular values, both affect significantly the outcome. However, the SVD-based appears more robust for various N/S ratios and, generally, provides a better approximation of the original response signal. The combined use of both techniques might, yet, be the optimal strategy.

• In examining the effects of denoising on the overall values (RMS, peak), it is found that non-stationary records lead to better results than stationary ones. Moreover, the SVD-based denoising seems to flatten out the peaks less than the DWT.

• The effectiveness of denoising has also been assessed in the frequency domain. Quite satisfactory results are obtained for non-stationary records, in particular those deriving from the SVD-based approach, whereas for ambient vibration the power spectrum does not seem to be so coherent with the original one, especially at the frequency bands where the noise affects the signal. This may constitute a strong motivation for further research on this latter specific class of structural vibration response.

Similar outcomes are drawn from applying the methods to real operational data. In particular, the benefits of denoising in purifying the vibration acceleration response signals of the considered railway bridge are evident in the time domain, where the outcomes of the DWT- and SVD-based denoising seem to be in comparable agreement to each other. Expectedly, the overall values of the denoised signals appear to be flatter. The frequency domain representation of the denoised records preserves the useful information in the band of interest, and suppresses high frequency components, which are attributed either to instrumentation noise, or to dynamic phenomena that are unrelated to the examined infrastructure (e.g., vehicle-rail interaction).

The encouraging results of the study promote the implementation of denoising as an integrated element of monitoring campaigns in civil and structural engineering, for the purposes of structural identification and health monitoring [49,50,51,52,53,54,55]. The examined methods can be appropriately applied to other data types as well, including displacement and strain gauge records, and may be effectively combined with model-based approaches, such as heterogeneous data fusion [27]. As the current trend in monitoring qualifies the implementation of low-cost, large numbers of sensors, instead of a few expensive ones, proper and careful denoising may assist in extracting accurate information from large-scale infrastructures.