Skip to main content

On the denoising of structural vibration response records from low-cost sensors: a critical comparison and assessment

Abstract

Due to the increasing quest of adopting low-cost sensors in structural health monitoring (SHM) processes, which may lead to detecting signals contaminated by significant levels of noise, the need to devise appropriate and effective denoising strategies, at the post-processing stage, is becoming more and more essential. Among several approaches proposed in the literature, it has been demonstrated that the employment of discrete wavelet transform (DWT) as a multi-rate filter bank, as well as the use of singular value decomposition (SVD), may result to be quite effective in signal denoising within various research fields, as biological, acoustic and mechanical. Here, DWT- and SVD-based denoising techniques are first independently reconsidered and reimplemented, aiming at exploring their optimal calibration in purifying noise-corrupted vibration response signals encountered in civil engineering applications. Then, a systematic performance evaluation is provided within a comparative framework, developed at an increasing level of noise affecting the measurements, in terms of noise-to-signal (N/S) ratio. In the study, two specific classes of synthetic response signals are first considered, namely earthquake and ambient vibration signals, since they may be assumed as representative of more general non-stationary and stationary signal typologies, respectively. To achieve a complete description of the clarified signal, strengths and weaknesses of the two denoising approaches are explored, in both time and frequency domains. The results prove the effectiveness of the analyzed implementations, especially in purifying seismic response signals, while some limitations may arise concerning the treatment of ambient vibration signals, in particular for the DWT-based denoising technique. Finally, a real case study is analyzed, where both denoising approaches are adapted and employed for clarifying acceleration signals detected on a modern short-span railway bridge, with rather satisfactory results, for both techniques.

Introduction

Structural Health Monitoring (SHM) refers to the process of a continuous assessment of the current health conditions of a structural system, aiming at improving its integrity, by detecting potential manifestations of damage, before this may reach a critical state, possibly detrimental for structural safety. Dealing with the structural engineering context in general terms, virtuous examples in which SHM procedures have been successfully implemented, on different typologies of structures, may be found in the literature (e.g. Capellari et al. [10], Chatzi and Smyth [12], Ferrari et al. [26,27,28,29], Koo et al. [41], Lee et al. [42], Roberts and Dodson [63], and cited references). It is clear that the acquisition stage of signals, from which the current structural conditions may be detected, constitutes a crucial phase for the global success of such an analysis.

Referring to the civil engineering field, in particular, the most common typologies of sensors employed for the detection of response signals may be considered as accelerometers and displacement sensors, as the kind of response data that can be recorded may be good enough to describe the current structural health conditions. However, the raising need to adopt a low-cost and easy-to-use monitoring instrumentation, toward recording the structural dynamic response under live external loads, may often lead to measurements affected by sensible amounts of noise. This spurious signal contamination, superposed upon the useful structural response signal, may lead to spoil the monitoring information content and alter the prediction of the structural response, leading to unreliable SHM implementations. Consequently, noise reduction from acquired response signals constitutes a crucial issue in an efficient SHM process of civil structures. This may hold true, specifically, for low-cost sensors, especially when adopting automated monitoring equipment and systems, markedly after long-term operation, in terms of potential implications about a higher endemic presence of spurious noise, to be cleaned by appropriate signal processing approaches, as enquired in the present investigation.

The procedure by which it becomes possible to reconstruct a certain source signal, starting from a recorded, noise-corrupted one, removing its noisy part, without losing its useful information, may be referred to as denoising Buades and Coll [9], Chen and Bui [15], Portilla and Strela [56]. Two main issues have to be accounted for in dealing with a denoising procedure, namely: (a) how the initial data may be affected by the added noise; (b) how the reconstructed signal, out of the denoising procedure, shall be able to correctly reproduce the original (truthful) signal, preserving its fundamental features. Thus, the crucial matter of each denoising process lies in removing most of the unwanted noise, without losing the useful part of the signal containing the true monitoring information.

Several computational techniques have been proposed in the literature to address the denoising of signals, some of which are currently in development. The most basic and traditional way to remove the noise affecting the signals is represented by the employment of band-pass filters with cut-off frequencies, i.e. high-pass or low-pass filters, which allow to remove all the frequency contents greater or lower than a certain frequency value, respectively. Moving average filters, as well as Gaussian filters, can be considered as typical examples belonging to such a category. However, although the application of such filters may be useful when the noise is located within a precise frequency band, which is different from the frequency band in which the signal shall lay, in most cases, when the noise displays a similar frequency content as that of the signal to be analyzed, they cannot be effective, since even much of the useful signal may be lost.

This also constitutes a main drawback in the use of the Fourier Transform toward denoising [22]. In other words, such methods act in a global sense, since they process the signal regardless of whether the noise displays a uniform frequency distribution or not. However, in real cases, the noise distribution may be far from being uniform and it may be desirable to apply a “localized” form of denoising. In the past decades, this has led to searching for different methodological approaches and alternative filtering techniques. Arezki and Berkani [5] and Kam et al. [36] proposed the application of adaptive filters to remove Gaussian white noise and impulse noise from signals. Within their works, an iterative procedure was set up for minimizing in real time the error between the original signal and the denoised signal [45]. To suppress impulse noise, Veerakumar et al. [72] introduced a new algorithmic approach based on the combination of fuzzy logic and an asymmetric trimmed median filter, whereas in Premchaiswadi et al. [57] a kFill algorithm was combined with a median filter for reducing the impulse noise that could occur on images (denoising methods may also be applied on images, as an image may be interpreted as a two-dimensional signal, Ergen [22]).

A further powerful methodology for separating noise out of corrupted data involves the application of a discrete wavelet transform (DWT). In particular, in Dohono [20], a first DWT-based approach for denoising one-dimensional signals was provided. Afterwards, Chang et al. [11] introduced an innovative adaptive Wavelet thresholding for image denoising and compression, called BaeyShrink method. Furthermore, the use of singular value decomposition (SVD) for denoising purposes has also attracted considerable interest, as demonstrated e.g. in Konstantinides et al. [39] and Konstantinides and Yao [40], where a new filtering and noise estimation technique was introduced, known as block-based singular value decomposition (BSVD) filtering. Finally, a possible alternative approach in enhancing the quality of noise-affected signals may concern the application of a Kalman filter (KF) [13, 14], Ravizza et al. [59], and works cited therein). For instance, in Ravizza et al. [59], a KF has been successfully employed within a Heterogeneous Data Fusion procedure, to correct noisy displacement measurements, by enhancing them through a few reliable acceleration signals, toward cleaning structural acquisition extraction and modal dynamic identification.

In this paper, two of the above-mentioned approaches are systematically reconsidered and developed, in tackling first a controlled denoising problem set on reference structural response signals. In particular, the development of a DWT procedure, as a multi-rate filter bank, as well as the implementation of a SVD technique are extensively inspected, in their employment toward denoising structural response signals. Here, the two approaches are implemented and assessed on noise-corrupted structural vibration response signals that may be typical of practical applications belonging to the civil engineering context.

In signal processing, DWT is commonly applied for many other purposes, in addition to signal denoising. For instance, its employment shall be rather useful in detecting trends, breakdown points, discontinuities in higher derivatives and self-similarity of signals [69]. Moreover, as a denoising technique, it has been performed on signals of a different nature, i.e. gravity and magnetic signals [24], biological signals [3, 6], such as electroencephalograms (EEG) or electrocardiograms (ECG), but also on acoustic signals [25] and pressure signals [68]. However, its application in denoising structural signals of the civil engineering field does not seem to have been deeply inspected yet. Similarly, also SVD has found applications in digital signal processing as a technique for noise reduction. In particular, its effectiveness has already been proven dealing with audio signals (e.g. in Baravdish et al. [7], where it is combined with a non-linear PDE method), or biomedical signals [67], or even for radar target recognition of electromagnetic signals [43], but also for improving the quality of images [33]. Moreover, in the civil engineering domain, it has been exploited for many purposes, including those of modal dynamic identification (e.g. in Pioldi et al. [49,50,51]) and of structural damage detection at an early stage of development [64].

In the present investigation, the performances of DWT- and SVD-based denoising techniques is first assessed on earthquake and ambient vibration synthetic response signals, and a critical comparison based on the effectiveness of such methods is provided, at variable added noise. The selection of these two types of response signals is also motivated by the fact that they can be considered as being well representative of two great families of signals. In particular, the earthquake excitation input is known as a typical non-stationary signal, whereas the ambient vibration signal is instead considered as a common stationary signal, since it may almost be constant in time and frequency. To inspect advantages and possible limitations of the mentioned denoising techniques, in relation to the typology of the processed signal, constitutes an important goal of this study. To synthetically generate numerical response signals, the seismic and ambient vibration input signals are separately examined and applied on a reference structure, namely a one-bay ten-story shear-type frame, used as a benchmark mockup. In particular, the first- and last-floor acceleration responses are monitored, in the cleaning of the response signals (accelerations). Furthermore, to simulate the effect of the errors that may occur on the measurements during a real signal acquisition stage, within the analysis, several noise-to-signal (N/S) ratios are considered, for a superimposed noise signal affecting the data.

It is worth noting that the present investigation is first based on synthetic signals, namely pseudo-experimental signals numerically generated prior to the denoising analysis, for a crucial, necessary condition validation of the denoising procedures. Then, in the final part of the work, a real case study involving real signals, directly acquired “in situ” by appropriate sensors placed on a specific structure (a railway RC bridge), are eventually presented. Preliminary denoising simulations of the present efforts were presented in conference papers [60, 61].

It may be stressed out that, while the present focus is placed on dynamic measurements, the enquired techniques shall effectively be applied to static and quasi-static data as well. In fact, for data with such attributes, the denoising process shall become, in principle, more effective, than for dynamic measurements, where the noise sources may be propagated in time, through the associated structural dynamics, forming a more challenging denoising scenario and quest.

The main goals that this study aims to pursue are the following:

  • to explore which is the optimal calibration of a DWT-based denoising technique, for dealing with both seismic and ambient vibration response signals;

  • to examine the possibility to successfully exploit a SVD-based implementation, toward the clarification of the above-mentioned response signal typologies;

  • to provide a critical comparison of strengths and weaknesses of each denoising method, for exploring their effectiveness, at a controlled, increasing level of noise inserted on the source signals;

  • to recover the original signal in the time domain, with the target also of preserving its spectrum in the frequency domain, to achieve a comprehensive and more reliable reconstruction of the response signal, to be considered as rather truthful toward real monitoring purposes;

  • to inspect if the two explored denoising techniques may also become effective in real applications, for clarifying real experimentally recorded vibration response signals.

The paper is organized as follows. In Sect. 2, a brief theoretical framework on the adoption of DWT and SVD techniques for denoising purposes is provided, together with the needed strategies for their adaptation in the handling of the present civil engineering scenario. Section 3 presents the analysis procedure, through the description of the benchmark dynamical system, the process of generation of the simulated noise-affected signals and the obtained results. Here, particular importance is placed on the preliminary calibration of the denoising technique based on DWT, through the search for the optimal configuration of its characteristic parameters; additionally, the criterion through which a SVD approach may effectively be exploited toward denoising purposes is also explored. Within the same section, a performance assessment of the two studied approaches is presented, and a critical comparison is provided. Section 4 covers a real case study, in which both DWT- and SVD-based denoising methods are reinterpreted, in clarifying real (noise-affected) acceleration signals acquired from a railway bridge. Conclusions and global remarks are finally outlined in Sect. 5, as well as some possible future developments of the present study.

Theoretical framework

Problem statement

A typical denoising problem pertains at decomposing a noise-corrupted signal y[k], \(k=0,1,\dots ,N-1\), sampled at a rate \(F_s\) (Hz), as

$$\begin{aligned} y[k]=x[k]+n[k] \end{aligned},$$
(1)

where x[k] corresponds to the noise-free counterpart and n[k] is a zero-mean, stationary, Gaussian white noise process of variance \(\sigma _{nn}^2\) [1, 47]. Writing Eq. 1 in vector form [4],

$$\begin{aligned} {\mathbf {y}}={\mathbf {x}}+{\mathbf {n}} \end{aligned},$$
(2)

where

$$\begin{aligned} {\mathbf {y}}&= \begin{bmatrix}y[0]&y[1]&\dots&y[N-1]\end{bmatrix}^T,\end{aligned}$$
(3a)
$$\begin{aligned} {\mathbf {x}}&= \begin{bmatrix}x[0]&x[1]&\dots&x[N-1]\end{bmatrix}^T, \end{aligned}$$
(3b)
$$\begin{aligned} {\mathbf {n}}&= \begin{bmatrix}n[0]&n[1]&\dots&n[N-1]\end{bmatrix}^T, \end{aligned}$$
(3c)

one is thus interested in extracting an “optimal” approximation of \({\mathbf {x}}\), \(\hat{{\mathbf {x}}}\), as well as an estimate of the noise contamination variance, \({\hat{\sigma }}_{nn}^2\).

Signal y[k] is treated as a realization of a stochastic process that might be either stationary, or non-stationary. Under this setting, in the following the DWT- and SVD-based denoising approaches are first outlined, and then an extensive benchmark study is performed, using artificial and real-world structural vibration response data.

Discrete wavelet transform-based denoising

Wavelet Transform (WT) may be seen as the process through which a certain signal can be decomposed into its low and high frequency components, via the introduction of a set of orthonormal wavelet functions, constituting a wavelet basis, which originate from the mother wavelet by scaling and shifting operations through two parameters known as scale parameter j and shift parameter k. In particular, when scale parameter j is chosen as a power of two, a dyadic orthonormal wavelet transform is obtained [74].

The measure of frequency content similarity between the signal that has to be denoised and the selected wavelet function is expressed by the so-called wavelet coefficients, determined as a convolution of the signal and the scaled wavelet function, which may be considered as an expanded band-pass filter [62]. In particular, approximation coefficients are associated with low frequency components, whereas detail coefficients refer to high frequency components. In this way, WT is implemented with a reconstruction filter bank using an orthogonal wavelet family. Such an employment of WT as a filter bank for signal denoising purposes is also known as DWT [22].

A DWT denoising procedure may be conceptually summarized as follows:

  • (i) Decomposition of the original signal into the wavelet domain: digital signal y[k], \(k=0,...,N-1\), is transformed into the wavelet domain by [46]

    $$\begin{aligned} Y_{j,k}=\sum \limits _{i=0}^{N-1} y_i W_{i,(j,k)} \end{aligned},$$
    (4)

    where N is a power of two and \(W_{i,(j,k)}\) is the forward transform operator, defined as

    $$\begin{aligned} W_{i,(j,k)}=2^{-j/2}\psi (2^{-j}i-k) \end{aligned}.$$
    (5)

    Equation 5 indicates that the orthonormal basis functions are obtained through translations and dilations of a certain mother wavelet \(\psi \) by means of shift (or translation) parameter \(k\in {\mathbb {N}}\) and scale (or dilation) parameter \(j\in {\mathbb {N}}\), sampled along the dyadic sequence [8, 79]. Mother wavelet \(\psi \) is an oscillating function that may be selected among a set of pre-defined ones, including Symlet, Coiflet, Daubechies, Biorthogonal, Reverse Biorthogonal and Discrete Meyer, among others.

    Applying the DWT to Eq. 1 implies

    $$\begin{aligned} Y_{j,k}=X_{j,k}+N_{j,k} \end{aligned}.$$
    (6)

    Accordingly, by introducing denoising operator \(D:{\mathbb {R}}^+\rightarrow [0,1]\), the DWT of x[k] can be recovered by

    $$\begin{aligned} {\widehat{X}}_{j,k}=DY_{j,k} \end{aligned},$$
    (7)

    and the denoising problem aims at designing D for minimizing the Mean Squared Error (MSE) between \(X_{j,k}\)and \({\widehat{X}}_{j,k}\)

    $$\begin{aligned} r=E\{(X_{j,k}-DY_{j,k})^2\}=\frac{1}{N}\sum \limits _{i=1}^{N} (X_{j,k}-DY_{j,k})^2 \end{aligned},$$
    (8)

    where \(E\{\cdot \}\) denotes expected value.

  • (ii) Thresholding of the DWT coefficients: introducing a certain non-dimensional threshold value \(\lambda \), which may be a function of DWT decomposition level l (level-dependent threshold) or not (level-independent threshold) [19], the noise affecting the signals may be considerably reduced by scaling the wavelet coefficients smaller than \(\lambda \), so that only the coefficients greater than \(\lambda \) are considered to be associated to reliable data, whereas the ones below \(\lambda \) are set equal to zero.

    The idea is that the noise in the transformed domain tends to disperse over all the wavelet coefficients, so that a strategy in which values below a proper threshold are set to zero removes most of the noise, while preserving the information. In fact, switching into the wavelet domain, transformed signal \(Y_{j,k}\) will have many more non-zero coefficients than \(X_{j,k}\), due to the noise contribution. Thus, by selecting an appropriate threshold \(\lambda \), which allows to switch from wavelet coefficients \(Y_{j,k}\) to thresholded coefficients \(Y_{j,k}(D,\lambda )\), it may be possible to alleviate the noise contamination.

    Two different types of thresholding may be performed, i.e., hard thresholding and soft thresholding [18]. The former zeros out all the wavelet coefficients (in absolute value) smaller than \(\lambda \), by preserving the others unaltered:

    $$\begin{aligned} Y_{j,k}(D,\lambda )=\left\{ \begin{array}{lcl} Y_{j,k} &{}if &{}\left| Y_{j,k}\right| \ge \lambda \\ 0 &{}if &{}\left| Y_{j,k}\right| <\lambda \\ \end{array} \right. \end{aligned}.$$
    (9)

    In contrast, the latter reduces all wavelet coefficients (in absolute value) greater than \(\lambda \) by a quantity equal to the threshold itself:

    $$\begin{aligned} Y_{j,k}(D,\lambda )=\left\{ \begin{array}{lcl} \left| Y_{j,k}\right| -\lambda &{}if &{}\left| Y_{j,k}\right| \ge \lambda \\ 0 &{}if &{}\left| Y_{j,k}\right| <\lambda \\ \end{array} \right. \end{aligned}.$$
    (10)

    Since typically the noise mainly affects the high frequency components, it is common to apply the thresholding to the so-called detail coefficients, i.e. to the low-frequency band that usually contains the most important components of the signal.

    Both thresholding types are amenable to some drawbacks. Soft thresholding (also referred to as wavelet shrinkage), is characterized by bias when the preserved coefficients are large, while hard thresholding is “sensitive” to small changes in the signal and, consequently, suffers from stability issues. To overcome these problems, new hybrid solutions have been provided [30, 31]. In specific, Eqs. 9-10 indicate that the choice of an appropriate threshold \(\lambda \) value is crucial. Indeed, adopting a “small” threshold might lead to a reconstructed signal that will still be considerably affected by noise, whereas a “large” threshold might lead to the suppression of useful signal information. Although for a generic signal of length N the Universal Threshold \(\lambda \propto \sqrt{2lnN}\) has been proposed, a fair amount of studies examines other level-dependent or block-dependent approaches [18, 19, 37].

    In this study, thresholding rules that belong to the Donoho-Johnson class are considered, namely:

    • Visu shrink (Sqtwolog) [18]. This rule provides \(\lambda \) regardless of the DWT decomposition level in accordance to

      $$\begin{aligned} \lambda =\sigma \sqrt{2lnN} \end{aligned},$$
      (11)

      where \(\sigma \) is the standard deviation of the Gaussian white noise added to the clean signal.

    • SURE shrink (Rigorous SURE) [19]. In this case, a threshold level is assigned to each decomposition level of the Wavelet Transform, based on the principle of minimizing the Stein Unbiased Estimate of Risk (SURE)

      $$\begin{aligned} {\text{SURE}}(\lambda ,\mathbf{y })=N+\sum \limits _{i=0}^{N-1}[\text {min}(\left| Y_{j,k}\right| ,\lambda )]^2 -2(i:\left| Y_{j,k}\right| <\lambda ) \end{aligned}.$$
      (12)

      The underlying procedure is very versatile and suitable for denoising a wide range of functions.

    • Heuresure (Heuristic SURE) [37]. This rule is a combination of the two previous ones. If \(\lambda _1\) and \(\lambda _2\) are the threshold values obtained from Universal and Rigorous SURE methods, respectively, the Heuristic SURE rule assigns a threshold \(\lambda \) as

      $$\begin{aligned} \lambda =\left\{ \begin{array}{lcl} \lambda _1 &{}if &{}A<B\\ \text {min}(\lambda _1,\lambda _2) &{}if &{}A\ge B\\ \end{array} \right. \end{aligned},$$
      (13)

      where

      $$\begin{aligned} A=\frac{s-N}{N}, \qquad B=\sqrt{N(\text{log}_2N)^3} \end{aligned},$$
      (14)

      and \(s=\sum \limits _{i=1}^{N} Y_{j,k}^2\) is the sum of the squared wavelet coefficients [73].

    • Minimax [21, 37]. This is a level-independent method based on a Minimax statistical principle. The denoised signal may be assimilated to the estimator of the unknown regression function, so that Minimax realizes the minimum of the maximum MSE obtained for a given set of functions and, consequently, optimal threshold \(\lambda \) is determined by

      $$\begin{aligned} \lambda =\left\{ \begin{array}{lcl} 0.3936+0.1829\cdot log_2N &{}if &{}N>32 \\ 0 &{}if &{}N\le 32\\ \end{array} \right. \end{aligned}.$$
      (15)
  • (iii) Signal reconstruction: finally, the decomposed signal \({\widehat{x}}[k]\) is reconstructed by applying the inverse Wavelet Transform

    $$\begin{aligned} {\widehat{x}}_i=\sum \limits _{j=1}^{J}\sum \limits _{k=1}^{K}{\widehat{X}}_{j,k} W^*_{i,(j,k)}=\sum \limits _{j=1}^{J}\sum \limits _{k=1}^{K} DY_{j,k} W^*_{i,(j,k)} \end{aligned},$$
    (16)

    where \(W^*_{i,(j,k)}\) is the inverse wavelet transform operator, defined as

    $$\begin{aligned} W^*_{i,(j,k)}=2^{j/2}\psi (2^{j}i-k) \end{aligned}.$$
    (17)

Singular value decomposition-based denoising

The SVD is a numerical method that factorizes a matrix \({\mathbf {Y}}\in {\mathbb {R}}^{m \times n},\) of rank r as [76]

$$\begin{aligned} {\mathbf {Y}}= {\mathbf {U}}{\mathbf {S}}{\mathbf {V}}^T \end{aligned},$$
(18)

in which \({\mathbf {U}}\in {\mathbb {R}}^{m \times m},\) and \({\mathbf {V}} \in {\mathbb {R}}^{n \times n},\) are orthogonal matrices, and \({\mathbf {S}} \in {\mathbb {R}}^{m \times n},\) is a matrix of the form

$$\begin{aligned} {\mathbf {S}}= \begin{bmatrix} {\mathbf {S}}_r &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {0}} \end{bmatrix} \end{aligned},$$
(19)

where

$$\begin{aligned} {\mathbf {S}}_r=diag\{s_1,s_2,\dots ,s_r\} \end{aligned},$$
(20)

and \(s_1>s_2>\dots ,s_r>0\) are the singular values. The SVD has been extensively applied in inverse engineering methods, as, indicatively, subspace identification [38, 70, 75] and spectral estimation [49,50,51,52,53,54,55]. In implementing it as a denoising tool, notice that Eq. 18 can be expanded as

$$\begin{aligned} {\mathbf {Y}}= \begin{bmatrix} {\mathbf {U}}_r&{\mathbf {U}}_{m-r} \end{bmatrix} \begin{bmatrix} {\mathbf {S}}_r &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {0}} \end{bmatrix} \begin{bmatrix} {\mathbf {V}}_r^T \\ {\mathbf {U}}_{n-r}^T \end{bmatrix}={\mathbf {U}}_r{\mathbf {S}}_r{\mathbf {V}}_r^T \end{aligned},$$
(21)

admitting thus a truncated, reduced representation. In view of Eq. 21, a denoising procedure can then be drafted as follows [78]:

(i):

Construction of a Hankel matrix: The noise-corrupted data is assembled into a Hankel matrix as [34]

$$\begin{aligned} {\mathbf {Y}}_{m,n} = \begin{bmatrix} y_1 &{} y_2 &{} \ldots &{} y_n \\ y_2 &{} y_3 &{} \ldots &{} y_{n+1} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ y_m &{} y_{m+1} &{} \ldots &{} y_N \\ \end{bmatrix} \end{aligned},$$
(22)

where m, n are user parameters [17]. Then, from Eqs. 1-2

$$\begin{aligned} {\mathbf {Y}}_{m,n} = {\mathbf {X}}_{m,n}+{\mathbf {N}}_{m,n} \end{aligned},$$
(23)

with matrices \({\mathbf {X}}_{m,n}\) and \({\mathbf {N}}_{m,n}\) retaining a similar Hankel structure and defining the signal and noise subspaces, respectively.

(ii):

Data matrix decomposition: The application of SVD to data matrix \({\mathbf {Y}}_{m,n}\) yields

$$\begin{aligned} {\mathbf {Y}}={\mathbf {U}}{\mathbf {S}}{\mathbf {V}}^T=\sum _{i=1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T \end{aligned},$$
(24)

expressing \({\mathbf {Y}}_{m,n}\) as the superposition of m outer product sub-matrices \(\mathbf{Y }_i=s_i \mathbf{u }_i \mathbf{v }^T_i\), with each one of them corresponding to a singular component of the noise-corrupted signal.

(iii):

Signal denoising: in extracting noise-free counterpart \({\mathbf {X}}_{m,n}\), one seeks a pattern of the form [58, 77]

$$\begin{aligned} s_1>s_2>\dots >s_\ell \gg s_{\ell +1}=s_{\ell +2}=\dots =s_m\approx 0 \end{aligned},$$
(25)

in the singular values. Accordingly, from Eq. 24

$$\begin{aligned} {\mathbf {Y}}=\sum _{i=1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T=\sum _{i=1}^\ell s_i{\mathbf {u}}_i{\mathbf {v}}_i^T+\sum _{i=\ell +1}^m s_i{\mathbf {u}}_i{\mathbf {v}}_i^T={\mathbf {X}}_{m,n}+{\mathbf {N}}_{m,n} \end{aligned},$$
(26)

e.g. the noise-free information is approximated by the superposition of the largest \(\ell \) singular components of \({\mathbf {Y}}_{m,n}\).

(iv):

Signal reconstruction: Noise-free data \(\hat{{\mathbf {x}}}\) is recovered by applying diagonal averaging to matrix \({\mathbf {X}}_{m,n}\) [66].

Fig. 1
figure1

Sketch of the 10-DOF shear frame

Table 1 Structural parameters of the shear frame
Table 2 Modal parameters of the shear frame

The aforementioned steps render the SVD as a particularly attractive denoising method, due to the robustness of the underlying numerical procedure and the small number of user-defined parameters (m and \(\ell \) for a given N). In fact, the crucial step pertains in estimating \(\ell \) for successfully isolating the noise-free part of the signal. In this regard, many different approaches have been reported in the literature and the reader is referred to Chen and Zhang [16], Jing et al. [35] and Zhao and Ye [76] for further details.

Numerical assessment

Benchmark structure and simulation

The denoising methods outlined in the previous section are initially assessed and compared via a numerical study that considers the single-bay, ten-story shear frame of Fig. 1, with structural parameters listed in Table 1. Table 2 further displays the vibration modes of the frame, in terms of undamped natural frequencies and damping ratios (uncoupled modal damping is assumed).

Simulation is succeeded by first transforming the structural equations of motion into state-space and accordingly discretizing the system using the zero-order hold scheme, at a sampling rate \(F_s=200\) Hz. Two types of base-excitation with diverse frequency content are implemented. The first, shown in Fig. 2, is a non-stationary input that pertains to the 1986 Kalamata earthquake (0.24 g peak ground acceleration, 5.9 magnitude, 22 km focal depth [44]), while the second, shown in Fig. 3, is a stationary, zero-mean Gaussian white noise of variance 0.01 g, corresponding to ambient vibration [23]. For the subsequent denoising tasks, the availability of structural vibration response at the first and the last story of the frame is initially assumed, in the form of absolute acceleration. Indicatively, Figs. 4-5 display the response signals, noise-corrupted at 25% N/S ratio, for the excitations of Figs. 2-3, respectively.

Fig. 2
figure2

Non-stationary excitation: 1986 Kalamata earthquake

Fig. 3
figure3

Stationary excitation: zero-mean Gaussian white noise

Fig. 4
figure4

Vibration acceleration responses for the 1986 Kalamata earthquake, noise-corrupted at 25% N/S ratio

Fig. 5
figure5

Vibration acceleration responses for the ambient excitation, noise-corrupted at 25% N/S ratio

In evaluating the performance of the denoising techniques, the percentage root mean square difference (PrmsD) [37, 65], defined as

$$\begin{aligned} \text {PrmsD}=100\sqrt{\frac{\sum \limits _{k=1}^{N}\big (x[k]-{\widehat{x}}[k]\big )^2}{\sum \limits _{k=1}^{N} \big (x[k]-\mu _x\big )^2}} \end{aligned},$$
(27)

where x[k] is the simulated structural vibration response, of mean value \(\mu _x\), \({\hat{x}}[k]\) is the denoised vibration response and N the length of the data, is used as a measure of effectiveness, among other quality measures, such as the cross-correlation value [2], the mean square error (MSE) [65] and the signal-to-noise ratio (SNR) [32].

DWT-based denoising

Table 3 Percentage PrmsD index between the noise-free and the denoised signal, for different mother wavelets and thresholding rules (\(10\%\) N/S ratio, decomposition level 3)

The effectiveness of the DWT-based denoising depends on several pre-defined parameters (user choices), including the mother wavelet typology, the thresholding rule, the type of thresholding and the wavelet decomposition level. In assessing these, for both the non-stationary and the stationary excitation cases, the N/S ratio is set equal to 10% and the wavelet decomposition level is set at 3. The Symlet, Coiflet, Daubechies, Biorthogonal, Reverse Biorthogonal and Discrete Meyer mother wavelets are considered, combined with the thresholding rules of Sect. 2.2, that is, the Heuristic SURE, the Sqtwolog, the Minimax and the Rigorous SURE ones. In addition, different numbers of oscillations are considered (indicated with a number next to the wavelet mother name) for each mother wavelet. As the performed analysis does not reveal substantial differences between the structural responses of the first and the last story, only results concerning the latter signal are reported for the rest of the benchmark study.

Table 4 Percentage PrmsD index between the noise-free and the denoised signal for different decomposition levels and thresholding types (\(10\%\) N/S ratio)

The PrmsD index for the explored combinations of mother wavelets and thresholding rules is illustrated in Table 3. It is observed that in the seismic excitation case the results do not seem to be significantly affected by the number of oscillations characterizing the considered mother wavelet. On the contrary, for the ambient excitation a considerable dispersion of the index occurs. For the non-stationary signal, the lowest PrmsD value is attained for the Smylet-2 mother wavelet with the Heursure thresholding rule, whereas for the stationary signal the lowest value of the index is produced by the combination of Coiflet-4 mother wavelet with the Minimax rule.

By fixing these parameters, the decomposition level minimizing the PrmsD index is then examined, for both soft and hard thresholding, and the results are printed in Table 4. In the non-stationary case and for \(10\%\) N/S ratio, a hard thresholding at level 2 is suggested, while for the stationary case and the same N/S ratio the optimal setting is provided by a hard thresholding at level 3. In light of the derived results, it clearly emerges how the choice of the mother wavelet type as well as the selection of the decomposition level, play a crucial role for the efficacy of the DWT-based denoising approach. The mother wavelet should be chosen such that it can be able to well characterize the processed signal and this may be established according to the correlation between the wavelet and the signal itself. Additionally, if the noise floor is hardly detectable, more decomposition levels might be required, in order to reconstruct even the finest details of the signal; consequently, in some cases, it may be useful to work with multiple decomposition levels. Since the optimal decomposition level of the wavelet may be different, depending on the level of noise affecting the data, within the present analysis, the optimal decomposition level will be each time re-adjusted, according to the considered N/S ratio, to get the best denoising estimates.

Figure 6 shows the denoised seismic response signal for \(25\%\) N/S ratio, for the Smylet2 mother wavelet and the Heursure thresholding at level 2. The benefits of the denoising process are quite tangible and demonstrate that the noise is largely successfully removed and the original data set is almost completely reconstructed. On the contrary, the same results are not drawn for the processing of ambient vibration response signals: as illustrated in Fig. 7, for denoising via the Coiflet4 wavelet with Minimax thresholding at level 3, the error in terms of difference between the original clean signal and the denoised one is clearly higher than for the non-stationary case. These findings indicate that the effectiveness of the DWT-based denoising depends on the nature of the source signal. Indeed, denoising of non-stationary response signals can be very effective, whereas the same procedure exhibits visible limitations when applied to stationary records.

Fig. 6
figure6

DWT-based denoising of \(\ddot{x}_{10}[k]\) under seismic input (\(25\%\) N/S ratio)

Fig. 7
figure7

DWT-based denoising of \(\ddot{x}_{10}[k]\) under ambient vibration (\(25\%\) N/S ratio)

SVD-based denoising

Turning our attention to the SVD, the critical parameters of the method pertain to the size of the Hankel matrix (parameter m), as well as to the truncation of the sum of Eq. 26 (parameter \(\ell \)). Table 5 displays the percentage PrmsD index for the denoising of a 10% noise-corrupted acceleration signal, for various choices of the size and truncation parameters, where the latter is calculated as the subset of singular values that are lower than a certain percentage with respect to the largest one. It can be deduced that both m and \(\ell \) affect the quality of the denoising process, while the lowest index values are succeeded for \(m=300\) and \(\ell =5\%\). Adopting these parameter values, Figs. 8 and 9 illustrate the denoised structural responses for the non-stationary and stationary cases, respectively, for the 25% noise-corrupted signals. The behaviour seems to be comparable to the previously treated DWT-based denoising, exhibiting a slightly better performance in the non-stationary case, where the error is somewhat smaller than for the DWT one.

Table 5 Percentage PrmsD index between the noise-free and the denoised signal for different values of m and \(\ell \) (SVD, \(10\%\) N/S ratio)
Fig. 8
figure8

SVD-based denoising of \(\ddot{x}_{10}[k]\) under seismic input (\(25\%\) N/S ratio)

Fig. 9
figure9

SVD-based denoising of \(\ddot{x}_{10}[k]\) under ambient vibration (\(25\%\) N/S ratio)

Time and frequency domain comparison

Table 6 and Fig. 10 show the performance of the two denoising methods for various N/S ratios, in terms of the PrmsD index, for both the non-stationary and stationary excitations. Focusing first on the seismic excitation, the DWT-based approach seems to outperform the SVD one for N/S ratios lower than 20%, values that are more often encountered in real applications. However, for higher N/S ratios, the SVD appears to be more robust, since it is not so affected by the level of noise considered on the measurements, and the PrmsD index “converges” asymptotically to 0.60-0.70%. On the contrary, for the ambient excitation case, the SVD-based approach is clearly more effective.

Table 6 Percentage PrmsD index for different signal types and N/S ratios
Fig. 10
figure10

Effectiveness evaluation of DWT- and SVD-based denoising techniques for increasing N/S ratios and different signal typologies: (left) earthquake excitation; (right) ambient vibration. Indicated trends come from a polynomial fit of degree 2

Proceeding further with the assessment in the time domain, the effects of denoising on the signal’s main peak value are examined, since this value plays oftentimes a key role in the design, or assessment phase. To this end, the vibration acceleration peak values resulting from the two denoising methods, as well as their percentage alteration with respect to the noise-corrupted peak (\(\Delta _{na}\)), and the peak value (\(\Delta _{nd}\)) of the original signal are summarized in Table 7. From the obtained results, it can be deduced that the denoising process always leads to a peak reduction; this feature is attributed to the fact that the artificial noise acts additively. In the non-stationary case, the SVD-based denoising seems to lead to a flattening effect on the main signal peak, in comparison to the DWT-based denoising, while the stationary case is generally associated with greater percentage variation values \(\Delta _{nd}\). However, this main peak analysis is more pertinent to the non-stationary signals, for which \(\Delta _{nd}\) presents low values, at least for common N/S ratios. In addition to having a greater practical relevance for vibration assessment purposes, the analysis conducted on non-stationary response signals turns out to be more significant than that performed on stationary signals, since the main acceleration peak clearly emerges and it is immediate to be detected, configuring itself as a characteristic feature describing this kind of signal.

Table 7 Peak acceleration values of the DWT and SVD denoised signals, and their variation with respect to noise-affected signal peak \(\Delta _{na}\) and to numerically determined signal peak \(\Delta _{nd}\), for different typologies of signals (i.e. non-stationary (seismic) signal and stationary (white noise) signal) and N/S ratios
Fig. 11
figure11

Power spectrum of the original and the denoised signals (seismic excitation, 10% N/S ratio)

Table 8 Original and identified natural frequencies (peak-picking on Welch’s spectral estimates)

Figures 11, 12, 13 and 14 display the power spectrum (Welch’s method, \(N_{_{FFT}}=1024\), 50% overlap), of the original and the denoised records for two N/S ratios, e.g. 10% and 25%, respectively. For both the excitation cases, the quality of the denoising methods within the effective band ([0, 10] Hz) is quite comparable; a small discrepancy is observed in the SVD-based spectra, in a small frequency band before 10 Hz. This suggests that the 5% threshold in the execution of the SVD causes distortion of important information. On the contrary, the DWT-based denoising seems to increase the energy of the signal after approximately 10 Hz.

Fig. 12
figure12

Power spectrum of the original and the denoised signals (ambient excitation, 10% N/S ratio)

Fig. 13
figure13

Power spectrum of the original and the denoised signals (seismic excitation, 25% N/S ratio)

Fig. 14
figure14

Power spectrum of the original and the denoised signals (ambient excitation, 25% N/S ratio)

Finally, Table 8 illustrates the “denoised”, undamped natural frequencies, calculated by applying the peak-picking method to the Welch spectral estimates. The estimates \({\hat{f}}_n\) in the third column of the table correspond to nonparametric estimates from the power spectrum of the noise--free vibration acceleration signal. Expectedly, the results for the ambient excitation case are better, succeeding in identifying more frequencies than in the non-stationary case, for both N/S ratios. Rather similar results apply to the SVD-based denoising, showing that it is more the signal typology that may affect the estimates, rather than the employed denoising method.

Fig. 15
figure15

The UHPFRC railway bridge in Sempachersee, Switzerland

Operational assessment

The examined methods are now applied to the denoising problem of real vibration acceleration records, obtained during a long-term monitoring campaign of an Ultra-High Performance Fiber Reinforced Concrete (UHPFRC) railway bridge in Sempachersee, Switzerland (canton Lucerne). The bridge (Fig. 15a), which is located in “Linie 500”, at km 78.330 along the Sursee-Sempach route, consists of two UHPFRC plates with a ribbed profile and a 6 meter span length. The height of the ribs varies between 0.45 m at the supports and 0.5 m at midspan. These plates were then connected on-site by a mortar. Since there is no structure of similar traits to compare its performance to, a monitoring campaign, covering a period from December 2018 up to September 2020, is initiated to measure the long-term behaviour of the structure (and material), which would help for future constructions using the same material. An additional benefit comes from the continuous tracking of several performance indicators which could help identifying any unforeseen changes and thus aid to plan inspection and intervention programmes.

Fig. 16
figure16

Selected vibration acceleration response records from the railway bridge (position A4, respectively)

Fig. 17
figure17

Spectrograms of the records of Fig. 16

The complete instrumentation network is shown in Fig. 15b and consists of low-cost MEMS accelerometers (Ax), laser distometers (not visible), tiltmeters (Tx), strain gauges (SGx), as well as a temperature and a humidity sensor (not visible) for tracking the environmental effects on the bridge. The associated data acquisition process operates in triggered mode, every time a train crosses the bridge, and stores 22 s (including 2 s of pre-triggered samples) of bridge vibration response at a sampling rate \(F_s=2000\) Hz. Two representative such records are displayed in Fig. 16 and herein used for assessing the two denoising methods. Both records correspond to crossings of typical revenue trains with five wagons. It is noted that, due to confidentiality reasons, axes scales are suppressed. Figure 17 illustrates the spectrograms of the associated time series (Welch’s method with \(N_{_{FFT}}=512\) and 50% overlap). Five frequency zones are observed, corresponding to the passage of the associated wagons, confirming the transient nature of the vibration records.

Fig. 18
figure18

DWT-based denoising of the records of Fig. 16

In applying the DWT for the denoising of the signals of Fig. 16, all parameters from the numerical case study remain unaltered, except from the number of oscillations in the mother wavelet. In more detail, the Smylet mother wavelet is herein adopted, combined with the Heursure thresholding and a decomposition level 2, whereas the number of oscillations is increased from 2 to 20, aiming at achieving a better clarification of the real acceleration signal in the time domain and, at the same time, for obtaining a response spectrum coherent with the frequency content of the original signal. The results are shown in Fig. 18, in a time window during the crossing of the first and second wagons, respectively, for \(y_1[k]\) and \(y_2[k]\). All high frequency content of the signals, usually associated with complex vehicle-rail interaction effects for this class of problems, apart from sensor and instrumentation noise, has been greatly suppressed, rendering the denoised signals \({\hat{x}}_1[k]\) and \({\hat{x}}_2[k]\) representative of the structural dynamics of the infrastructure (rail, ballast and bridge).

Fig. 19
figure19

SVD-based denoising of the records of Fig. 16

Fig. 20
figure20

DWT- vs. SVD-based denoising of the records of Fig. 16

Similar results are attained for the SVD-based denoising, as it can be inspected in Fig. 19, for \(m=300\) and \(\ell =30\%\). The truncation of the signals using such a high value for \(\ell \) is deemed necessary, due to the presence of considerable “local” high-frequency dynamics in the original records, as explained before. Indeed, lower values of \(\ell \) maintain these local oscillating phenomena to a great extend. In appreciating the effects of SVD-based denoising, Fig. 20 compares the outcomes of the denoising process for both methods. Although a good agreement emerges between the two treated techniques in denoising the examined real acceleration signals, the SVD-based approach seems to produce higher overall values, compared to the DWT-based one, for the first record, whereas the opposite holds for the second record. This visual evidence is confirmed by calculating the percentage RMS and peak values reduction, which are presented in Table 9. Indeed, while for \(y_1[k]\) the figures between DWT and SVD are comparable, the reduction after the denoising of \(y_2[k]\) is almost double for the SVD. Taking into account the fact that the two records have been acquired from similar operational conditions, e.g. crossing of similar trains (wagon number) at comparable velocities (the spectral peaks in Fig. 17 are located at the same areas in both spectrograms), the different results in the denoising process are rather attributed to an alteration of the instrumentation noise. This, in turn, might be explained by many factors, with the environmental effects being the most prominent. In any case, this typical performance of long-term monitoring installations in infrastructures emphasizes on the necessity of adopting appropriate denoising measures before any post-processing and estimation operations.

Table 9 Percentage reduction of overall values (RMS, Peak) for the denoised bridge records
Fig. 21
figure21

Spectrograms of the bridge’s denoised records

Finally, Fig. 21 shows the spectrograms of the denoised signals (Welch’s method with \(N_{_{FFT}}=512\) and 50% overlap). Compared to Fig. 16, the spectrograms of the acquired signals, the reduction of the signals’ energy above approximately 200 Hz is evident; the denoising process leaves only frequency zones of small energy. Interestingly, the effective frequency band of the SVD-based denoised signals is narrower, with respect to the one of the DWT-based records. This is an indication that threshold parameter \(\ell \) might be overestimated. In any case, the frequency band of the denoised acceleration records is now concentrated in regions where the structural dynamics of the infrastructure usually dominate.

As final considerations, the following additional remarks may be supplied, on behalf of the interested reader, about the investigated operational case study:

  • concerning the unscaled depictions in Figs. 1618, 19 and 20, the acquired data were subject to certain restrictions posed from the infrastructure manager. That being noted, first, the adopted representations aim to focus on the appreciation of the signal features, and the achieved denoising effects. It is believed that the interpretation of the results is then accomplished in a way that indeed quantifies the efficacy of the assessed denoising methods.

  • the denoising process has been applied to all the data acquired from the various deployed acceleration sensors. Two most representing and characteristic signals have been selected, as reference samples, and for relative comparison. Indeed, it was decided not to include further data, since: (i) the results turn out very similar, due to the similarity of the sensors and the recorded data; and (ii) to avoid lengthening the presentation, while likely distracting from the main message, of achieved effective denoising, as already condensed and revealed, for the two adopted signals.

Conclusions

In this paper, the DWT and the SVD as denoising methods for civil engineering applications have been investigated and assessed, in both a numerical and an operational setting.

In the former case, some critical remarks can be summarized as follows:

  • In qualifying the DWT parameters, it is found that in the non-stationary (seismic) excitation case, the combination of Smylet having two oscillations in its mother wavelet and Heursure hard thresholding type at decomposition level 2, renders the smallest PrmsD value. For ambient records, an optimal configuration is succeeded by the combination of Coiflet having four oscillations in its mother wavelet and Minimax hard thresholding type at decomposition level 3.

  • Although the tuning of the SVD requires only two parameters, the size of the Hankel matrix and the truncation of the singular values, both affect significantly the outcome. However, the SVD-based appears more robust for various N/S ratios and, generally, provides a better approximation of the original response signal. The combined use of both techniques might, yet, be the optimal strategy.

  • In examining the effects of denoising on the overall values (RMS, peak), it is found that non-stationary records lead to better results than stationary ones. Moreover, the SVD-based denoising seems to flatten out the peaks less than the DWT.

  • The effectiveness of denoising has also been assessed in the frequency domain. Quite satisfactory results are obtained for non-stationary records, in particular those deriving from the SVD-based approach, whereas for ambient vibration the power spectrum does not seem to be so coherent with the original one, especially at the frequency bands where the noise affects the signal. This may constitute a strong motivation for further research on this latter specific class of structural vibration response.

Similar outcomes are drawn from applying the methods to real operational data. In particular, the benefits of denoising in purifying the vibration acceleration response signals of the considered railway bridge are evident in the time domain, where the outcomes of the DWT- and SVD-based denoising seem to be in comparable agreement to each other. Expectedly, the overall values of the denoised signals appear to be flatter. The frequency domain representation of the denoised records preserves the useful information in the band of interest, and suppresses high frequency components, which are attributed either to instrumentation noise, or to dynamic phenomena that are unrelated to the examined infrastructure (e.g., vehicle-rail interaction).

The encouraging results of the study promote the implementation of denoising as an integrated element of monitoring campaigns in civil and structural engineering, for the purposes of structural identification and health monitoring [49,50,51,52,53,54,55]. The examined methods can be appropriately applied to other data types as well, including displacement and strain gauge records, and may be effectively combined with model-based approaches, such as heterogeneous data fusion [27]. As the current trend in monitoring qualifies the implementation of low-cost, large numbers of sensors, instead of a few expensive ones, proper and careful denoising may assist in extracting accurate information from large-scale infrastructures.

References

  1. 1.

    Alfaouri M, Daqrouq K (2008) ECG signal denoising by wavelet transform thresholding. Am J Appl Sci 5(3):276–281

    Article  Google Scholar 

  2. 2.

    Al-Qazzaz NK, Ali S, Ahmad SA, Islam MS, Arif MI (2015) Selection of mother wavelets thresholding methods in denoising multi-channel EEG signals during working memory task. Sensors 15(11):15–35

    Article  Google Scholar 

  3. 3.

    Alyasseri ZAA, Khader AT, Al-Betar MA (2017) Electroencephalogram signals denoising using various mother wavelet functions: a comparative analysis. In: Proceedings of the international conference on imaging, signal processing and communication, July 26–28 2017, Penang, Malaysia, pp 100–105

  4. 4.

    Antoni J, Randall RB (2006) The spectral kurtosis: application to the vibratory surveillance and diagnostics of rotating machines. Mech Syst Signal Process 20(2):308–331

    Article  Google Scholar 

  5. 5.

    Arezki M, Berkani D (2009) Fast algorithms with low complexity for adaptive filtering. WSEAS Trans Signal Process 5(1):23–31

    Google Scholar 

  6. 6.

    Aqil M, Jbari A, Bourouhou A (2017) ECG signal denoising by discrete wavelet transform. Int J Online Eng 13(9):51–68

    Article  Google Scholar 

  7. 7.

    Baravdish G, Evangelista G, Svensson O, Sofya F (2012) Effective heterogeneous data fusion procedure via Kalman filtering. In: 5th international symposium on communications, control and signal processing, 2–4 May 2012. Italy, Rome, pp 1–4

  8. 8.

    Blu T (1998) A new design algorithm for two-band orthonormal rational filter banks and orthonormal rational wavelets. IEEE Trans Signal Process 46(6):1494–1504

    MathSciNet  Article  Google Scholar 

  9. 9.

    Buades A, Coll B (2006) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530

    MathSciNet  MATH  Article  Google Scholar 

  10. 10.

    Capellari G, Chatzi EN, Mariani S (2018) Structural health monitoring sensor network optimization through Bayesian experimental design. J Risk Uncertain Eng Syst 4(2):401–427

    Google Scholar 

  11. 11.

    Chang SG, Bin Y, Vetterli M (2000) Adaptive wavelet thresholding for image denoising and compression. IEEE Trans Image Process 9(9):1532–1546

    MathSciNet  MATH  Article  Google Scholar 

  12. 12.

    Chatzi EN, Smyth AW (2016) Particle filter scheme with mutation for the estimation of time-invariant parameters in structural health monitoring applications. Struct Control Health Monit 20(7):1081–1095

    Article  Google Scholar 

  13. 13.

    Chatzi EN, Fuggini C (2012) Structural identification of a super-tall tower by GPS and accelerometer data fusion using a multi-rate Kalman filter. In: Proceedings of the 3rd international symposium on life-cycle civil engineering, Delft, Netherlands, 3–6 October 2012, vol 10, pp 144–151

  14. 14.

    Chatzi EN, Fuggini C (2015) Online correction of drift in structural identification using artificial white noise observations and an Unscented Kalman filter. Smart Struct Syst 16(2):296–328

    Article  Google Scholar 

  15. 15.

    Chen G, Bui T (2003) Multiwavelets denoising using neighboring coefficients. IEEE Signal Process Lett 10(7):211–214

    Article  Google Scholar 

  16. 16.

    Chen Y, Zhang P (2012) Bearing fault detection based on SVD and EMD. Appl Mech Mater 184–185:70–74

    Article  Google Scholar 

  17. 17.

    DiMonte CL, Arun KS (1990) Tracking the frequencies of superimposed time-varying harmonics. Int Conf Acoust Speech Signal Process 5:2539–2542

    Article  Google Scholar 

  18. 18.

    Dohono DL, Johnston LM (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3):425–455

    MathSciNet  Article  Google Scholar 

  19. 19.

    Dohono DL, Johnston LM (1995) Adapting to unknown smoothness via wavelet shrinkage. J Am Stat Assoc 90(432):1200–1224

    MathSciNet  Article  Google Scholar 

  20. 20.

    Dohono DL (1995) Denoising by soft thresholding. IEEE Trans Inf Theory 41(3):613–627

    Article  Google Scholar 

  21. 21.

    Dohono DL, Johnston LM (1998) Minimax estimation via wavelet shrinkage. Ann Stat 26(3):879–921

    MathSciNet  Google Scholar 

  22. 22.

    Ergen B (2012) Signal and image denoising using wavelet transform. Extract from: advances in wavelet theory and their applications in engineering, physics and technology, Dumitru Baleanu editor, Fırat University, Turkey

  23. 23.

    Farrar CR, James GH III (1997) System identification from ambient vibration measurements on a bridge. J Sound Vib 205(1):1–18

    Article  Google Scholar 

  24. 24.

    Fedi M, Lenarduzzi R, Primiceri R, Quarta T (2000) Localized denoising filtering using the wavelet transform. Pure Appl Geophys 157(9):1463–1491

    Article  Google Scholar 

  25. 25.

    Feng Y, Thanagasundram S, Schlindwein FS (2006) Discrete wavelet-based thresholding study on acoustic emission signals to detect bearing defect on a rotating machine. In: Proceedings of the 13th international congress on sound and vibration

  26. 26.

    Ferrari R, Froio D, Chatzi EN, Gentile C, Pioldi F, Rizzi E (2015) Experimental and numerical investigation for the structural characterization of a historic RC arch bridge. In: COMPDYN 2015, 5th ECCOMAS thematic conference on computational methods in structural dynamics and earthquake engineering, Crete Island, Greece, 25–27 May 2015, vol 1, pp 2337–2353. Available online in Eccomas Proceedia. www.eccomasproceedia.org/conferences/thematic-conferences/compdyn-2015/3542

  27. 27.

    Ferrari R, Pioldi F, Rizzi E, Gentile C, Chatzi EN, Klis R, Serantoni E, Wieser A (2015) Heterogeneous sensor fusion for reducing uncertainty in Structural Health Monitoring. In: UNCECOMP 2015, 1st ECCOMAS thematic conference on international conference on uncertainty quantification in computational sciences and engineering, Crete Island, Greece, 25–27 May 2015, pp 511–528. Available online in Eccomas Proceedia. www.eccomasproceedia.org/conferences/thematic-conferences/uncecomp-2015/4289

  28. 28.

    Ferrari R, Pioldi F, Rizzi E, Gentile C, Chatzi EN, Serantoni E, Wieser A (2016) Fusion of wireless and non-contact technologies for the dynamic testing of a historic RC bridge. Meas Sci Technol 27(12):1–19

    Article  Google Scholar 

  29. 29.

    Ferrari R, Froio D, Rizzi E, Gentile C, Chatzi EN (2019) Model updating of a historic concrete bridge by sensitivity- and global optimization-based latin hypercube sampling. Eng Struct 179:139–160

    Article  Google Scholar 

  30. 30.

    Gao HY, Andrew GB (1997) WaveShrink with firm shrinkage. Statistica Sinica 7(4):855–874

    MathSciNet  MATH  Google Scholar 

  31. 31.

    Gao HY (1998) Wavelet shrinkage denoising using non-negative garrote. J Comput Graphical Stat 7(4):469–488

    MathSciNet  Google Scholar 

  32. 32.

    Gradolewski D, Redlarski G (2013) The use of wavelet analysis to denoising of electrocardiography signal. In: XV international Ph.D. workshop OWD, 19–22 October 2013, pp 456–461

  33. 33.

    Guo Q, Zhang C, Zhang Y, Liu H (2016) An efficient SVD-based method for image denoising. IEEE Trans Circuits Syst Video Technol 26(5):868–880

    Article  Google Scholar 

  34. 34.

    Jensen SH, Hansen PC, Hansen SD, Sorensen JA (1995) Reduction of broad-band noise in speech by truncated QSVD. IEEE Trans Speech Audio Process 3(6):439–448

    MATH  Article  Google Scholar 

  35. 35.

    Jing J, Hu Y, Li X, Huang Z (2007) Feature extraction of pulse signal based on Hilbert–Huang transformation and singular value decomposition. In: Proceedings of the 1st international conference on bioinformatics and biomedical engineering, Wuhan, China, 6–8 July 2007

  36. 36.

    Kam HS, Cheong SN, Tan WH (2005) An adaptive fuzzy image smoothing filter for Gaussian noise. WSEAS Int Conf Autom Inf 2005:323–328

    Google Scholar 

  37. 37.

    Karthikeyan P, Murugappan M, Yaacob S (2012) ECG signal denoising using wavelet thresholding technique in human stress assessment. Int J Electr Eng Inf 4(2):306–319

    Google Scholar 

  38. 38.

    Katayama T (2006) Subspace methods for system identification. Springer, Berlin

    MATH  Google Scholar 

  39. 39.

    Konstantinides K, Natarajan B, Yovanof GS (1997) Noise estimation and filtering using block-based singular value decomposition. IEEE Trans Image Process 6(3):479–483

    Article  Google Scholar 

  40. 40.

    Konstantinides K, Yao K (1998) Statistical analysis of effective singular values in matrix rank determination. IEEE Trans Acoust Speech Signal Process 36(5):757–763

    MATH  Article  Google Scholar 

  41. 41.

    Koo KY, Brownjohn JMV, List DI, Cole R (2003) Structural health monitoring of the Tamar suspension bridge. Struct Control Health Monit 20(4):609–625

    Article  Google Scholar 

  42. 42.

    Lee JJ, Fukuda Y, Shinozuka M, Cho S, Yun CB (2007) Development and application of a vision-based displacement measurement system for structural health monitoring of civil structures. Smart Struct Syst 3(3):373–384

    Article  Google Scholar 

  43. 43.

    Lee KC, Ou JS, Fang MC (2008) Application of SVD noise-reduction technique to PCA based radar target recognition. Prog Electromagn Res 81:447–459

    Article  Google Scholar 

  44. 44.

    Lyon-Caen H, Armijo R, Drakopoulos J, Baskoutass J, Delibassis N, Gaulon R, Kouskouna V, Latousakis J, Makropoulos K, Papadimitriou P, Panastasiou D, Pedotti G (1988) The 1986 Kalamata (South Peloponnesus) earthquake: detailed study of a normal fault, evidences for east-west extension in the Hellenic arc. J Geophys Res Atmos 93(B12):14967–15000

    Article  Google Scholar 

  45. 45.

    Madisetti VK, Williams DB (1999) Digital signal processing handbook. CRC Press LLC, Boca Raton

    Google Scholar 

  46. 46.

    Mallat SG (1999) A wavelet tour of signal processing. Academic Press, London

    MATH  Google Scholar 

  47. 47.

    Moulin P, Liu J (1999) Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors. IEEE Trans Inf Theory 45(3):909–919

    MathSciNet  MATH  Article  Google Scholar 

  48. 48.

    Otis MS Jr (1991) PSD computations using Welch’s method. Sandia National Laboratories Albuquerque, Printed in the USA, December 1991

  49. 49.

    Pioldi F, Ferrari R, Rizzi E (2016) Output-only modal dynamic identification of frames by a refined FDD algorithm at seismic input and high damping. Mech Syst Signal Process 68–69:265–291

    Article  Google Scholar 

  50. 50.

    Pioldi F, Ferrari R, Rizzi E (2015) Earthquake structural modal estimates of multi-storey frames by a refined FDD algorithm. J Vib Control 23(13):2037–2063

    Article  Google Scholar 

  51. 51.

    Pioldi F, Ferrari R, Rizzi E (2017) Seismic FDD modal identification and monitoring of building properties from real strong-motion structural response signals. Struct Control Health Monit 24(11):1–20

    Article  Google Scholar 

  52. 52.

    Pioldi F, Rizzi E (2017) Refined Frequency Domain Decomposition modal dynamic identification from earthquake-induced structural responses. Meccanica 52(13):3165–3179

    MathSciNet  Article  Google Scholar 

  53. 53.

    Pioldi F, Rizzi E (2017) A refined frequency domain decomposition tool for structural modal monitoring in earthquake engineering. Earthq Eng Eng Vib 16(3):627–648

    Article  Google Scholar 

  54. 54.

    Pioldi F, Rizzi E (2018) Assessment of frequency versus time domain enhanced technique for response-only modal dynamic identification under seismic excitation. Bull Earthq Eng 16(3):1547–1570

    Article  Google Scholar 

  55. 55.

    Pioldi F, Rizzi E (2018) Earthquake-induced structural response output-only identification by two different operational modal analysis techniques. Earthq Eng Struct Dyn 47(1):257–264

    Article  Google Scholar 

  56. 56.

    Portilla J, Strela V (2003) Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Process 12(11):1338–1351

    MathSciNet  MATH  Article  Google Scholar 

  57. 57.

    Premchaiswadi N, Yimgnagmand S, Premchaiswadi W (2010) A scheme for salt and pepper noise reduction and its application for OCR system. WSEAS Trans Comput 9(4):351–360

    Google Scholar 

  58. 58.

    Qiao Z, Pan Z (2015) SVD principle analysis and fault diagnosis for bearings based on the correlation coefficient. Meas Sci Technol 26(8):85–114

    Article  Google Scholar 

  59. 59.

    Ravizza G, Ferrari R, Rizzi E, Chatzi EN (2018) Effective heterogeneous data fusion procedure via Kalman filtering. Smart Struct Syst 22(5):631–641

    Google Scholar 

  60. 60.

    Ravizza G, Ferrari R, Rizzi E, Dertimanis V, Chatzi EN (2019) Denoising corrupted structural vibration response: critical comparison and assessment of related methods. In: Papadrakakis M, Fragiadakis M (eds) Proceedings of the 7th international conference on computational methods in structural dynamics and earthquake engineering (COMPDYN 2019), an ECCOMAS thematic conference, an IACM special interest conference, 24–26 June 2019, Hersonissos, Crete Island, Greece, Institute of Structural Analysis and Antiseismic Research, National Technical University of Athens (NTUA), ECCOMAS Proceedia, Volume 3, 2019, pp 3893–3904, COMPDYN 2019, Category: C-RS 02—algorithms for structural health monitoring, Eccomas Proceedia ID: 7194, conference Proceeding ID: 19291. https://doi.org/10.7712/120119.7194.19291

  61. 61.

    Ravizza G, Ferrari R, Rizzi E, Dertimanis V, Chatzi EN (2019) An integrated monitoring strategy for current condition assessment of historic bridges. In: Papadrakakis M, Fragiadakis M, Papadimitriou C (eds) Proceedings of the 11th international conference on structural dynamics (EURODYN 2020), EASD (European Association for Structural Dynamics) thematic conference, 23–26 November 2020, streamed online from Athens, Greece, conference Proceeding ID: E20148, Category: MS8—structural health monitoring, vol 1, pp 1373–1387, Institute of Structural Analysis and Antiseismic Research, National Technical University of Athens (NTUA). Published in EASD Procedia, EURODYN (2020) 1373–1387, EASD Procedia ID: 9111, ISSN: 2311-9020. https://doi.org/10.47964/1120.9111.20148.

  62. 62.

    Rioul O, Vetterli M (1991) Wavelets and signal processing. Signal Process Mag 8(4):14–38

    Article  Google Scholar 

  63. 63.

    Roberts GW, Dodson AH (2003) A remote bridge health monitoring system using computational simulation and GPS sensor data. In: Proceedings of the 11th FIG symposium on deformation measurements, Santorini, Greece, 2003

  64. 64.

    Ruotolo R, Surace C (1999) Using SVD to detect damage in structures with different operational conditions. J Sound Vib 226(3):425–439

    Article  Google Scholar 

  65. 65.

    Sadooghi SS, Khadem EK (2016) A new performance evaluation scheme for jet engine vibration signal denoising. Mech Syst Signal Process 76–77:201–212

    Article  Google Scholar 

  66. 66.

    Sanliturk KY, Cakar O (2005) Noise elimination from measured frequency response functions. Mech Syst Signal Process 19(3):615–631

    Article  Google Scholar 

  67. 67.

    Schanze T (2017) Removing noise in biomedical signal recordings by singular value decomposition. Curr Directions Biomed Eng 3(2):253–256

    Article  Google Scholar 

  68. 68.

    Shanxue Z, Chao G (2010) Wavelet transform threshold noise reduction methods in the oil pipeline leakage monitoring and positioning systems. J Electron 27(3):405–411

    Google Scholar 

  69. 69.

    Sifuzzaman M, Islam MR, Ali MZ (2009) Application of wavelet transform and its advantages compared to Fourier transform. J Phys Sci 13(2009):121–134

    Google Scholar 

  70. 70.

    Van Overschee P, De Moor BL (1996) Subspace identification for linear systems: theory—implementation—applications. Kluwer Academic Publishers, Dordrecht

    MATH  Book  Google Scholar 

  71. 71.

    Vasilescu G (2006) Electronic noise and interfering signals: principles and applications. Springer, Berlin

  72. 72.

    Veerakumar T, Esakkirajan S, Vennila I (2012) Combined fuzzy logic and unsymmetric trimmed median filter approach for the removal of high density impulse noise. WSEAS Trans Signal Process 8(1):32–42

    Google Scholar 

  73. 73.

    Verma N, Verma AK (2012) Performance analysis of wavelet thresholding methods in denoising of audio signals of some Indian musical instruments. Int J Eng Sci Technol 4(5):2047–2052

    Google Scholar 

  74. 74.

    Vetterli M, Herley C (1992) Wavelets and filter banks: theory and design. IEEE Trans Signal Process 40(9):2207–2232

    MATH  Article  Google Scholar 

  75. 75.

    Verhaegen M, Verdult V (2007) Filtering and system identification: a least squares approach. Cambridge University Press, Cambridge

    MATH  Book  Google Scholar 

  76. 76.

    Zhao X, Ye B (2009) Similarity of signal processing effect between Hankel matrix-based SVD and wavelet transform and its mechanism analysis. Mech Syst Signal Process 23(4):1062–1075

    Article  Google Scholar 

  77. 77.

    Zhao X, Ye B (2011) Selection of effective singular values using difference spectrum and its application to fault diagnosis of headstock. Mech Syst Signal Process 25(5):1617–1631

    MathSciNet  Article  Google Scholar 

  78. 78.

    Zhao M, Jia X (2017) A novel strategy for signal denoising using reweighted SVD and its application to weak fault feature enhancement of rotating machinery. Mech Syst Signal Process 94:129–147

  79. 79.

    Zou H, Tewfik AH (1993) Parametrization of compactly supported orthonormal wavelets. IEEE Trans Signal Process 41(3):1428–1431

    MATH  Article  Google Scholar 

Download references

Acknowledgements

Public research support from “Fondi di Ricerca d’Ateneo ex 60%” and a ministerial doctoral grant and funds at the ISA Doctoral School, University of Bergamo, Department of Engineering and Applied Sciences (Dalmine), are gratefully acknowledged.

Funding

Open access funding provided by Università degli studi di Bergamo within the CRUI-CARE Agreement.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Egidio Rizzi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ravizza, G., Ferrari, R., Rizzi, E. et al. On the denoising of structural vibration response records from low-cost sensors: a critical comparison and assessment. J Civil Struct Health Monit 11, 1201–1224 (2021). https://doi.org/10.1007/s13349-021-00502-y

Download citation

Keywords

  • Noise-corrupted signal
  • Denoising techniques
  • Discrete wavelet transform
  • Singular value decomposition
  • Seismic
  • Ambient and railway vibration responses