# Response Simulation, Data Cleansing and Restoration of Dynamic and Static Measurements Based on Deep Learning Algorithms

- 126 Downloads

**Part of the following topical collections:**

## Abstract

In this study, an output-based neuro controller was built based on the idea of the adaptive neuro-fuzzy inference system (ANFIS) and its capabilities in response simulation, data cleansing and restoration capability were verified using measurement data from actual structural testing. The ANFIS is a family of the deep learning algorithm, which incorporates the benefits of adaptive control technique, artificial neural network, and the fuzzy inference system. Thus, it is expected to produce very accurate predictions even for the highly nonlinear system. Forced vibration responses of a five-story steel building were simulated by ANFIS and its accuracy was compared with the results of Recurrent Neural Network (RNN), which is a type of traditional artificial neural networks. Simulations by ANFIS were very accurate with a much lower root means square error (RMSE) than RNN. Simulated data by ANFIS showed an almost perfect match with the original. Even the small ripples in the power spectrum plot outside the dominant frequency were successfully reproduced. In addition, the ANFIS was used to increase the sampling rate of dynamic data. It was shown that missing high-frequency contents could be successfully reproduced when the ANFIS was properly trained. Lastly, The ANFIS was applied to remove the noise in the measured data from RC column cyclic load tests. The outliers were corrected effectively, but the tendency of flattening the peak values was observed.

## Keywords

artificial intelligence deep learning adaptive neuro-fuzzy inference system response simulation data cleansing noise reduction data restoration## 1 Introduction

Performance data collected from structural testing have been the fundamental resource for the development of structural engineering. Efforts to simulate observed responses from controlled input by mathematical modeling lead to a finding of principal laws and progress of design equations. Simple parametric modeling, although they are effective in extracting key parameters affecting the behaviors, has limitations in accurate simulation of observed responses especially for complicated or nonlinear systems.

Artificial neural networks have been used to determine the parameters or simulate the response of unknown systems from correlations between inputs and outputs of training data (Imam et al. 2015; Sathyan et al. 2018). Nowadays, advances in artificial intelligence such as the machine learning and deep learning techniques integrated with adaptive control algorithms and/or fuzzy logic inference system allow us accurate simulations of complicated and nonlinear systems. According to Fu (1970) and Gupta and Saridis (1977), the intelligent intelligence can be defined as a combination of control theory and information technology to imitate human behavior such as ‘learning’ and ‘problem solving’. It involves interpretation of input data, treatment of the ambiguity in human perception for transformation to machine knowledge (Gupta and Saridis 1977; Saridis and Lee 1979; Fu and Gonzalez 1987; Fu 1970).

Deep learning is a class of machine learning algorithms that use multiple layers of nonlinear processing neuron units for feature extraction. An adaptive neuro-fuzzy inference system (ANFIS) is a kind of deep learning algorithm that is a combination of the adaptive control technique, artificial neural network, and the fuzzy inference system. Fuzzy logic can represent the ambiguity of human perception or decision into a mathematical model (Zeung 1997). The Artificial neural network is an implemented algorithm that performs “learning”. The neuro-fuzzy system is the combination of the two theories, which enables learning in terms of human-like reasoning. Adding the adaptive control technique minimizes the errors between the original data and the estimate by the neuro-fuzzy system.

In this study, an output-based neuro controller was built based on the idea of the ANFIS and its capabilities in response simulation, data cleansing and restoration capability were verified using measured data from actual structural testing. Forced vibration responses of a five-story steel building was simulated by ANFIS and accuracy of simulations was compared with the results of the recurrent neural network, which is a family of traditional artificial neural networks. In addition, a capability of the ANFIS was extended to up-sampling of data with a lower sampling rate. Missing information in high frequency range was reproduced by ANFIS based on learned knowledge from previous training. Lastly, learning and prediction capability were applied to data cleansing. Contaminated data with noise collected from RC column cyclic load tests were corrected by means of ANFIS.

## 2 Deep Learning Algorithm

### 2.1 RNN (Recurrent Neural Network)

_{i}contains the accumulated knowledge acquired from a training dataset. When a new input is provided, the artificial neurons test similarity with stored information and indicate the result. This process can be stated mathematically as follows: The confluence operator calculates a weighted sum of the new input, and the activation function generates the output by comparing the value with the threshold. The activation function

*h*(

*x*) can be expressed as Eq. 1.

The functions of the neural network are two-fold, which are ‘output generation’ stated above and ‘learning’. The learning is a process of determining the adjustment \( \Delta w_{i} \) to the current weighting vector \( w_{i} \) representing learned experience and knowledge up to now. The methods of learning fall into two classes: supervised learning and unsupervised learning. In supervised learning, training data are provided so that the relationship between input and output can be inferred and applied to predict output to new input. The backpropagation algorithm that is widely used for current neural network systems is a kind of supervised learning (Fahlman 1988; Becker 1988).

*y*

_{d}(

*t*) and the output

*y*(

*t*) of the neurons is fed back to the network to optimize the weights in each neuron to minimize the error.

However, since the neuron structure in Fig. 2 performs ‘learning’ from each data set, simulating time-dependent (or order-dependent) events in which current response is affected by the previous ones is not feasible. Thus, a form of recurrent neural networks (RNN) was developed for modeling of a system whose property changes with time. The issue of many recent type of researches in ANN is how to improve the performance by adopting a new structure or adding another algorithm to deal with sequential information data (Mikolov 2010; Zaremba et al. 2014; Sak et al. 2014; Barbounis et al. 2006).

### 2.2 ANFIS (Adaptive Neuro-Fussy Inference System)

Fuzzy logic is an intelligent controller that simulates human behaviors by incorporating ‘If–then rules’ into the system, which contains human experience or knowledge (Mendel 2001; Klir and Yuan 1995; Lee 1990; Ross 2009). Fuzzy logic control (FLC) has been widely used in industries. A well-known example is a human-in-the-loop system that is applied for vehicle control, flight control, and automation of startup and shutdown of large chemical processes. Operation of such machines or factory by the human is actually of a nonlinear control based on know-hows of operators or engineers. The human-in-the-loop system is intended to mimic the human behaviors.

Since the purpose of artificial neural networks and the fuzzy logic is identically to simulate the human memory and cognition, some researches tried to solve this problem by combining the two methods. Gupta limited the inputs *x*_{a}(*t*) and weights *ω*_{a}(*t*) to [0, 1]^{n+1} to obtain the results similar to those from the fuzzy set (Gupta and Saridis 1977). Yamakawa identified the nonlinear system by calculating the weights considering the correlation between only two adjacent neurons rather than those of all neurons (Yamakawa 1993).

*e*(

*k*) =

*y*(

*k*) −

*y*

_{m}(

*k*) vanishes is known, the control function

*u*(

*k*) and output

*y*(

*k*) can be derived easily as Eqs. 3 and 4. Here,

*y*

_{m}(

*k*) indicates the output of the model.

*e*(

*k*) is used instead. When the fuzzy logic is adopted, \( \hat{g}[ \cdots ] \) can be identified by a feedback-parallel model as in Fig. 4.

*δy*/

*δu*calculated based on the output of the fuzzy model. The essential advantage of ANFIS is that it can approximate arbitrary nonlinear functions. Therefore, ANFIS is considered as a universal estimator. In this study, the fuzzy model proposed by (Sugeno 1985; Silverman 2018; Bak and Son 2017) is used in ANFIS.

## 3 Performance Comparison of RNN and ANFIS

As stated in Sect. 2, the deep learning technique based on the artificial neural network is capable of approximating the system properties by finding the similarity in training data. Such ‘learning’ ability can be used to simulate the response of a system to arbitrary inputs. Especially, when the relationship between the input and response of the system is of a nonlinear fashion and its responses are dependent on the responses at the previous time, the RNN and ANFIS are accepted as viable algorithms. In this section, performances of simulation (i.e. response prediction) by RNN and ANFIS were compared using actual vibration data measured from a real building. Ambient vibration measurement and forced vibration testing were performed on the building (Heo et al. 2017a, b; Lee et al. 2017). Measured acceleration data were provided for learning, and then simulations by RNN and ANFIS, respectively, were made subsequently. Simulated responses were compared with actual response measurement.

Measured data of ambient vibration during strong winds and forced vibration testing were used for “learning” of the neural networks. Forced vibration testing consists of white noise excitation and harmonic excitation at 0.96 Hz which corresponds to the natural frequency of the third mode. The neural networks were configured with four inputs, four outputs, and four in-depth networks. Vibrations of shakers were provided as the input and simulation of output (acceleration response at top floor) were made by the RNN and ANFIS, respectively. The number of neurons used for neural networks were determined by trial and error. Simulations using from two layers to ten layers were made and it was decided to use four layers which is the minimum number of layers that generated sufficiently accurate results.

*x*

_{1}and

*x*

_{2}.

As can be seen from the comparison of Figs. 8, 9, the ANFIS showed better accuracy than RNN. The RMSE in the case of ANFIS was as low as 0.5 m/s^{2}, while those of RNN ranged from 0.15 to 0.57 m/s^{2}. In the case of RNN, deviations of simulated data from measured ones can be seen more explicitly in the frequency domain. The peak of PSD appeared around 0.96 Hz which is the natural frequency of the structure. However, the shape of the PSD did not exactly match with the measured ones. Especially, as can be seen in (c2) graph representing the PSD of the response in a perpendicular direction to the direction of excitation, the simulated response has one single peak while actual measurement showed two closely spaced peaks. Since the frequency contents of the simulation are different from the measured data, it can be concluded that the simulation by RNN is not complete. On the contrary, simulations by ANFIS as shown in Fig. 9 showed an almost perfect match for all cases. The PSD plot in c2 graph has two peaks and even the small ripples outside the dominant frequency were successfully reproduced.

The problem of ANN-based simulation or control is that the identification process is black-boxed and thus the cause of failure cannot be found easily. The meaning of the weight vectors connected to the neural network is hard to understand. When the results are unsatisfactory, one may try with new training data set and/or a different number of neuron layers. The intrinsic weakness of RNN is that it constructs the system (learning) mainly to find a dominant relationship in the training data. Thus, it is difficult to correspond to multiple modes and non-dominant parts. The neuro-fuzzy identifier incorporated in ANFIS has the capability of complementing this weakness by handling mathematically ambiguous parts with the fuzzy logic.

## 4 Applications of ANFIS

It was showed in the previous section that the ANFIS has better performance than RNN. In this section, applications of ANFIS to both dynamic data and static data are represented.

### 4.1 Restoration of Lost Information in Dynamic Data

In dynamic analysis or testing, the important values extracted from the data would be peak values and their frequencies. Generally, measurements during dynamic testing are collected at a much higher sampling frequency than the natural frequencies of the structure. Accordingly, contents contained in high frequency range have relatively little importance when the behavior of the structure is the main concern of the test. Thus, it is customary to decimate the original data to a low sampling frequency for system identification analyses. However, frequency contents in the high frequency range may become important when the interactions between the building and nonstructural components are investigated since the natural frequencies of some nonstructural components are much higher than those of the building structures.

It is well known that the lost information due to the decimation of the signal to a lower sampling frequency cannot be recovered. Frequency contents of signals at a higher sampling rate by simple interpolation have completely different from those of the original signal. However, the lost information in high frequency range can be reproduced by using the ANFIS. In this section, the signal restoration capability of ANFIS is demonstrated using the same dataset as in Sect. 3. A channel of measured floor acceleration at the test building was intentionally decimated to 1/20 of the original sampling rate. The ANFIS trained in Sect. 3 was reconfigured to produce up-sampled data from decimated data.

The procedure of data restoration in this Section is similar to the response simulation in Sect. 3. Difference between two applications is the structure of input and output of the ANFIS. Once the neuro-fuzzy inference system was trained properly for response simulation, lost information in high frequency range can be successfully recovered with simple modifications in input and output structures in ANFIS, applying the rules acquired by learning.

^{2}. Low frequency range below 10 Hz, which affects the response of building structures, was restored to almost the same as the original. The high frequency band above 10 Hz showed some differences. This is seemingly caused because the ANFIS was trained using response data of a building rather than ground motions. However, considering the uncertainty of ground motions, which may vary with fault distance and soil property etc., reproduced one seems acceptable when real data are not available.

### 4.2 Data Cleansing of Static Data—RC Column Cyclic Load Test

## 5 Conclusion

The adaptive neuro-fuzzy inference system (ANFIS) is a family of deep learning algorithm, which incorporates the benefits of adaptive control technique, artificial neural network, and the fuzzy inference system. Fuzzy logic can represent ambiguity of human perception or decision into a mathematical model. The artificial neural network is an implemented algorithm that performs “learning”. The neuro-fuzzy system is the combination of the two theories, which enables learning in terms of human-like reasoning. Adding the adaptive control technique minimizes the errors between the original data and the estimate by the neuro-fuzzy system. Thus, the ANFIS is expected to produce very accurate predictions even for a highly nonlinear system. In this study, an output-based neuro controller was built based on the idea of the adaptive neuro-fuzzy inference system (ANFIS) and its capabilities in response simulation, data cleansing and restoration capability were verified using measurement data from actual structural testing.

For verification of response simulation, forced vibration responses of a five-story steel building were simulated by ANFIS and its accuracy was compared with the results of RNN, which is a type of traditional artificial neural networks. Simulations by ANFIS were very accurate with a much lower root means square error (RMSE) than RNN. Simulated data by ANFIS showed an almost perfect match with the original. Even the small ripples in PSD outside the dominant frequency were successfully reproduced.

In addition, the ANFIS was applied to increase the sampling rate of dynamic data. To this end, the ANFIS trained for the response simulation of test building was reconfigured. Measured vibration data of the test building were decimated to 1/20 of the original sampling rate and up-sampled. It was shown that missing high frequency contents were reproduced with small differences from the original.

Lastly, learning and prediction capability of ANFIS were applied in data cleansing. Contaminated data with noise collected from RC column cyclic load tests were corrected by means of ANFIS. The outliers were corrected effectively but the tendency of flattening the peak values were observed. The ANFIS corrected the outliers effectively but the tendency of flattening the peak values was observed, the latter could be complemented by additional use of the median filter.

## Notes

### Authors’ contributions

SH modeled ANFIS controller and performed simulation using measured data. ZC performed literature survey regarding ANN. EY was a major contributor in writing the manuscript. All authors read and approved the final manuscript.

### Acknowledgements

This research received financial support from Ministry of Land, Infrastructure and Transport of Korean government, Project No. 18AUDP-B106327-04 And the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIP) (No. NRF-2015R1A2A1A10054506). This support is gratefully acknowledged.

### Competing interests

The authors declare that they have no competing interests

### Availability of data and materials

Not applicable.

### Funding

Ministry of Land, Infrastructure and Transport of Korean government, National Research Foundation of Korea.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

- Bak, C., & Son, H. (2017). Development of prediction model for root industry production process using artificial neural network.
*Journal of the Korean Society for Precision Engineering, 34*(1), 23–27.CrossRefGoogle Scholar - Barbounis, T. G., Theocharis, J. B., Alexiadis, M. C., & Dokopoulos, P. S. (2006). Long-term wind speed and power forecasting using local recurrent neural network models.
*IEEE Transactions on Energy Conversion.,**21*(1), 273–284.CrossRefGoogle Scholar - Becker, S., & Le Cun, Y. (1988) Improving the convergence of back-propagation learning with second order methods. In
*Proceedings of the 1988 Connectionist Models Summer School*(pp. 29–37). San Matteo: Morgan Kaufmann.Google Scholar - Brown, M., & Harris, C. J. (1994).
*Neurofuzzy adaptive modelling and control*. Upper Saddle River: Prentice Hall.Google Scholar - Fahlman S. E. (1988). An empirical study of learning speed in back-propagation networks.Google Scholar
- Feng, G. (2006). A survey on analysis and design of model-based fuzzy control systems.
*IEEE Transactions on Fuzzy systems.,**14*(5), 676–697.CrossRefGoogle Scholar - Fu, K.-S. (1970). Learning control systems–Review and outlook.
*IEEE Transactions on Automatic Control,**15*(2), 210–221.MathSciNetCrossRefGoogle Scholar - Fu, K. S., Gonzalez, R., & Lee, C. S. G. (1987).
*Robotics: control sensing. Vis*. New York: Tata McGraw-Hill Education.Google Scholar - Gupta, M. M., & Saridis, G. N. (1977).
*Fuzzy automata and decision processes*. New York: North Holland.zbMATHGoogle Scholar - Heo, S. J., Park, H. H., Mun, D. H., & Lee, S. H. (2017a). System identification of a 5 story steel frame under ambient wind loads using RTK-GPS.
*Journal of the Wind Engineering Institute of Korea,**21*(1), 29–35.Google Scholar - Heo, S. J., Park, H. H., Mun, D. H., Lee, S. H. (2017b). System identification of five-story building on wind load using RD method. In
*Proceedings of WEIIK Symposium*(pp. 55–58).Google Scholar - Imam, A., Anifowose, F., & Azad, A. K. (2015). Residual strength of corroded reinforced concrete beams using an adaptive model based on ANN.
*International Journal of Concrete Structures and Materials, 9*(2), 159–172.CrossRefGoogle Scholar - Jang, J.-S. R. (1993). ANFIS: adaptive-network-based fuzzy inference system.
*IEEE Transactions on Systems, Man, and Cybernetics,**23*(3), 665–685.CrossRefGoogle Scholar - Klir, G., & Yuan, B. (1995).
*Fuzzy sets and fuzzy logic*. New Jersey: Prentice hall.zbMATHGoogle Scholar - Kurnaz, S., Cetin, O., & Kaynak, O. (2010). Adaptive neuro-fuzzy inference system based autonomous flight control of unmanned air vehicles.
*Expert Systems with Applications.,**37*(2), 1229–1234.CrossRefGoogle Scholar - Lee, C.-C. (1990). Fuzzy logic in control systems: fuzzy logic controller. I.
*IEEE Transactions on Systems, Man, and Cybernetics,**20*(2), 404–418.MathSciNetCrossRefGoogle Scholar - Lee, S. H., Jeon, M. J., Mun, D. H., Heo, S. J., Park, C. Y., & Woo, S. S. (2017). The multi-mode identification of the 5-story structure using active mass damper.
*Proceedings of AIK Symposium,**37*(2), 696–697.Google Scholar - Mendel, J. M. (2001).
*Uncertain rule-based fuzzy logic systems: introduction and new directions*. Upper Saddle River: Prentice Hall PTR.zbMATHGoogle Scholar - Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010). Recurrent neural network-based language model. In
*Eleventh Annual Conference of the International Speech Communication Association*.Google Scholar - Ross, T. J. (2009).
*Fuzzy logic with engineering applications*. Hoboken: Wiley.Google Scholar - Sak. H., Senior. A., & Beaufays. F. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In
*Fifteenth Annual Conference of the International Speech Communication Association*. 2014.Google Scholar - Saridis, G. N., & Lee, C.-S. G. (1979). An approximation theory of optimal control for trainable manipulators.
*IEEE Transactions on Systems, Man, and Cybernetics,**9*(3), 152–159.MathSciNetCrossRefGoogle Scholar - Sathyan, D., Anand, K. B., Prakash, A. J., & Premjith, B. (2018). Modeling the fresh and hardened stage properties of self-compacting concrete using random kitchen sink algorithm.
*International Journal of Concrete Structures and Materials, 12*(1), 24.CrossRefGoogle Scholar - Schalkoff, R. J. (1997).
*Artificial neural networks*. New York: McGraw-Hill.zbMATHGoogle Scholar - Silverman, B. W. (2018).
*Density estimation for statistics and data analysis*. Abringdon: Routledge.CrossRefGoogle Scholar - Sugeno, M. (1985).
*Industrial applications of fuzzy control*. Amsterdam: Elsevier Science Inc.zbMATHGoogle Scholar - Yamakawa, T. (1993). A fuzzy inference engine in nonlinear analog mode and its application to a fuzzy logic control.
*IEEE Transactions on Neural Networks.,**4*(3), 496–522.CrossRefGoogle Scholar - Zaremba, W,, Sutskever, I., Vinyals, O. (2014). Recurrent neural network regularization. arXiv preprint arXiv:1409.2329.
- Zeung, N. B. (1997).
*Fuzzy logic control*. South Korea: Hong Reung Science Publishing Co.Google Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.