Abstract
The Long Short-Term Memory (LSTM) model has been applied in recent years to handle time series data in multiple application domains, such as speech recognition and financial prediction. While the LSTM prediction model has shown promise in anomaly detection in previous research, uncorrelated features can lead to unsatisfactory analysis result and can complicate the prediction model due to the curse of dimensionality. This paper proposes a novel method of clustering and predicting multidimensional aircraft time series. The purpose is to detect anomalies in flight vibration in the form of high dimensional data series, which are collected by dozens of sensors during test flights of large aircraft. The new method is based on calculating the Spearman’s rank correlation coefficient between two series, and on a hierarchical clustering method to cluster related time series. Monotonically similar series are gathered together and each cluster of series is trained to predict independently. Thus series which are uncorrelated or of low relevance do not influence each other in the LSTM prediction model. The experimental results on COMAC’s (Commercial Aircraft Corporation of China Ltd) C919 flight test data show that our method of combining clustering and LSTM model significantly reduces the root mean square error of predicted results.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Cao, Z., Zhu, Y., et al.: Improving prediction accuracy in LSTM network model for aircraft testing flight data. In: IEEE International Conference on Smart Cloud (2018)
Hsu, H., Hsieh, C.: Feature selection via correlation coefficient clustering. J. Softw. 5(12), 1371–1377 (2010)
Gauthier, T.: Detecting trends using spearman’s rank correlation coefficient. Environ. Forensics 2, 359–362 (2001)
Nanduri, A., Sherry, L.: Anomaly detection in aircraft data using recurrent neural networks. In: Integrated Communications Navigation and Surveillance (ICNS) Conference (2016)
Grabusts, P., Borisov, A.: Clustering methodology for time series mining. Sci. J. Riga Tech. Univ. 40(1), 81–86 (2009)
Singhal, A., Seborg, D.: Clustering multivariate time-series data. J. Chemom. 19, 427–438 (2005)
Funie, A.-I., Grigoras, P., Burovskiy, P., Luk, W., Salmon, M.: Run-time reconfigurable acceleration for genetic programming fitness evaluation in trading strategies. J. Signal Process. Sys. 90(1), 39–52 (2018)
Gai, K., Qiu, M., Zhao, H., et al.: Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing. J. Netw. Comput. Appl. 59, 46–54 (2016)
Bara, A., Niu, X., Luk, W.: A dataflow system for anomaly detection analysis. In: International Conference on Field Programmable Technology (2014)
Graves, A.: Generating sequences with recurrent neural networks. https://arxiv.org/abs/1308.0850
Cui, L., Luo, Y., Li, G., Lu, N.: Artificial bee colony algorithm with hierarchical groups for global numerical optimization. In: Qiu, M. (ed.) SmartCom 2016. LNCS, vol. 10135, pp. 72–85. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52015-5_8
Gai, K., Qiu, M., Liu, M., Zhao, H.: Smart resource allocation using reinforcement learning in content-centric cyber-physical systems. In: Qiu, M. (ed.) SmartCom 2017. LNCS, vol. 10699, pp. 39–52. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73830-7_5
Acknowledgment
This work is partially supported by National Key Research & Development Program of China (2017YFA0206104), Shanghai Municipal Science and Technology Commission and Commercial Aircraft Corporation of China, Ltd. (COMAC) (175111105000), Shanghai Municipal Science and Technology Commission (18511111302, 18511103502), Key Foreign Cooperation Projects of Bureau of International Co-operation Chinese Academy of Sciences (184131KYSB20160018) and UK EPSRC (EP/L016796/1, EP/N031768/1 and EP/P010040/1).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, H. et al. (2018). Correlation Coefficient Based Cluster Data Preprocessing and LSTM Prediction Model for Time Series Data in Large Aircraft Test Flights. In: Qiu, M. (eds) Smart Computing and Communication. SmartCom 2018. Lecture Notes in Computer Science(), vol 11344. Springer, Cham. https://doi.org/10.1007/978-3-030-05755-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-05755-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05754-1
Online ISBN: 978-3-030-05755-8
eBook Packages: Computer ScienceComputer Science (R0)