Correlation Coefficient Based Cluster Data Preprocessing and LSTM Prediction Model for Time Series Data in Large Aircraft Test Flights
The Long Short-Term Memory (LSTM) model has been applied in recent years to handle time series data in multiple application domains, such as speech recognition and financial prediction. While the LSTM prediction model has shown promise in anomaly detection in previous research, uncorrelated features can lead to unsatisfactory analysis result and can complicate the prediction model due to the curse of dimensionality. This paper proposes a novel method of clustering and predicting multidimensional aircraft time series. The purpose is to detect anomalies in flight vibration in the form of high dimensional data series, which are collected by dozens of sensors during test flights of large aircraft. The new method is based on calculating the Spearman’s rank correlation coefficient between two series, and on a hierarchical clustering method to cluster related time series. Monotonically similar series are gathered together and each cluster of series is trained to predict independently. Thus series which are uncorrelated or of low relevance do not influence each other in the LSTM prediction model. The experimental results on COMAC’s (Commercial Aircraft Corporation of China Ltd) C919 flight test data show that our method of combining clustering and LSTM model significantly reduces the root mean square error of predicted results.
KeywordsCluster Time series Correlation coefficient LSTM
This work is partially supported by National Key Research & Development Program of China (2017YFA0206104), Shanghai Municipal Science and Technology Commission and Commercial Aircraft Corporation of China, Ltd. (COMAC) (175111105000), Shanghai Municipal Science and Technology Commission (18511111302, 18511103502), Key Foreign Cooperation Projects of Bureau of International Co-operation Chinese Academy of Sciences (184131KYSB20160018) and UK EPSRC (EP/L016796/1, EP/N031768/1 and EP/P010040/1).
- 1.Cao, Z., Zhu, Y., et al.: Improving prediction accuracy in LSTM network model for aircraft testing flight data. In: IEEE International Conference on Smart Cloud (2018)Google Scholar
- 4.Nanduri, A., Sherry, L.: Anomaly detection in aircraft data using recurrent neural networks. In: Integrated Communications Navigation and Surveillance (ICNS) Conference (2016)Google Scholar
- 5.Grabusts, P., Borisov, A.: Clustering methodology for time series mining. Sci. J. Riga Tech. Univ. 40(1), 81–86 (2009)Google Scholar
- 9.Bara, A., Niu, X., Luk, W.: A dataflow system for anomaly detection analysis. In: International Conference on Field Programmable Technology (2014)Google Scholar
- 10.Graves, A.: Generating sequences with recurrent neural networks. https://arxiv.org/abs/1308.0850