Multivariate Time Series Early Classification with Interpretability Using Deep Learning and Attention Mechanism

  • En-Yu Hsu
  • Chien-Liang Liu
  • Vincent S. TsengEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11441)


Multivariate time-series early classification is an emerging topic in data mining fields with wide applications like biomedicine, finance, manufacturing, etc. Despite of some recent studies on this topic that delivered promising developments, few relevant works can provide good interpretability. In this work, we consider simultaneously the important issues of model performance, earliness, and interpretability to propose a deep-learning framework based on the attention mechanism for multivariate time-series early classification. In the proposed model, we used a deep-learning method to extract the features among multiple variables and capture the temporal relation that exists in multivariate time-series data. Additionally, the proposed method uses the attention mechanism to identify the critical segments related to model performance, providing a base to facilitate the better understanding of the model for further decision making. We conducted experiments on three real datasets and compared with several alternatives. While the proposed method can achieve comparable performance results and earliness compared to other alternatives, more importantly, it can provide interpretability by highlighting the important parts of the original data, rendering it easier for users to understand how the prediction is induced from the data.


Early classification on time-series Deep neural network Attention 



This research was partially supported by Ministry of Science and Technology, Taiwan, under grant no. 107-2218-E-009-005 and 107-2218-E-009-050.


  1. 1.
    Chen, Y., Zhao, D., Lv, L., Li, C.: A visual attention based convolutional neural network for image classification. In: 2016 12th World Congress on Intelligent Control and Automation (WCICA), pp. 764–769. IEEE (2016)Google Scholar
  2. 2.
    Cui, Z., Chen, W., Chen, Y.: Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995 (2016)
  3. 3.
    Ding, H., Trajcevski, G., Scheuermann, P., Wang, X., Keogh, E.: Querying and mining of time series data: experimental comparison of representations and distance measures. Proc. VLDB Endow. 1(2), 1542–1552 (2008)CrossRefGoogle Scholar
  4. 4.
    Ghalwash, M.F., Ramljak, D., Obradović, Z.: Early classification of multivariate time series using a hybrid HMM/SVM model. In: 2012 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1–6. IEEE (2012)Google Scholar
  5. 5.
    Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L.: Learning time-series shapelets. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 392–401. ACM (2014)Google Scholar
  6. 6.
    He, Q., Dong, Z., Zhuang, F., Shang, T., Shi, Z.: Fast time series classification based on infrequent shapelets. In: 2012 11th International Conference on Machine Learning and Applications (ICMLA), vol. 1, pp. 215–219. IEEE (2012)Google Scholar
  7. 7.
    Huang, H.S., Liu, C.L., Tseng, V.S.: Multivariate time series early classification using multi-domain deep neural network. In: 2018 IEEE International Conference on Data Science and Advanced Analytics (DSAA) (2018)Google Scholar
  8. 8.
    Hübner, R., Steinhauser, M., Lehle, C.: A dual-stage two-phase model of selective attention. Psychol. Rev. 117(3), 759 (2010)CrossRefGoogle Scholar
  9. 9.
    Kadous, M.W., et al.: Temporal Classification: Extending The Classification Paradigm to Multivariate Time Series. University of New South Wales, Kensington (2002)Google Scholar
  10. 10.
    Lin, Y.-F., Chen, H.-H., Tseng, V.S., Pei, J.: Reliable early classification on multivariate time series with numerical and categorical attributes. In: Cao, T., Lim, E.-P., Zhou, Z.-H., Ho, T.-B., Cheung, D., Motoda, H. (eds.) PAKDD 2015. LNCS (LNAI), vol. 9077, pp. 199–211. Springer, Cham (2015). Scholar
  11. 11.
    Lipton, Z.C.: The mythos of model interpretability. arXiv preprint arXiv:1606.03490 (2016)
  12. 12.
    Liu, Q., Yu, F., Wu, S., Wang, L.: Mining significant microblogs for misinformation identification: an attention-based approach. ACM Trans. Intell. Syst. Technol. 9(5), 50:1–50:20 (2018). Scholar
  13. 13.
    Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., Cottrell, G.: A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971 (2017)
  14. 14.
    Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)Google Scholar
  15. 15.
    Wang, W., Chen, C., Wang, W., Rai, P., Carin, L.: Earliness-aware deep convolutional networks for early time series classification. arXiv preprint arXiv:1611.04578 (2016)
  16. 16.
    Xing, Z., Pei, J., Dong, G., Yu, P.S.: Mining sequence classifiers for early prediction. In: Proceedings of the 2008 SIAM International Conference on Data Mining, pp. 644–655. SIAM (2008)Google Scholar
  17. 17.
    Xing, Z., Pei, J., Philip, S.Y.: Early prediction on time series: a nearest neighbor approach. In: IJCAI, pp. 1297–1302. Morgan Kaufmann (2009)Google Scholar
  18. 18.
    Xing, Z., Pei, J., Philip, S.Y.: Early classification on time series. Knowl. Inf. Syst. 31(1), 105–127 (2012)CrossRefGoogle Scholar
  19. 19.
    Xing, Z., Pei, J., Yu, P.S., Wang, K.: Extracting interpretable features for early classification on time series. In: Proceedings of the 2011 SIAM International Conference on Data Mining, pp. 247–258. SIAM (2011)Google Scholar
  20. 20.
    Ye, L., Keogh, E.: Time series shapelets: a new primitive for data mining. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 947–956. ACM (2009)Google Scholar
  21. 21.
    Zheng, Y., Liu, Q., Chen, E., Ge, Y., Zhao, J.L.: Time series classification using multi-channels deep convolutional neural networks. In: Li, F., Li, G., Hwang, S., Yao, B., Zhang, Z. (eds.) WAIM 2014. LNCS, vol. 8485, pp. 298–310. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceNational Chiao Tung UniversityHsinchuTaiwan, ROC
  2. 2.Department of Industrial Engineering and ManagementNational Chiao Tung UniversityHsinchuTaiwan, ROC

Personalised recommendations