Skip to main content

Uncovering Multivariate Structural Dependency for Analyzing Irregularly Sampled Time Series

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14173))

  • 908 Accesses

Abstract

Predictive analytics on Irregularly Sampled Multivariate Time Series (IS-MTS) presents a challenging problem in many real-world applications. Previous methods have primarily focused on incorporating temporal information into prediction while little effort is made to exploit the intrinsic structural information interchange among different IS-MTS at the same or across different timestamps. Recent developments in graph-based learning have shown promise in modeling spatial and structural dependencies of graph data. However, when applied to IS-MTS, they face significant challenges due to the complex data characteristics: 1) variable time intervals between observations; 2) asynchronous time points across dimensions resulting in missing values; 3) a lack of prior knowledge of connectivity structure for information propagation. To address these challenges, we propose a multivariate temporal graph network that coherently captures structural interactions, learns time-aware dependencies, and handles challenging characteristics of IS-MTS data. Specifically, we first develop a multivariate interaction module that handles the frequent missing values and adaptively extracts graph structural relations using a novel reinforcement learning module. Second, we design a correlation-aware neighborhood aggregation mechanism to capture within and across time dependencies and structural interactions. Third, we construct a novel masked time-aware self-attention to explicitly consider timestamp information and interval irregularity for determining optimal attention weights and distinguishing the influence of observation embeddings. Based on an extensive experimental evaluation, we demonstrate that our method outperforms a variety of competitors for the IS-MTS classification task.

Z. Wang and T. Jiang—These authors contributed equally.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  2. Che, Z., Purushotham, S., Cho, K., Sontag, D., Liu, Y.: Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8(1), 6085 (2018)

    Article  Google Scholar 

  3. Baytas, I.M., Xiao, C., Zhang, X., Wang, F., Jain, A.K., Zhou, J.: Patient subtyping via time-aware LSTM networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 65–74 (2017)

    Google Scholar 

  4. Hong, S., et al.: Holmes: health online model ensemble serving for deep learning models in intensive care units. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1614–1624 (2020)

    Google Scholar 

  5. Wang, Q., et al.: BiT-MAC: mortality prediction by bidirectional time and multi-feature attention coupled network on multivariate irregular time series. Comput. Biol. Med. 155, 106586 (2023)

    Article  Google Scholar 

  6. Horn, M., Moor, M., Bock, C., Rieck, B., Borgwardt, K.: Set functions for time series. In: International Conference on Machine Learning, pp. 4353–4363. PMLR (2020)

    Google Scholar 

  7. Mulyadi, A.W., Jun, E., Suk, H.I.: Uncertainty-aware variational-recurrent imputation network for clinical time series. IEEE Trans. Cybern. 52(9), 9684–9694 (2021)

    Article  Google Scholar 

  8. Wang, Y., Min, Y., Chen, X., Wu, J.: Multi-view graph contrastive representation learning for drug-drug interaction prediction. In: Proceedings of the Web Conference, pp. 2921–2933 (2021)

    Google Scholar 

  9. Li, J., et al.: Predicting path failure in time-evolving graphs. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1279–1289 (2019)

    Google Scholar 

  10. Huang, Z., Sun, Y., Wang, W.: Coupled graph ode for learning interacting system dynamics. In: The 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2021)

    Google Scholar 

  11. Silva, I., Moody, G., Scott, D.J., Celi, L.A., Mark, R.G.: Predicting in-hospital mortality of ICU patients: the physionet/computing in cardiology challenge 2012. In: 2012 Computing in Cardiology, pp. 245–248. IEEE (2012)

    Google Scholar 

  12. Reyna, M.A., et al.: Early prediction of sepsis from clinical data: the PhysioNet/Computing in Cardiology Challenge 2019. In: 2019 Computing in Cardiology. IEEE (2019)

    Google Scholar 

  13. Reiss, A., Stricker, D.: Introducing a new benchmarked dataset for activity monitoring. In: 2012 16th International Symposium on Wearable Computers, pp. 108–109. IEEE (2012)

    Google Scholar 

  14. Zhang, X., Zeman, M., Tsiligkaridis, T., Zitnik, M.: Graph-guided network for irregularly sampled multivariate time series. In: International Conference on Learning Representations (2022)

    Google Scholar 

  15. Clark, J.S., Bjørnstad, O.N.: Population time series: process variability, observation errors, missing values, lags, and hidden states. Ecology 85(11), 3140–3150 (2004)

    Article  Google Scholar 

  16. Sezer, O.B., Gudelek, M.U., Ozbayoglu, A.M.: Financial time series forecasting with deep learning: a systematic literature review: 2005–2019. Appl. Soft Comput. 90, 106181 (2020)

    Article  Google Scholar 

  17. Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T. and Gao, J. Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, pp. 1903–1911 (2017)

    Google Scholar 

  18. Shickel, B., Tighe, P.J., Bihorac, A., Rashidi, P.: Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J. Biomed. Health Inform. 22(5), 1589–1604 (2017)

    Article  Google Scholar 

  19. Cao, D., Wang, Y., Duan, J., Zhang, C., Zhu, X., Huang, C., Tong, Y., Xu, B., Bai, J., Tong, J., Zhang, Q.: Spectral temporal graph neural network for multivariate time-series forecasting. Adv. Neural. Inf. Process. Syst. 33, 17766–17778 (2020)

    Google Scholar 

  20. Wang, D., et al.: Temporal-aware graph neural network for credit risk prediction. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 702–710. Society for Industrial and Applied Mathematics (2021)

    Google Scholar 

  21. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lió, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  22. Chen, Z., Villar, S., Chen, L., Bruna, J.: On the equivalence between graph isomorphism testing and function approximation with GNNs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  23. Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-GCN: geometric graph convolutional networks. In: International Conference on Learning Representations (2020)

    Google Scholar 

  24. Hallikainen, M., et al.: Interaction between cholesterol and glucose metabolism during dietary carbohydrate modification in subjects with the metabolic syndrome. Am. J. Clin. Nutr. 84(6), 1385–1392 (2006)

    Article  Google Scholar 

  25. Shukla, S.N., Marlin, B.M.: A survey on principles, models and methods for learning from irregularly sampled time series. arXiv preprint arXiv:2012.00168 (2020)

  26. Shukla, S.N., Marlin, B.: Multi-time attention networks for irregularly sampled time series. In: International Conference on Learning Representations (2020)

    Google Scholar 

  27. Tan, Q., et al.: Data-GRU: dual-attention time-aware gated recurrent unit for irregular multivariate time series. Proc. AAAI Conf. Artif. Intell. 34(01), 930–937 (2020)

    Google Scholar 

  28. Bahadori, M.T., Lipton, Z.C.: Temporal-clustering invariance in irregular healthcare time series. arXiv preprint arXiv:1904.12206 (2019)

  29. Zhang, Y.: ATTAIN: attention-based time-aware LSTM networks for disease progression modeling. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 4369–4375, Macao, China (2019)

    Google Scholar 

  30. Yin, C., Liu, R., Zhang, D., Zhang, P.: Identifying sepsis subphenotypes via time-aware multi-modal auto-encoder. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 862–872 (2020)

    Google Scholar 

  31. Yang, S., et al.: Financial risk analysis for SMEs with graph-based supply chain mining. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 4661–4667 (2021)

    Google Scholar 

  32. Wang, X., et al.: Traffic flow prediction via spatial temporal graph neural network. In: Proceedings of the Web Conference, pp. 1082–1092 (2020)

    Google Scholar 

  33. Yan, C., Gao, C., Zhang, X., Chen, Y., Malin, B.: Deep imputation of temporal data. In: 2019 IEEE International Conference on Healthcare Informatics (ICHI), pp. 1–3. IEEE (2019)

    Google Scholar 

  34. Kidger, P., Morrill, J., Foster, J., Lyons, T.: Neural controlled differential equations for irregular time series. Adv. Neural. Inf. Process. Syst. 33, 6696–6707 (2020)

    Google Scholar 

  35. Rubanova, Y., Chen, R.T., Duvenaud, D.K.: Latent ordinary differential equations for irregularly-sampled time series. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  36. Schirmer, M., Eltayeb, M., Lessmann, S., Rudolph, M.: Modeling irregular time series with continuous recurrent units. In: International Conference on Machine Learning, pp. 19388–19405. PMLR (2022)

    Google Scholar 

  37. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)

    Article  MATH  Google Scholar 

  38. Sun, Z., Sun, Z., Dong, W., Shi, J., Huang, Z.: Towards predictive analysis on disease progression: a variational Hawkes process model. IEEE J. Biomed. Health Inform. 25(11), 4195–4206 (2021)

    Article  Google Scholar 

  39. De Brouwer, E., Simm, J., Arany, A., Moreau, Y.: GRU-ODE-Bayes: continuous modeling of sporadically-observed time series. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  40. Oskarsson, J., Sidén, P., Lindsten, F.: Temporal graph neural networks for irregular data. In: International Conference on Artificial Intelligence and Statistics. PMLR (2023)

    Google Scholar 

Download references

Acknowledgement

This research is supported by the National Key R &D Program of China (Grant No. 2022YFF0608000), the Natural Science Foundation of China (No. 62172372, No. 62272487, No. 62076178), Zhejiang Provincial Natural Science Foundation (No. LZ21F030001) and Zhejiang Lab (K2023KG0AC02).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ji Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Z. et al. (2023). Uncovering Multivariate Structural Dependency for Analyzing Irregularly Sampled Time Series. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43424-2_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43423-5

  • Online ISBN: 978-3-031-43424-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics