Abstract
Multivariable time series classification problems are increasingly prevalent and complex across various domains, such as biology and finance. While deep learning methods are an effective tool for these problems, they often lack interpretability. In this work, we propose a modular prototype learning framework for multivariable time series classification. In the first stage of our framework, we use contrastive learning to train a set of long short-term memory (LSTM) encoders for each variable. We then use prototype matching layers to learn representative prototypes for the variables that contribute to the classification task. The visualization of these prototypes offers an interpretation of the representative patterns within each variable. We validated our framework on a simulated dataset with embedded patterns. On this simulated dataset, we found that our model produces interpretations consistent with the embedded patterns. Overall, our findings pave the road for constructing interpretable deep learning-based models for time series classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Schäfer, P., Leser, U. (2017). Multivariate time series classification with weasel+ muse. arXiv preprint arXiv:1711.11343
Kwon, D.-H., Kim, J.-B., Heo, J.-S., Kim, C.-M., & H, Y-H. (2019). Time series classification of cryptocurrency price trend based on a recurrent lstm neural network. Journal of Information Processing Systems, 15(3), 694–706.
Xu, H., Duan, Z., Bai, Y., Huang, Y., Ren, A., Yu, Q., Zhang, Q., Wang, Y., Wang, X., Sun, Y., & Wang, W. (2020). Multivariate time series classification with hierarchical variational graph pooling
Azar, J., Makhoul, A., & Couturier, R. (2020). Using densenet for iot multivariate time series classification. In 2020 IEEE Symposium on Computers and Communications (ISCC), (pp. 1–6). IEEE
Gupta, A., Gupta, H.P., Biswas, B., & Dutta, T. (2020). A divide-and-conquer-based early classification approach for multivariate time series with different sampling rate components in iot. ACM Trans. Internet Things, 1(2).
Zhai, B., Perez-Pozuelo, I., Clifton, E.A.D., Palotti, J., Guan, Y. (2020). Making sense of sleep: Multimodal sleep stage classification in a large, diverse population using movement and cardiac sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 4(2).
Ayata, D., Yaslan, Y., Kamasak, M.E. (2020). Emotion recognition from multimodal physiological signals for emotion aware healthcare systems. Journal of Medical and Biological Engineering, 1–9.
Yixiang, D., Xue, W., Pengbo, Z., & Weihang, Z. (2017). Wearable biosensor network enabled multimodal daily-life emotion recognition employing reputation-driven imbalanced fuzzy classification. Measurement, 109, 408–424.
Lee, J. H., Gamper, H., Tashev, I., Dong, S., Ma, S., Remaley, J., Holbery, J. D., & Yoon, S. H. (2020). Stress monitoring using multimodal bio-sensing headset. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA ’20 (pp. 1–7). Association for Computing Machinery: New York, NY, USA.
Ziegler, D. A., Simon, A. J., Gallen, C. L., Skinner, S., Janowich, J. R., Volponi, J. J., Rolle, C. E., Mishra, J., Kornfield, J., Anguera, J. A., et al. (2019). Closed-loop digital meditation improves sustained attention in young adults. Nature Human Behaviour, 3(7), 746–757.
Mishra, J., Lowenstein, M., Campusano, R., Hu, Y., Diaz-Delgado, J., Ayyoub, J., Jain, R., & Gazzaley, A. (2021). Closed-loop neurofeedback of alpha synchrony during goal-directed attention. Journal of Neuroscience.
Wen, D., Liang, B., Zhou, Y., Chen, H., Jung, T.P. (2020). The current research of combining multi-modal brain-computer interfaces with virtual reality. IEEE Journal of Biomedical and Health Informatics, 1–1.
Jung, Y., Ik Yoon, Y. (2016). Monitoring senior wellness status using multimodal biosensors. In 2016 International Conference on Big Data and Smart Computing (BigComp) (pp. 435–438). IEEE.
Jung, Y., & Ik Yoon, Y. (2017). Multi-level assessment model for wellness service based on human mental stress level. Multimedia Tools and Applications, 76(9), 11305–11317.
Zhang, D., Yao, L., Zhang, X., Wang, S., Chen, W., Boots, R., & Benatallah, B. (2018). Cascade and parallel convolutional recurrent neural networks on eeg-based intention recognition for brain computer interface. In Proceedings of the AAAI Conference on Artificial Intelligence, 32.
Abbasi-Asl, R., & Yu, B. (2017). Structural compression of convolutional neural networks. arXiv preprint arXiv:1705.07356.
Abbasi-Asl, R., Chen, Y., Bloniarz, A., Oliver, M., Willmore, B.D.B, Gallant, J.L., & Yu, B. (2018). The deeptune framework for modeling and characterizing neurons in visual cortex area v4. bioRxiv, pp. 465534.
James Murdoch, W., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019). Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44), 22071–22080.
Gokce Baydogan, M., & Runger, G. (2015). Learning a symbolic representation for multivariate time series classification. Data Mining and Knowledge Discovery, 29(2), 400–422.
Grabocka, J., Wistuba, M., & Schmidt-Thieme, L. (2016). Fast classification of univariate and multivariate time series through shapelet discovery. Knowledge and Information Systems, 49(2), 429–454.
Ghalwash, M. F., & Obradovic, Z. (2012). Early classification of multivariate temporal observations by extraction of interpretable shapelets. BMC Bioinformatics, 13(1), 1–12.
Tang, W., Liu, L., & Long, G. (2020) Interpretable time-series classification on few-shot samples. In 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). IEEE.
Gee, A.H., Garcia-Olano, D., Ghosh, J., & Paydarfar, D. (2019) Explaining deep classification of time-series data with learned prototypes. arXiv preprint arXiv:1904.08935.
Zhang, X., Gao, Y., Lin, J., & Lu, C-T. (2020). Tapnet: Multivariate time series classification with attentional prototypical network. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020 (pp. 6845–6852). AAAI Press.
Li, G., Choi, B., & Jianliang, X. (2021). Sourav S Bhowmick, Kwok-Pan Chun, and Grace LH Wong. Shapenet: A shapelet-neural network approach for multivariate time series classification. AAAI.
Hadsell, R., Chopra, S., & LeCun, Y. (2006) Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) (vol. 2, pp. 1735–1742). IEEE.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, Sasank Chilamkurthy, & A., Steiner, B., Fang, L., Bai, J., & Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 32 (pp. 8024–8035). Curran Associates Inc.
McInnes, L., Healy, J., & Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.
Navani, D., Block, V., Cree, B., & Abbasi-Asl, R. (2023). Diurnal step count patterns in progressive multiple sclerosis (p3-3.011).
Acknowledgements
We gratefully acknowledge the feedback and comments provided by the members of the Neuroscape Center at UCSF.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Ghosal, G.R., Abbasi-Asl, R. (2024). Interpretable Prototype Discovery in Deep Learning-Based Time Series Classification. In: Alam, MR., Fathi, M. (eds) Integrated Systems: Data Driven Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-53652-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-53652-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53651-9
Online ISBN: 978-3-031-53652-6
eBook Packages: EngineeringEngineering (R0)