Abstract
Time series averages are one key input to temporal data mining techniques such as classification, clustering, forecasting, etc. In practice, the optimality of estimated averages often impacts the performance of such temporal data mining techniques. Practically, an estimated average is presumed to be optimal if it minimizes the discrepancy between itself and members of an averaged set while preserving descriptive shapes. However, estimating an average under such constraints is often not trivial due to temporal shifts. To this end, all pioneering averaging techniques propose to align averaged series before estimating an average. Practically, the alignment gets performed to transform the averaged series, such that, after the transformation, they get registered to their arithmetic mean. However, in practice, most proposed alignment techniques often introduce additional challenges. For instance, Dynamic Time Warping (DTW)-based alignment techniques make the average estimation process non-smooth, non-convex, and computationally demanding. With such observation in mind, we approach time series averaging as a generative problem. Thus, we propose to mimic the effects of temporal alignment in the latent space of multi-tasking neural networks. We also propose to estimate (augment) time domain averages from the latent space representations. With this approach, we provide state-of-the-art latent space registration. Moreover, we provide time domain estimations that are better than the estimates generated by some pioneering averaging techniques.
Similar content being viewed by others
Change history
19 September 2023
A Correction to this paper has been published: https://doi.org/10.1007/s10115-023-01981-9
Notes
The Python implementation of the proposed architectures, the training setup, and 1NCC evaluation can be found at the following github repository: https://github.com/tsegaterefe/Estimating-Time-Series-Averages-from-Latent-Space-of-Multi-tasking-Neural-Networks/tree/main.
References
Aghabozorgi S, Shirkhorshidi AS, Wah TY (2015) Time-series clustering: a decade review. Inf Syst 53:16–38
Bagnall A, Davis L, Hills J, Lines J (2012) Transformation based ensembles for time series classification. In: Proceedings of the 2012 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, Anaheim,CA, USA, pp 307–318
Bagnall A, Lines J (2014) An experimental evaluation of nearest neighbour time series classification. Technical report, University of East Angelina arXiv:1406.4757
Bock H-H (2008) Origins and extensions of the -means algorithm in cluster analysis. Journal Électronique d’Histoire des Probabilités et de la Statistique [electronic only] 4:1–18
Bulteau L, Froese V, Niedermeier R (2020) Tight hardness results for consensus problems on circular strings and time series. SIAM J. Discrete Math. 34(3):1854–1883
Chen C, Srivastava A (2021) Srvfregnet: elastic function registration using deep neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops. IEEE Computer Society, New Orleans, Louisiana, USA, pp 4462–4471
Chen Y, Keogh E, Hu B, Begum N, Bagnall A, Mueen A, Batista G (2015) The UCR time series classification archive. www.cs.ucr.edu/~eamonn/time_series_data/
Christian S, Liu W, Jia Y, Pierre S, Scott R, Dragomir A, Dumitru E, Vincent V, Andrew R (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, Boston, MA, USA, pp 1–9
Cuturi M, Blondel M (2017) Soft-dtw: a differentiable loss function for time-series. In: Proceedings of the 34th international conference on machine learning. JMLR.org, Sydney, NSW, Australia, pp 894–903
Debella TT, Shawel BS, Devanne M, Weber J, Woldegebreal DH, Pollin S, Forestier G (2022) Deep representation learning for cluster-level time series forecasting. In: 8th International conference on time series and forecasting (ITISE). MDPI, Gran Canaria, Spain, pp 1–11
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Der Maaten LV, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605
Detlefsen NS, Freifeld O, Hauberg S (2018) Deep diffeomorphic transformer networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). In: IEEE Computer Society, Lake City, UT, USA, pp 4403–4412
Diederik PK, Max W (2014) Auto-encoding variational bayes. In: 2nd International conference on learning representations (ICLR 2014). ICLR, Banff, AB, Canada, pp 1–14
Dong G, Liao G, Liu H, Kuang G (2018) A review of the autoencoder and its variants: a comparative perspective from target recognition in synthetic-aperture radar images. IEEE Geosci Remote Sens Mag 6(3):44–68
Fawaz HI, Benjamin L, Forestier G, Charlotte P, Schmidt DF, Weber J, Webb GI, Idoumghar L, Muller PA, Petitjean F (2020) Inceptiontime: finding alexnet for time series classification. Data Min Knowl Disc 34:1936–1962
Fawaz HI, Forestier G, Weber J, Idoumghar L, Alain-Muller P (2019) Deep learning for time series classification: a review. Data Min Knowl Disc 33(4):917–963
Gee AH, Garcia-Olano D, Ghosh J, Paydarfar D (2019) Explaining deep classification of time-series data with learned prototypes. CEUR Workshop Proc 2429:15–22
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th conference on artificial intelligence and statistics. PMLR, Chia Laguna Resort, Sardinia, Italy, pp 249–256
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks, pp 1–9
Gupta L, Molfese D, Tammana R, Simos P (1996) Nonlinear alignment and averaging for estimating the evoked potential. IEEE Trans Biomed Eng 43(4):348–356
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, Las Vegas, NV, USA, pp 770–778
Itakura F (1975) Minimum prediction residual principle applied to speech recognition. IEEE Trans Acoust Speech Signal Process 23(1):67–72
Iwana BK, Uchida S (2021) An empirical survey of data augmentation for time series classification with neural networks. PLOS ONE 16(7):1–32
Jain B, Froese V, Schultz D (2019) An average-compress algorithm for the sample mean problem under dynamic time warping. CoRR arXiv:abs/1909.13541
Jain J, Schultz D (2018) Asymmetric learning vector quantization for efficient nearest neighbor classification in dynamic time warping spaces. Pattern Recogn 76:349–366
Junyuan X, Ross G, Ali F (2016) Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33rd international conference on international conference on machine learning, vol 48, pp 478–487
Kaiming H, Xiangyu Z, Shaoqing R, Jian S (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE international conference on computer vision (ICCV). IEEE Computer Society, Santiago, Chile, pp 1026–1034
Kowsar Y, Moshtaghi M, Velloso E, Bezdek JC, Kulik L, Leckie C (2022) Shape-sphere: a metric space for analysing time series by their shape. Inf Sci 582:198–214
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. Neural Information Processing Systems Foundation Inc. (NeurIPS), Lake Tahoe, Nevada, USA, pp 1106–1114
Lafabregue B, Weber J, Gançarski P, Forestier G (2021a) End-to-end deep representation learning for time series clustering: a comparative study. In: Data mining and knowledge discovery, pp 1–53
Lafabregue B, Weber J, Gançarski P, Forestier G (2021) End-to-end deep representation learning for time series clustering: a comparative study. Data Min Knowl Disc 36:29–81
Lin J, Keogh E, Wei L, Lonardi S (2007) Experiencing sax: a novel symbolic representation of time series. Data Min Knowl Disc 15(2):107–144
Lin J, Li Y (2009) Finding structural similarity in time series data using bag-of-patterns representation. In: International conference on scientific and statistical database management. Springer, New Orleans, LA, USA, pp 461–477
Lines J (2015) Time series classification through transformation and ensembles. Ph.D. Thesis, School of Electrical and Computer Engineering, University of East Anglia
Niennattrakul V, Ratanamahatana CA (2009) Shape averaging under time warping. In: 2009 6th international conference on electrical engineering/electronics, computer, telecommunications and information technology. IEEE, Chonburi, Thailand, pp 626–629
Niennattrakul V, Srisai D, Ratanamahatana CA (2012) Shape-based template matching for time series data. Knowl Based Syst 26:1–8
Ongwattanakul S, Srisai D (2009) Contrast enhanced dynamic time warping distance for time series shape averaging classification. In: Proceedings of the 2nd international conference on interaction sciences: information technology, culture and human. Association for Computing Machinery, pp 976–981
Paparrizos J, Gravano L (2015) K-shape: Efficient and accurate clustering of time series. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data. Association for Computing Machinery, Melbourne, Victoria, Australia, pp 1855–1870
Petitjean F, Gançarski P (2012) Summarizing a set of time series by averaging: From Steiner sequence to compact multiple alignment. Theoret Comput Sci 414(1):76–91
Petitjean F, Ketterlin A, Gançarski P (2011) A global averaging method for dynamic time warping, with applications to clustering. Pattern Recogn 44(3):678–693
Ruiz EV, Casacuberta Nolla F, Segovia HR (1985) Is the DTW “distance’’ really a metric? An algorithm reducing the number of DTW comparisons in isolated word recognition. Speech Commun 4(4):333–344
Sakoe H, Chiba S (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoust Speech Signal Process 26(1):43–49
Salvador S, Chan P (2007) Toward accurate dynamic time warping in linear time and space. Intell Data Anal 11(5):561–580
Schultz D, Jain B (2018) Nonsmooth analysis and subgradient methods for averaging in dynamic time warping spaces. Pattern Recogn 74:340–358
Shapira Weber RA, Eyal M, Skafte N, Shriki O, Freifeld O (2019a) Diffeomorphic temporal alignment nets. In: Advances in neural information processing systems 32 (NeurIPS 2019). Neural Information Processing Systems Foundation Inc. (NeurIPS), Vancouver,Canada, pp 6574–6585. http://papers.nips.cc/paper/8884-diffeomorphic-temporal-alignment-nets.pdf
Shapira Weber RA, Eyal M, Skafte N, Shriki O, Freifeld O (2019b) Diffeomorphic temporal alignment nets: supplementary material. In: Advances in neural information processing systems 32 (NeurIPS 2019). Neural Information Processing Systems Foundation Inc. (NeurIPS), Vancouver,Canada, pp 6574–6585
Shawel BS, Debella TT, Tesfaye G, Tefera YY, Woldegebreal DH (2020) Hybrid prediction model for mobile data traffic: a cluster-level approach. In: 2020 International joint conference on neural networks (IJCNN). IEEE, Glasgow, UK, pp 1–8
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International conference on learning representations (ICLR). ICLR, San Diego, CA, USA, pp 1–14
Srisai D, Ratanamahatana CA (2009) Efficient time series classification under template matching using time warping alignment. In: 2009 Fourth international conference on computer sciences and convergence information technology. IEEE, Seoul, Korea, pp 685–690
Srivastava A, Klassen EP (2016) Functional and shape data analysis, vol 1. Springer, NY
Tavenard R, Faouzi J, Vandewiele G, Divo F, Androz G, Holtz C, Payne M, Yurchak R, Rußwurm M, Kolar K, Woods E (2020) Tslearn, a machine learning toolkit for time series data. J Mach Learn Res 21(118):1–6
Terefe T, Devanne M, Weber J, Hailemariam D, Forestier G (2020) Time series averaging using multi-tasking autoencoder. In 2020 IEEE 32nd international conference on tools with artificial intelligence (ICTAI). IEEE Computer Society, Baltimore, MD, USA, pp 1065–1072
Wei W (2006) Time series analysis: univariate and multivariate methods, 2nd edn. Pearson Addison Wesley, New York
Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33rd international conference on international conference on machine learning (ICML’16). ICML, NY, USA, pp 478–487
Ye L, Keogh E (2009) Time series shapelets: A new primitive for data mining. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery, Paris, France, pp 947–956
Acknowledgements
The authors would like to thank the creators and providers of the UCR datasets: Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Eamonn Keogh and Mustafa Baydogan. Moreover, the authors would also like to thank the university of Strasbourg for allowing us to use its HPC clusters (Mesocentrer). Last but not least, we would also like to thank Ouloufa Dorani and Sophia Nicée, and Dr. Esayas Gebreyouhannes for their roles in the continuation of the Ethio-France Ph.D. program under challenging circumstances. This work got conducted under the support of the French embassy for the African Union and Ethiopia and the former Ethiopian Ministry of Science and Higher Education (MOSHE).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Terefe, T., Devanne, M., Weber, J. et al. Estimating time series averages from latent space of multi-tasking neural networks. Knowl Inf Syst 65, 4967–5004 (2023). https://doi.org/10.1007/s10115-023-01927-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-023-01927-1