Skip to main content
SpringerLink
Log in

Time Series Classification Based on Image Transformation Using Feature Fusion Strategy

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Time series classification is an important branch of data analysis. Scholars have proposed a large number of time series classification methods in recent years. However, time series classification remains a challenging problem due to feature selection in time series classification. In order to further simplify the feature selection procedure and improve time series classification accuracy, an automatic feature selection of a time series classification method based on an image feature fusion strategy and a deep learning algorithm is proposed. First, a time series is transformed into images using different types of image transformation methods, i.e. the recurrence plot, Gramian angle difference field, Gramian angle summation field and Markov transition field. Second, the above four images are encoded into a new image type, that is the combined image, by a feature fusion strategy. Finally, a convolutional neural network is used for combined image classification and forecasting model selection. Time series from the M1 and M3 competition datasets are used to verify the effectiveness of the proposed method. The experimental results show that the algorithm has a higher classification accuracy and smaller prediction error compared to the benchmark models. Moreover, the forecasting error MAPE of combined image method is reduced by 0.2020 and 1.7454 compared with the traditional image method and single forecasting methdd respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Yang S , Zheng X , Ji C , et al (2021) Multi-layer representation learning and its application to electronic health records. Neural Process Lett 1–17

  2. Liu B , Zhang Z , Cui R (2020) Efficient time series augmentation methods. In: 2020 13th international congress on image and signal processing, BioMedical engineering and informatics (CISP-BMEI)

  3. Auge D, Hille J, Mueller E et al (2021) A survey of encoding techniques for signal processing in spiking neural networks. Neural Process Lett 53:4693–4710

    Article  Google Scholar 

  4. Ghanem W, Jantan A (2019) Training a neural network for cyberattack classification applications using hybridization of an artificial bee colony and monarch butterfly optimization. Neural Process Lett 51:905–946

    Article  Google Scholar 

  5. Zg A, Vk B, Mi B et al (2020) Weighted kNN and constrained elastic distances for time-series classification - ScienceDirect. Expert Syst Appl 162:113829

    Article  Google Scholar 

  6. Lines J, Bagnall A (2015) Time series classification with ensembles of elastic distance measures. Data Min Knowl Disc 29(3):565–592

    Article  MathSciNet  Google Scholar 

  7. Xiao X, Lu Y, Huang X et al (2021) Temporal series crop classification study in rural china based on sentinel-1 SAR data. IEEE J Sel Top Appl Earth Obs Remote Sens 99:1–1

    Google Scholar 

  8. Geler Z , Kurbalija V , Ivanovic M , et al. (2020) Time-series classification with constrained DTW distance and inverse-square weighted k-NN. In: 2020 international conference on innovations in intelligent systems and applications (INISTA)

  9. Lines J, Bagnall A (2015) Time series classification with ensembles of elastic distance measures. Data Min Knowl Disc 29(3):565–592

    Article  MathSciNet  Google Scholar 

  10. Buza K, Nanopoulos A, Schmidt-Thieme L (2011) Time-series classification based on individualised error prediction. In: IEEE international conference on computational science engineering. IEEE

  11. Morchen F, Ultsch A, Thies M et al (2006) Modeling timbre distance with temporal statistics from polyphonic music. IEEE Trans Audio Speech Lang Process 14(1):81–90

    Article  Google Scholar 

  12. Wang X, Smith K, Hyndman R (2006) Characteristic-based clustering for time series data. Data Min knowl discov 13(3):335–364

    Article  MathSciNet  Google Scholar 

  13. Lin J, Keogh E, Wei L et al (2007) Experiencing SAX: a novel symbolic representation of time series. Data Min Knowl Discov 15(2):107–144

    Article  MathSciNet  Google Scholar 

  14. Baydogan MG, Runger G, Tuv E (2013) A bag-of-features framework to classify time series. IEEE Trans Pattern Anal Mach Intell 35(11):2796–2802

    Article  Google Scholar 

  15. Bagnall A, Lines J, Bostrom A, Large J, Keogh E (2017) The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min Knowl Disc 31(3):606–660

    Article  MathSciNet  Google Scholar 

  16. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  17. Xu J-L et al (2021) Deep learning for classification of time series spectral images using combined multi-temporal and spectral features. Anal Chim Acta 1143:9–20. https://doi.org/10.1016/j.aca.2020.11.018

    Article  Google Scholar 

  18. Yang C, Jiang W, Guo Z (2019) Time series data classification based on dual path cnn-rnn cascade network. IEEE Access 7:155304–155312

    Article  Google Scholar 

  19. Gupta S, Kumar M, Garg A (2019) Improved object recognition results using SIFT and ORB feature detector. Multimed Tools Appl 78(23):34157–34171

    Article  Google Scholar 

  20. Kumar M, Gupta S (2021) 2D-human face recognition using SIFT and SURF descriptors of face’s feature regions. Vis Comput 37(11)

  21. Chhabra P, Garg NK, Kumar M (2020) Content-based image retrieval system using ORB and SIFT features. Neural Comput Appl 32(7):2725–2733

    Article  Google Scholar 

  22. Kumar M , Kumar R , Saluja K K , et al. (2021) Gait recognition based on vision systems: a systematic survey. J Vis Commun Image Represent 75(6)

  23. Goldberg Y (2016) A primer on neural network models for natural language processing. Artif Intell Res 57(1):345–420

    Article  MathSciNet  Google Scholar 

  24. Bansal M, Kumar M, Kumar M, et al. (2020) An efficient technique for object recognition using Shi-Tomasi corner detection algorithm. Soft Comput 1–10

  25. Kumar M, Bansal M, Kumar M (2020) XGBoost: 2D-object recognition using shape descriptors and extreme gradient boosting classifier. In: International conference on computational methods and data engineering (ICMDE 2020)

  26. Kumar Munish, Chhabra et al (2018) An efficient content based image retrieval system using BayesNet and K-NN. Multimed Tools Appl 77(16):21557–21570

    Article  Google Scholar 

  27. Garg Diksha, Naresh et al (2018) Underwater image enhancement using blending of CLAHE and percentile methodologies[J]. Multimed Tools Appl 77(20):26545–26561

    Article  Google Scholar 

  28. Lecun Y, Boser B, Denker J et al (2014) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  29. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in mri images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  30. Campanharo ASLO, Sirer MI, Malmgren RD, Ramos FM, Amaral LAN (2011) Duality between time series and networks. PLoS ONE 6(8):1–13

    Article  Google Scholar 

  31. Wang Z, Oates T (2015) Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth aaai conference on artificial intelligence

  32. Li X, Kang Y, Li F (2020) Forecasting with time series imaging. Expert Syst Appl 1(3):113–130

    Google Scholar 

  33. Akbar S, Ali F, Khan S et al (2020) Deep-AntiFP: prediction of antifungal peptides using distanct multi-information fusion incorporating with deep neural networks. Chemom Intell Lab Syst 208:104214

    Google Scholar 

  34. Zhang X, Zhao H (2021) Hyperspectral-cube-based mobile face recognition: a comprehensive review. Inf Fusion 74(24)

  35. Ye TA, Xm A, Hc A et al (2021) Using Z-number to measure the reliability of new information fusion method and its application in pattern recognition. Appl Soft Comput 111:107658

    Article  Google Scholar 

  36. Campanharo AS, Sirer MI, Malmgren RD, Ramos FM, Amaral LAN (2011) Duality between time series and networks. PLoS ONE 6(8):233–248

    Article  Google Scholar 

  37. Eckmann JP, Kamphorst SO, Ruelle D (1987) Recurrence plots of dynamical systems. Europhys Lett 4(9):973–977

    Article  Google Scholar 

  38. Thiel M, Romano MC, Jürgen Kurths (2004) How much information is contained in a recurrence plot? Phys Lett A 330(5):343–349

    Article  MathSciNet  Google Scholar 

  39. Xu JL, Hugelier S, Zhu H et al (2020) Deep learning for classification of time series spectral images using combined multi-temporal and spectral features. Anal Chim Acta 1143:9–20

    Article  Google Scholar 

  40. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. IEEE Comput Soc 33(5):243–249

    Google Scholar 

  41. Jaworek-Korjakowska J, Kleczek P, Gorgon M (2019) Melanoma thickness prediction based on convolutional neural network with VGG-19 model transfer learning. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE

  42. Zhu Y, Newsam S (2017) DenseNet for dense flow. Comput Sci 6(2):790–794

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (71971089,72001083) and Natural Science Foundation of Guangdong Province (No. 2022A1515011612).

Funding

Natural Science Foundation of Guangdong Province (No.2022A1515011612). Thanks for your coorperation.

Author information

Authors and Affiliations

  1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou, China

    Wentao Jiang, Dabin Zhang, Liwen Ling & Ruibin Lin

Authors
  1. Wentao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dabin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liwen Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruibin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

WJ Software, Data curation, Writing, Methodology. DZ Writing-review, Supervision. LL Conceptualization, Methodology, Writing—review, Supervision. RL Software, Data curation.

Corresponding author

Correspondence to Liwen Ling.

Ethics declarations

Conflict of interest

All Authors declar that they have no conflict of interest.

Ethical approval

This artical does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, W., Zhang, D., Ling, L. et al. Time Series Classification Based on Image Transformation Using Feature Fusion Strategy. Neural Process Lett 54, 3727–3748 (2022). https://doi.org/10.1007/s11063-022-10783-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10783-z

Keywords

Search

Navigation