Skip to main content
SpringerLink
Log in

Attention to Warp: Deep Metric Learning for Multivariate Time Series

  • Conference paper
  • First Online:
Book cover Document Analysis and Recognition – ICDAR 2021 (ICDAR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12823))

Included in the following conference series:

Abstract

Deep time series metric learning is challenging due to the difficult trade-off between temporal invariance to nonlinear distortion and discriminative power in identifying non-matching sequences. This paper proposes a novel neural network-based approach for robust yet discriminative time series classification and verification. This approach adapts a parameterized attention model to time warping for greater and more adaptive temporal invariance. It is robust against not only local but also large global distortions, so that even matching pairs that do not satisfy the monotonicity, continuity, and boundary conditions can still be successfully identified. Learning of this model is further guided by dynamic time warping to impose temporal constraints for stabilized training and higher discriminative power. It can learn to augment the inter-class variation through warping, so that similar but different classes can be effectively distinguished. We experimentally demonstrate the superiority of the proposed approach over previous non-parametric and deep models by combining it with a deep online signature verification framework, after confirming its promising behavior in single-letter handwriting classification on the Unipen dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://human.ait.kyushu-u.ac.jp/~matsuo/ICDAR2021_Appendix.pdf.

References

  1. Ahrabian, K., BabaAli, B.: Usage of autoencoders and Siamese networks for online handwritten signature verification. Neural Comput. Appl. 31(12), 9321–9334 (2019)

    Article  Google Scholar 

  2. Bagnall, A.J., Lines, J., Bostrom, A., Large, J., Keogh, E.J.: The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 31(3), 606–660 (2017)

    Article  MathSciNet  Google Scholar 

  3. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)

    Google Scholar 

  4. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a Siamese time delay neural network. In: NIPS, pp. 737–744 (1993)

    Google Scholar 

  5. Che, Z., He, X., Xu, K., Liu, Y.: DECADE: a deep metric learning model for multivariate time series. In: Workshop on Mining and Learning from Time Series (2017)

    Google Scholar 

  6. Coskun, H., Tan, D.J., Conjeti, S., Navab, N., Tombari, F.: Human motion analysis with deep metric learning. In: ECCV, pp. 693–710 (2018)

    Google Scholar 

  7. Cpalka, K., Zalasinski, M., Rutkowski, L.: A new algorithm for identity verification based on the analysis of a handwritten dynamic signature. Appl. Soft Comput. 43, 47–56 (2016)

    Article  Google Scholar 

  8. Faúndez-Zanuy, M.: On-line signature recognition based on VQ-DTW. Pattern Recognit. 40(3), 981–992 (2007)

    Article  Google Scholar 

  9. Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.: Deep learning for time series classification: a review. Data Min. Knowl. Discov. 33(4), 917–963 (2019)

    Article  MathSciNet  Google Scholar 

  10. Fiérrez-Aguilar, J., Ortega-Garcia, J., Ramos, D., Gonzalez-Rodriguez, J.: HMM-based on-line signature verification: feature extraction and signature modeling. Pattern Recognit. Lett. 28(16), 2325–2334 (2007)

    Article  Google Scholar 

  11. Ge, X., Smyth, P.: Deformable Markov model templates for time-series pattern matching. In: KDD, pp. 81–90 (2000)

    Google Scholar 

  12. Grabocka, J., Schmidt-Thieme, L.: NeuralWarp: time-series similarity with warping networks. CoRR (2018)

    Google Scholar 

  13. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: UNIPEN project of on-line data exchange and recognizer benchmarks. In: ICPR, pp. 29–33 (1994)

    Google Scholar 

  14. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: CVPR, pp. 1735–1742 (2006)

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In: ICCV, pp. 1026–1034 (2015)

    Google Scholar 

  16. Kholmatov, A., Yanikoglu, B.A.: Identity authentication using improved online signature verification method. Pattern Recognit. Lett. 26(15), 2400–2408 (2005)

    Article  Google Scholar 

  17. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar 

  18. Lai, S., Jin, L., Lin, L., Zhu, Y., Mao, H.: SynSig2Vec: learning representations from synthetic dynamic signatures for real-world verification. In: AAAI, pp. 735–742 (2020)

    Google Scholar 

  19. Li, C., et al.: A stroke-based RNN for writer-independent online signature verification. In: ICDAR, pp. 526–532 (2019)

    Google Scholar 

  20. Lines, J., Bagnall, A.J.: Time series classification with ensembles of elastic distance measures. Data Min. Knowl. Discov. 29(3), 565–592 (2015)

    Article  MathSciNet  Google Scholar 

  21. Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: EMNLP, pp. 1412–1421 (2015)

    Google Scholar 

  22. Marteau, P.: Time warp edit distance with stiffness adjustment for time series matching. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 306–318 (2009)

    Article  Google Scholar 

  23. Martinez-Diaz, M., Fiérrez, J., Krish, R.P., Galbally, J.: Mobile signature verification: Feature robustness and performance comparison. IET Biom. 3(4), 267–277 (2014)

    Article  Google Scholar 

  24. Mueller, J., Thyagarajan, A.: Siamese recurrent architectures for learning sentence similarity. In: AAAI, pp. 2786–2792 (2016)

    Google Scholar 

  25. Nanni, L., Lumini, A.: A novel local on-line signature verification system. Pattern Recognit. Lett. 29(5), 559–568 (2008)

    Article  Google Scholar 

  26. Ortega-Garcia, J., et al.: MCYT baseline corpus: a bimodal biometric database. IEE Proc. Vis. Image Sig. Process. 150(6), 395–401 (2003)

    Article  Google Scholar 

  27. Rakthanmanon, T., et al.: Searching and mining trillions of time series subsequences under dynamic time warping. In: KDD, pp. 262–270 (2012)

    Google Scholar 

  28. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  29. Roy, D., Mohan, C.K., Murty, K.S.R.: Action recognition based on discriminative embedding of actions using Siamese networks. In: ICIP, pp. 3473–3477 (2018)

    Google Scholar 

  30. Sae-Bae, N., Memon, N.D.: Online signature verification on mobile devices. IEEE Trans. Inf. Forensics Secur. 9(6), 933–947 (2014)

    Article  Google Scholar 

  31. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)

    Article  Google Scholar 

  32. Tallec, C., Ollivier, Y.: Can recurrent neural networks warp time? In: ICLR (2018)

    Google Scholar 

  33. Tang, L., Kang, W., Fang, Y.: Information divergence-based matching strategy for online signature verification. IEEE Trans. Inf. Forensics Secur. 13(4), 861–873 (2018)

    Article  Google Scholar 

  34. Tolosana, R., Vera-Rodríguez, R., Fiérrez, J., Ortega-Garcia, J.: Exploring recurrent neural networks for on-line handwritten signature biometrics. IEEE Access 6, 5128–5138 (2018)

    Article  Google Scholar 

  35. Tolosana, R., Vera-Rodríguez, R., Fiérrez, J., Ortega-Garcia, J.: DeepSign: deep on-line signature verification. IEEE Trans. Biom. Behav. Identity Sci. 3(2), 229–239 (2021)

    Article  Google Scholar 

  36. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  37. Vivaracho-Pascual, C., Faúndez-Zanuy, M., Pascual, J.M.: An efficient low cost approach for on-line signature recognition based on length normalization and fractional distances. Pattern Recognit. 42(1), 183–193 (2009)

    Article  Google Scholar 

  38. Vorugunti, C.S., Guru, D.S., Mukherjee, P., Pulabaigari, V.: OSVNet: convolutional Siamese network for writer independent online signature verification. In: ICDAR, pp. 1470–1475 (2019)

    Google Scholar 

  39. Wu, X., Kimura, A., Iwana, B.K., Uchida, S., Kashino, K.: Deep dynamic time warping: end-to-end local representation learning for online signature verification. In: ICDAR, pp. 1103–1110 (2019)

    Google Scholar 

  40. Wu, X., Kimura, A., Uchida, S., Kashino, K.: Prewarping Siamese network: Learning local representations for online signature verification. In: ICASSP, pp. 2467–2471 (2019)

    Google Scholar 

  41. Yanikoglu, B.A., Kholmatov, A.: Online signature verification using Fourier descriptors. EURASIP J. Adv. Sig. Process. (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kyushu University, Fukuoka, Japan

    Shinnosuke Matsuo, Gantugs Atarsaikhan, Brian Kenji Iwana & Seiichi Uchida

  2. Communication Science Laboratories, NTT Corporation, Tokyo, Japan

    Xiaomeng Wu, Akisato Kimura & Kunio Kashino

Authors
  1. Shinnosuke Matsuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaomeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gantugs Atarsaikhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Akisato Kimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kunio Kashino
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brian Kenji Iwana
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Seiichi Uchida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shinnosuke Matsuo .

Editor information

Editors and Affiliations

  1. Universitat Autònoma de Barcelona, Barcelona, Spain

    Josep Lladós

  2. Lehigh University, Bethlehem, PA, USA

    Daniel Lopresti

  3. Kyushu University, Fukuoka-shi, Japan

    Seiichi Uchida

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Matsuo, S. et al. (2021). Attention to Warp: Deep Metric Learning for Multivariate Time Series. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12823. Springer, Cham. https://doi.org/10.1007/978-3-030-86334-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86334-0_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86333-3

  • Online ISBN: 978-3-030-86334-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation