Advertisement

Fixed-sized representation learning from offline handwritten signatures of different sizes

  • Luiz G. HafemannEmail author
  • Luiz S. Oliveira
  • Robert Sabourin
Special Issue Paper

Abstract

Methods for learning feature representations for offline handwritten signature verification have been successfully proposed in recent literature, using deep convolutional neural networks to learn representations from signature pixels. Such methods reported large performance improvements compared to handcrafted feature extractors. However, they also introduced an important constraint: the inputs to the neural networks must have a fixed size, while signatures vary significantly in size between different users. In this paper, we propose addressing this issue by learning a fixed-sized representation from variable-sized signatures by modifying the network architecture, using spatial pyramid pooling. We also investigate the impact of the resolution of the images used for training and the impact of adapting (fine-tuning) the representations to new operating conditions (different acquisition protocols, such as writing instruments and scan resolution). On the GPDS dataset, we achieve results comparable with the state of the art, while removing the constraint of having a maximum size for the signatures to be processed. We also show that using higher resolutions (300 or 600 dpi) can improve performance when skilled forgeries from a subset of users are available for feature learning, but lower resolutions (around 100dpi) can be used if only genuine signatures are used. Lastly, we show that fine-tuning can improve performance when the operating conditions change.

Keywords

Handwritten signature verification Representation learning Convolutional neural networks Transfer learning Domain adaptation 

Notes

Acknowledgements

This work was supported by the Fonds de recherche du Québec - Nature et technologies (FRQNT), the CNPq Grant #206318/2014-6 and by the Grant RGPIN-2015-04490 to Robert Sabourin from the NSERC of Canada. In addition, we gratefully acknowledge the support of NVIDIA with the donation of the Titan Xp GPU used in this research.

References

  1. 1.
    Plamondon, R., Lorette, G.: Automatic signature verification and writer identification the state of the art. Pattern Recognit. 22(2), 107–131 (1989)CrossRefGoogle Scholar
  2. 2.
    Leclerc, F., Plamondon, R.: Automatic signature verification: the state of the art—1989–1993. Int. J. Pattern Recognit. Artif. Intell. 8(03), 643–660 (1994)CrossRefGoogle Scholar
  3. 3.
    Impedovo, D., Pirlo, G.: Automatic signature verification: the state of the art. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 38(5), 609–635 (2008)CrossRefGoogle Scholar
  4. 4.
    Hafemann, L.G., Sabourin, R., Oliveira, L.S.: Offline handwritten signature verification—literature review. In: International Conference on Image Processing Theory, Tools and Applications (IPTA) (2017)Google Scholar
  5. 5.
    Hafemann, L.G., Sabourin, R., Oliveira, L.S.: Writer-independent feature learning for offline signature verification using convolutional neural networks. In: The 2016 International Joint Conference on Neural Networks (2016)Google Scholar
  6. 6.
    Hafemann, L.G., Sabourin, R., Oliveira, L.S.: Learning features for offline handwritten signature verification using deep convolutional neural networks. Pattern Recognit. 70, 163–176 (2017).  https://doi.org/10.1016/j.patcog.2017.05.012 CrossRefGoogle Scholar
  7. 7.
    Rantzsch, H., Yang, H., Meinel, C.: Signature embedding: writer independent offline signature verification with deep metric learning. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Porikli, F., Skaff, S., Entezari, A., Min, J., Iwai, D., Sadagic, A., Scheidegger, C., Isenberg, T. (eds.) Advances in Visual Computing. Lecture Notes in Computer Science, pp. 616–625. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-50832-0_60 Google Scholar
  8. 8.
    Zhang, Z., Liu, X., Cui, Y.: Multi-phase offline signature verification system using deep convolutional generative adversarial networks. In: 2016 9th International Symposium on Computational Intelligence and Design (ISCID), vol. 02, pp. 103–107 (2016).  https://doi.org/10.1109/ISCID.2016.2033
  9. 9.
    Hafemann, L.G., Oliveira, L.S., Sabourin, R.: Analyzing features learned for offline signature verification using Deep CNNs. In: 23rd International Conference on Pattern Recognition (2016)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision, pp. 346–361. Springer (2014)Google Scholar
  11. 11.
    He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015).  https://doi.org/10.1109/TPAMI.2015.2389824 CrossRefGoogle Scholar
  12. 12.
    Guerbai, Y., Chibani, Y., Hadjadji, B.: The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters. Pattern Recognit. 48(1), 103–113 (2015).  https://doi.org/10.1016/j.patcog.2014.07.016 CrossRefGoogle Scholar
  13. 13.
    Bertolini, D., Oliveira, L.S., Justino, E., Sabourin, R.: Reducing forgeries in writer-independent off-line signature verification through ensemble of classifiers. Pattern Recognit. 43(1), 387–396 (2010).  https://doi.org/10.1016/j.patcog.2009.05.009 CrossRefzbMATHGoogle Scholar
  14. 14.
    Rivard, D., Granger, E., Sabourin, R.: Multi-feature extraction and selection in writer-independent off-line signature verification. Int. J. Doc. Anal. Recognit. 16(1), 83–103 (2013)CrossRefGoogle Scholar
  15. 15.
    Eskander, G., Sabourin, R., Granger, E.: Hybrid writer-independent-writer-dependent offline signature verification system. IET Biom. 2(4), 169–181 (2013).  https://doi.org/10.1049/iet-bmt.2013.0024 CrossRefGoogle Scholar
  16. 16.
    Nagel, R.N., Rosenfeld, A.: Computer detection of freehand forgeries. IEEE Trans. Comput. 100(9), 895–905 (1977)CrossRefGoogle Scholar
  17. 17.
    Justino, E.J., El Yacoubi, A., Bortolozzi, F., Sabourin, R.: An off-line signature verification system using HMM and graphometric features. In: Fourth IAPR International Workshop on Document Analysis Systems (DAS), pp. 211–222. Rio de, Citeseer (2000)Google Scholar
  18. 18.
    Oliveira, L.S., Justino, E., Freitas, C., Sabourin, R.: The graphology applied to signature verification. In: 12th Conference of the International Graphonomics Society, pp. 286–290 (2005)Google Scholar
  19. 19.
    Vargas, J.F., Ferrer, M.A., Travieso, C.M., Alonso, J.B.: Off-line signature verification based on grey level information using texture features. Pattern Recognit. 44(2), 375–385 (2011).  https://doi.org/10.1016/j.patcog.2010.07.028 CrossRefzbMATHGoogle Scholar
  20. 20.
    Ylmaz, M.B., Yankolu, B.: Score level fusion of classifiers in off-line signature verification. Inf. Fusion 32, 109–119 (2016).  https://doi.org/10.1016/j.inffus.2016.02.003 CrossRefGoogle Scholar
  21. 21.
    Pal, S., Chanda, S., Pal, U., Franke, K., Blumenstein, M.: Off-line signature verification using G-SURF. In: 12th International Conference on Intelligent Systems Design and Applications, pp. 586–591. IEEE (2012)Google Scholar
  22. 22.
    Alaei, A., Pal, S., Pal, U., Blumenstein, M.: An efficient signature verification method based on an interval symbolic representation and a fuzzy similarity measure. IEEE Trans. Inf. Forensics Secur. 12(10), 2360–2372 (2017)CrossRefGoogle Scholar
  23. 23.
    Ferrer, M., Diaz-Cabrera, M., Morales, A.: Static signature synthesis: a neuromotor inspired approach for biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 667–680 (2015).  https://doi.org/10.1109/TPAMI.2014.2343981 CrossRefGoogle Scholar
  24. 24.
    Diaz, M., Ferrer, M.A., Eskander, G.S., Sabourin, R.: Generation of duplicated off-line signature images for verification systems. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 951–964 (2016).  https://doi.org/10.1109/TPAMI.2016.2560810 CrossRefGoogle Scholar
  25. 25.
    Ferrer, M.A., Chanda, S., Diaz, M., Banerjee, C.K., Majumdar, A., Carmona-Duarte, C., Acharya, P., Pal, U.: Static and dynamic synthesis of Bengali and Devanagari signatures. IEEE Trans. Cybern. PP(99), 1–12 (2017).  https://doi.org/10.1109/TCYB.2017.2751740
  26. 26.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  27. 27.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  28. 28.
    Sabourin, R., Cheriet, M., Genest, G.: An extended-shadow-code based approach for off-line signature verification. In: Proceedings of the Second International Conference on Document Analysis and Recognition 1993, pp. 1–5 (1993).  https://doi.org/10.1109/ICDAR.1993.395795
  29. 29.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015). arXiv preprint arXiv:1502.03167
  30. 30.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  31. 31.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724(2014).  https://doi.org/10.1109/CVPR.2014.222
  32. 32.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets (2014). arXiv preprint arXiv:1405.3531
  33. 33.
    Vargas, J., Ferrer, M., Travieso, C., Alonso, J.: Off-line handwritten signature GPDS-960 corpus. In: 9th International Conference on Document Analysis and Recognition, vol. 2, pp. 764–768 (2007).  https://doi.org/10.1109/ICDAR.2007.4377018
  34. 34.
    Ortega-Garcia, J., Fierrez-Aguilar, J., Simon, D., Gonzalez, J., Faundez-Zanuy, M., Espinosa, V., Satue, A., Hernaez, I., Igarza, J.J., Vivaracho, C., et al.: MCYT baseline corpus: a bimodal biometric database. IEE Proc. Vis. Image Signal Process. 150(6), 395–401 (2003)CrossRefGoogle Scholar
  35. 35.
    Kalera, M.K., Srihari, S., Xu, A.: Offline signature verification and identification using distance statistics. Int. J. Pattern Recognit. Artif. Intell. 18(07), 1339–1360 (2004).  https://doi.org/10.1142/S0218001404003630 CrossRefGoogle Scholar
  36. 36.
    Freitas, C., Morita, M., Oliveira, L., Justino, E., Yacoubi, A., Lethelier, E., Bortolozzi, F., Sabourin, R.: Bases de dados de cheques bancarios brasileiros. In: XXVI Conferencia Latinoamericana de Informatica (2000)Google Scholar
  37. 37.
    Hu, J., Chen, Y.: Offline signature verification using real adaboost classifier combination of pseudo-dynamic features. In: 12th International Conference on Document Analysis and Recognition, pp. 1345–1349 (2013).  https://doi.org/10.1109/ICDAR.2013.272
  38. 38.
    Serdouk, Y., Nemmour, H., Chibani, Y.: New gradient features for off-line handwritten signature verification. In: 2015 International Symposium on Innovations in Intelligent Systems and Applications (INISTA), pp 1–4 (2015).  https://doi.org/10.1109/INISTA.2015.7276751
  39. 39.
    Soleimani, A., Araabi, B.N., Fouladi, K.: Deep multitask metric learning for offline signature verification. Pattern Recognit. Lett. 80, 84–90 (2016).  https://doi.org/10.1016/j.patrec.2016.05.023
  40. 40.
    Gilperez, A., Alonso-Fernandez, F., Pecharroman, S., Fierrez, J., Ortega-Garcia, J.: Off-line signature verification using contour features. In: 11th International Conference on Frontiers in Handwriting Recognition, Montral, Qubec-Canada, 19–21 Aug 2008, CENPARMI, Concordia University (2008)Google Scholar
  41. 41.
    Wen, J., Fang, B., Tang, Y.Y., Zhang, T.: Model-based signature verification with rotation invariant features. Pattern Recognit. 42(7), 1458–1466 (2009).  https://doi.org/10.1016/j.patcog.2008.10.006 CrossRefzbMATHGoogle Scholar
  42. 42.
    Ooi, S.Y., Teoh, A.B.J., Pang, Y.H., Hiew, B.Y.: Image-based handwritten signature verification using hybrid methods of discrete Radon transform, principal component analysis and probabilistic neural network. Appl. Soft Comput. 40, 274–282 (2016)CrossRefGoogle Scholar
  43. 43.
    Chen, S., Srihari, S.: A new off-line signature verification method based on graph. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 2, pp. 869–872 (2006).  https://doi.org/10.1109/ICPR.2006.125
  44. 44.
    Kumar, R., Kundu, L., Chanda, B., Sharma, J.D.: A writer-independent off-line signature verification system based on signature morphology. In: Proceedings of the First International Conference on Intelligent Interactive Technologies and Multimedia, ACM, New York, NY, IITM ’10, pp. 261–265 (2010).  https://doi.org/10.1145/1963564.1963610
  45. 45.
    Kumar, R., Sharma, J.D., Chanda, B.: Writer-independent off-line signature verification using surroundedness feature. Pattern Recognit. Lett. 33(3), 301–308 (2012).  https://doi.org/10.1016/j.patrec.2011.10.009 CrossRefGoogle Scholar
  46. 46.
    Bharathi, R., Shekar, B.: Off-line signature verification based on chain code histogram and support vector machine. In: 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 2063–2068 (2013).  https://doi.org/10.1109/ICACCI.2013.6637499
  47. 47.
    Batista, L., Granger, E., Sabourin, R.: Dynamic selection of generative–discriminative ensembles for off-line signature verification. Pattern Recognit. 45(4), 1326–1340 (2012).  https://doi.org/10.1016/j.patcog.2011.10.011 CrossRefzbMATHGoogle Scholar
  48. 48.
    Rua, E.A., Castro, J.L.A.: Online signature verification based on generative models. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(4), 1231–1242 (2012).  https://doi.org/10.1109/TSMCB.2012.2188508 CrossRefGoogle Scholar
  49. 49.
    Fierrez, J., Ortega-Garcia, J., Ramos, D., Gonzalez-Rodriguez, J.: HMM-based on-line signature verification: feature extraction and signature modeling. Pattern Recognit. Lett. 28(16), 2325–2334 (2007).  https://doi.org/10.1016/j.patrec.2007.07.012 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Laboratoire d’imagerie, de vision et d’intelligence artificielle (LIVIA)École de technologie supérieureMontrealCanada
  2. 2.Department of InformaticsFederal University of Parana (UFPR)CuritibaBrazil

Personalised recommendations