Bag-of-visual-words for signature-based multi-script document retrieval

  • Ranju Mandal
  • Partha Pratim Roy
  • Umapada Pal
  • Michael Blumenstein
Original Article

Abstract

An end-to-end architecture for multi-script document retrieval using handwritten signatures is proposed in this paper. The user supplies a query signature sample, and the system exclusively returns a set of documents that contain the query signature. In the first stage, a component-wise classification technique separates the potential signature components from all other components. A bag-of-visual-words powered by SIFT descriptors in a patch-based framework is proposed to compute the features and a support vector machine (SVM)-based classifier was used to separate signatures from the documents. In the second stage, features from the foreground (i.e., signature strokes) and the background spatial information (i.e., background loops, reservoirs etc.) were combined to characterize the signature object to match with the query signature. Finally, three distance measures were used to match a query signature with the signature present in target documents for retrieval. The ‘Tobacco’ (The Legacy Tobacco Document Library (LTDL). University of California, San Francisco, 2007. http://legacy.library.ucsf.edu/) document database and an Indian script database containing 560 documents of Devanagari (Hindi) and Bangla scripts were used for the performance evaluation. The proposed system was also tested on noisy documents, and the promising results were obtained. A comparative study shows that the proposed method outperforms the state-of-the-art approaches.

Keywords

Signature retrieval Logo retrieval SIFT Bag-of-visual-words Spatial pyramid matching Content-based document retrieval 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    The Legacy Tobacco Document Library (LTDL) (2007). University of California, San Francisco. http://legacy.library.ucsf.edu/
  2. 2.
    Suen CY, Xu Q, Lam L (1999) Automatic recognition of handwritten data on cheques—fact or fiction? Pattern Recogni Lett 20:1287–1295CrossRefGoogle Scholar
  3. 3.
    Levy S (2004) “google's two revolutions”. Newsweek. http://www.newsweek.com/googles-two-revolutions-123507
  4. 4.
    Roy PP, Vazquez E, Lladós J, Baldrich R, Pal U (2008) A system to segment text and symbols from color maps. In: Proceedings of the international workshop on graphics recognition (GREC), pp 245–256Google Scholar
  5. 5.
    Zhu G, Doermann D (2009) Logo matching for document image retrieval. In: Proceedings of the international conference on document analysis and recognition (ICDAR), pp 606–610Google Scholar
  6. 6.
    Zhu G, Jaeger S, Doermann D (2006) A robust stamp detection framework on degraded documents. In: Proceedings of SPIE conference on document recognition and retrieval, pp 1–9Google Scholar
  7. 7.
    Farooq F, Sridharan K, Govindaraju V (2006) Identifying handwritten text in mixed documents. In: Proceedings international conference on pattern recognition (ICPR), pp 1–4Google Scholar
  8. 8.
    Guo JK, Ma MY (2001) Separating handwritten material from machine printed text using Hidden Markov Models. In: Proceedings of international conference on document analysis and recognition (ICDAR), pp 439–443Google Scholar
  9. 9.
    Kumar J, Prasad R, Cao H, Abd-Almageed W, Doermann D, Natarajan P (2011) Shape codebook based handwritten and machine printed text zone extraction. In: Proceedings of SPIE, vol 7874.  https://doi.org/10.1117/12.876725
  10. 10.
    Peng X, Setlur S, Govindaraju V, Sitaram R, Bhuvanagiri K (2009) Markov Random Field-based text identification from annotated machine printed documents. In: Proceedings of the international conference on document analysis and recognition (ICDAR), pp 431–435Google Scholar
  11. 11.
    Zheng Y, Li H, Doermann D (2002) The segmentation and identification of handwriting in noisy document images. In: Proceedings of the document analysis systems (DAS), pp 95–105Google Scholar
  12. 12.
    Martinez-Diaz M, Fierrez J, Krish RP, Galbally J (2014) Mobile signature verification: feature robustness and performance comparison. IET Biom 3(4):267–277CrossRefGoogle Scholar
  13. 13.
    Galbally J, Diaz-Cabrera M, Ferrer MA, Gomez-Barrero M, Morales A, Fierrez J (2015) On-line signature recognition through the combination of real dynamic data and synthetically generated static data. Pattern Recognit 48(9):2921–2934CrossRefGoogle Scholar
  14. 14.
    Morocho D, Morales A, Fierrez J, Vera-Rodriguez R (2016) Towards human-assisted signature recognition: improving biometric systems through attribute-based recognition. In: Proceedings of the international conference on identity, security and behavior analysis (ISBA)Google Scholar
  15. 15.
    Blumenstein M, Ferrer Miguel A, Vargas JF (2010) The 4NSigComp2010 off-line signature verification competition: Scenario 2. In: Proceedings of the international conference on frontiers in handwriting recognition (ICFHR), vol 4, pp 721–726Google Scholar
  16. 16.
    Chalechale A, Naghdy G, Mertins A (2003) Signature-based document retrieval. In: Proceedings of the international symposium on signal processing and information technology (ISSPIT), pp 597–600Google Scholar
  17. 17.
    Zhu G, Zheng Y, Doermann D, Jaeger S (2009) Signature detection and matching for document image retrieval. IEEE Trans Pattern Anal Mach Intell (PAMI) 31(11):2015–2031CrossRefGoogle Scholar
  18. 18.
    Srinivasan H, Srihari S (2009) Signature-based retrieval of scanned documents using conditional random fields. In: Argamon S, Howard N (eds.) Computational methods for counterterrorism, Springer, Berlin, pp 17–32Google Scholar
  19. 19.
    Roy PP, Bhowmick S, Pal U, Ramel JY (2012) Signature based document retrieval using GHT of background information. In: Proceedings of the international conference on frontiers in handwriting recognition (ICFHR), pp 225–230Google Scholar
  20. 20.
    Mandal R, Roy PP, Pal U (2011) Signature segmentation from machine printed documents using Conditional Random Field. In: Proceedings of the international conference on document analysis and recognition (ICDAR), pp 1170–1174Google Scholar
  21. 21.
    Du X, AbdAlmageed W, Doermann D (2013) Large-scale signature matching using multi-stage hashing. In: Proceedings of the ICDAR, pp 976–980Google Scholar
  22. 22.
    Briceno JC, Travieso CM, Ferrer MA, Alonso JB, Vargas F (2009) Angular contour parameterization for signature identification. In: LNCS EUROCAST, vol 5717Google Scholar
  23. 23.
    Dewan H, Xichang W, Jiang L (2010) A content-based retrieval algorithm for document image database. In: Proceedings of the international conference on multimedia technology (ICMT), pp 1–5Google Scholar
  24. 24.
    Wang H (2010) Document logo detection and recognition using Bayesian model. In: Proceedings of the international conference on pattern recognition (ICPR), pp 1961–1964Google Scholar
  25. 25.
    Alaei A, Delalandre M (2014) A complete logo detection/recognition system for document images. In: Proceedings of the international workshop on document analysis systems (DAS), pp 324–328Google Scholar
  26. 26.
    Fischer A, Keller A, Frinken V, Bunke H (2010) Hmm-based word spotting in handwritten documents using subword models. In: Proceedings of the international conference on pattern recognition (ICPR), pp 3416–3419Google Scholar
  27. 27.
    Frinken V, Fischer A, Manmatha R, Bunke H (2012) A novel word spotting method based on recurrent neural networks. IEEE Trans Pattern Anal Mach Intell (PAMI) 3(3):211–224CrossRefGoogle Scholar
  28. 28.
    Rodríguez-Serrano JA, Perronnin F (2009) Handwritten word-spotting using hidden markov models and universal vocabularies. Pattern Recognit 42(9):2106–2116CrossRefMATHGoogle Scholar
  29. 29.
    Alhwarin F, Wang C, Durrant DR, Gräser A (2008) Improved sift-features matching for object recognition. In: Proceedings of the vision of computer science, pp 179–190Google Scholar
  30. 30.
    Hua Y, Lin J, Lin C (2010) An improved sift feature matching algorithm. In: Proceedings of the world congress on intelligent control and automation (WCICA), pp 6109–6113Google Scholar
  31. 31.
    Kai W, Bo C, Long T (2011) An improved sift feature matching algorithm based on maximizing minimum distance cluster. In: Proceedings of the international conference on computer science and information technology (ICCSIT), pp 255–259Google Scholar
  32. 32.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis (IJCV) 60(2):91–110CrossRefGoogle Scholar
  33. 33.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond Bags of Features: Spatial Pyramid Matching for recognizing natural scene categories. In: Proceedings of the computer vision and pattern recognition (CVPR), vol 2, pp 2169–2178Google Scholar
  34. 34.
    Fei-Fei L, Peronae P (2005) A bayesian hierarchical model for learning natural scene categories. In: Proceedings of the computer vision and pattern recognition (CVPR), pp 524–531Google Scholar
  35. 35.
    Vapnik V (1995) The nature of statistical learning theory. Springer, BerlinCrossRefMATHGoogle Scholar
  36. 36.
    Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the international conference on knowledge discovery and data mining (KDD), pp 226–231Google Scholar
  37. 37.
    Harris C, Stephens M (1988) A combined corner and edge detector. In: Proceedings of the Alvey vision conference (AVC), pp 147–151Google Scholar
  38. 38.
    Pal U, Belaid A, Choisy CH (2003) Touching numeral segmentation using water reservoir concept. Pattern Recognit Lett 24(1–3):261–272CrossRefGoogle Scholar
  39. 39.
    Pal S, Alaei A, Pal U, Blumenstein M (2012) Multi-script off-line signature identification. In: Proceedings of the international conference hybrid intelligent systems (HIS), pp 236–240Google Scholar
  40. 40.
    Logo dataset. University of Maryland, Laboratory for Language and Media Processing (LAMP) (2014). http://lamp.cfar.umd.edu/
  41. 41.
    Mandal R, Roy PP, Pal U (2012) Signature segmentation from machine printed documents using contextual information. Int J Pattern Recognit Artif Intell (IJPRAI) 26(7).  https://doi.org/10.1142/S0218001412530035

Copyright information

© The Natural Computing Applications Forum 2018

Authors and Affiliations

  1. 1.School of Information and Communication TechnologyGriffith UniversityGold CoastAustralia
  2. 2.Department of Computer Science and EngineeringIndian Institute of TechnologyRoorkeeIndia
  3. 3.Computer Vision and Pattern Recognition UnitIndian Statistical InstituteKolkataIndia
  4. 4.School of SoftwareUniversity of Technology SydneySydneyAustralia

Personalised recommendations