Arrow detection in biomedical images using sequential classifier

Original Article


Biomedical images are often complex, and contain several regions that are annotated using arrows. Annotated arrow detection is a critical precursor to region-of-interest (ROI) labeling, which is useful in content-based image retrieval (CBIR). In this paper, we propose a sequential classifier comprising of bidirectional long short-term memory (BLSTM) classifier followed by convexity defect-based arrowhead detection. Different image layers are first segmented via fuzzy binarization. Candidate regions are then checked whether they are arrows by using BLSTM classifier, where Npen++ features are used. In case of low confidence score (i.e., BLSTM classifier score), we take convexity defect-based arrowhead detection technique into account. Our test results on biomedical images from imageCLEF 2010 collection outperforms the existing state-of-the-art arrow detection techniques, by approximately more than 3% in precision, 12% in recall, and therefore 8% in \(\text{F}_1\) score.


Arrow detection Document images Biomedical publications Image region labeling Content-based image retrieval 



The authors would like to acknowledge Mr. Naved Alam for his work during his stay (Research Work) at the Department of Computer Science, Indian Institute of Technology (IIT) Roorkee.


  1. 1.
    Demner-Fushman D, Antani S, Simpson M, Rahman M (2010) Combining text and visual features for biomedical information retrieval and ontologies. Tech. rep, LHNCBC Board of Scientific Counselors, National Institutes of Health, BethesdaGoogle Scholar
  2. 2.
    Demner-Fushman D, Antani S, Simpson MS, Thoma GR (2012) Design and development of a multimodal biomedical information retrieval system. J Comput Sci Eng 6(2):168–177CrossRefGoogle Scholar
  3. 3.
    Dori D, Wenyin L (1999) Automated cad conversion with the machine drawing understanding system: concepts, algorithms, and performance. IEEE Trans Syst Man Cybern Part A Syst Humans 29:411–416CrossRefGoogle Scholar
  4. 4.
    Dori D, Member S, Liu W (1999) Sparse pixel vectorization: An algorithm and its performance evaluation. IEEE Trans Pattern Anal Mach Intell 21:202–215CrossRefGoogle Scholar
  5. 5.
    Park J, Rasheed W, Beak J (2008) Robot navigation using camera by identifying arrow signs. In: International Conference on Grid and Pervasive Computing—Workshops, pp 382–386Google Scholar
  6. 6.
    Cheng B, Stanley RJ, De S, Antani S, Thoma GR (2011) Automatic detection of arrow annotation overlays in biomedical images. Int J Healthc Inf Syst Inf 6(4):23–41CrossRefGoogle Scholar
  7. 7.
    You D, Simpson MS, Antani S, Demner-Fushman D, Thoma GR (2013) A robust pointer segmentation in biomedical images toward building a visual ontology for biomedical article retrieval. In: Zanibbi R, Coüasnon B (eds) Document recognition and retrieval, vol 8658 of SPIE Proceedings. SPIEGoogle Scholar
  8. 8.
    You D, Apostolova E, Antani S, Demner-Fushman D, Thoma GR (2009) Figure content analysis for improved biomedical article retrieval. In: Berkner K, Likforman-Sulem L (eds) Document recognition and retrieval, vol 7247 of SPIE Proceedings, SPIE, pp 1–10Google Scholar
  9. 9.
    You D, Antani S, Demner-Fushman D, Rahman MM, Govindaraju V, Thoma GR (2010) Biomedical article retrieval using multimodal features and image annotations in region-based cbir. In: Likforman-Sulem L, Agam G (eds) Document recognition and retrieval, vol 7534 of SPIE Proceedings. SPIE, pp 1–10Google Scholar
  10. 10.
    Hori O, Doermann DS (1995) Robust table-form structure analysis based on box-driven reasoning. Int Conf Document Anal Recogn 1:218–221CrossRefGoogle Scholar
  11. 11.
    Santosh KC, Wendling L, Antani S, Thoma G (2014) Scalable arrow detection in biomedical images. In: International Conference on Pattern Recognition. IEEE Computer Society, Stockholm, pp 3257–3262Google Scholar
  12. 12.
    Cheng H, Chen Y-H (1999) Fuzzy partition of two dimensional histogram and its application to thresholding. Pattern Recogn 32:825–843CrossRefGoogle Scholar
  13. 13.
    Jaeger S, Manke S, Reichert J, Waibel A (2001) Online handwriting recognition: the npen++ recognizer. Int J Document Anal Recogn 3(3):169–180CrossRefGoogle Scholar
  14. 14.
    Deans SR (1983) The Radon transform and some of its applications. A Wiley-Interscience publication, Wiley, New YorkMATHGoogle Scholar
  15. 15.
    Graves A, Liwicki M, Fernández S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. Pattern Anal Mach Intell IEEE Trans 31(5):855–868CrossRefGoogle Scholar
  16. 16.
    Liwicki M, Graves A, Bunke H, Schmidhuber J (2007) A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proc. 9th Int. Conf. on Document Analysis and Recognition, vol 1, pp 367–371Google Scholar
  17. 17.
    Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRefGoogle Scholar
  18. 18.
    Santosh KC, Wendling L, Antani SK, Thoma GR (2016) Overlaid arrow detection for labeling regions of interest in biomedical images. IEEE Intell Syst 31(3):66–75CrossRefGoogle Scholar
  19. 19.
    Santosh KC, Alam N, Roy PP, Wendling L, Antani S, Thoma G (2016) Arrowhead detection in biomedical images. In: Electronic imaging, document recognition and retrieval XXIII, vol 7. Society for Imaging Science and Technology, pp 1–7Google Scholar
  20. 20.
    Santosh KC, Alam N, Roy PP, Wendling L, Antani SK, Thoma GR (2016) A simple and efficient arrowhead detection technique in biomedical images. Int J Pattern Recogn Artif Intell 30(5):1–16CrossRefGoogle Scholar
  21. 21.
    Ramer U (1972) An iterative procedure for the polygonal approximation of plane curves. Comput Gr Image Process 1(3):244–256CrossRefGoogle Scholar
  22. 22.
    Douglas DH, Peucker TK (1973) Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Can Cartogr 10(2):112–122CrossRefGoogle Scholar
  23. 23.
    Prasad DK, Leung MK, Quek C, Cho S-Y (2012) A novel framework for making dominant point detection methods non-parametric. Image Vision Comput 30(11):843–859CrossRefGoogle Scholar
  24. 24.
    Sakoe H (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoust Speech Signal Process 26:43–49CrossRefMATHGoogle Scholar
  25. 25.
    Keogh EJ, Pazzani MJ (1999) Scaling up dynamic time warping to massive dataset. In: European PKDD, pp 1–11Google Scholar
  26. 26.
    Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, Kahn CE Jr, Hersh W (2010) Overview of the clef 2009 medical image retrieval track. Multilingual information access evaluation II. Multimedia Experiments. Springer, New York, pp 72–84Google Scholar
  27. 27.
    Zhang D, Lu G (2002) Shape-based image retrieval using generic fourier descriptor. Signal Process Image Commun 17:825–848CrossRefGoogle Scholar
  28. 28.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522CrossRefGoogle Scholar
  29. 29.
    Kim W-Y, Kim Y-S (2000) A region-based shape descriptor using zernike moments. Signal Process Image Commun 16(1–2):95–102CrossRefGoogle Scholar
  30. 30.
    Hoang TV, Tabbone S (2012) The generalization of the r-transform for invariant pattern representation. Pattern Recogn 45(6):2145–2163CrossRefMATHGoogle Scholar
  31. 31.
    Santosh KC, Lamiroy B, Wendling L (2013) Dtw-radon-based shape descriptor for pattern recognition. Int J Pattern Recogn Artif Intell 27(3):33MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of South DakotaVermillionUSA
  2. 2.Indian Institute of Technology RoorkeeDepartment of Computer ScienceRoorkeeIndia

Personalised recommendations