Applied Intelligence

, Volume 48, Issue 1, pp 166–181 | Cite as

Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine

  • Zahid MehmoodEmail author
  • Toqeer Mahmood
  • Muhammad Arshad Javid


In recent years, the rapid growth of multimedia content makes content-based image retrieval (CBIR) a challenging research problem. The content-based attributes of the image are associated with the position of objects and regions within the image. The addition of image content-based attributes to image retrieval enhances its performance. In the last few years, the bag-of-visual-words (BoVW) based image representation model gained attention and significantly improved the efficiency and effectiveness of CBIR. In BoVW-based image representation model, an image is represented as an order-less histogram of visual words by ignoring the spatial attributes. In this paper, we present a novel image representation based on the weighted average of triangular histograms (WATH) of visual words. The proposed approach adds the image spatial contents to the inverted index of the BoVW model, reduces overfitting problem on larger sizes of the dictionary and semantic gap issues between high-level image semantic and low-level image features. The qualitative and quantitative analysis conducted on three image benchmarks demonstrates the effectiveness of the proposed approach based on WATH.


Content-based image retrieval Bag-of-visual-words Support vector machine Dense SIFT Image classification 


Compliance with Ethical Standards

Competing Interest

All the authors declare no competing interest.


  1. 1.
    Alzu’bi A, Amira A, Ramzan N (2015) Semantic content-based image retrieval: A comprehensive study. J Vis Commun Image Represent 32:20–54CrossRefGoogle Scholar
  2. 2.
    Castellano G, Fanelli AM, Sforza G, Torsello AM (2016) Shape annotation for intelligent image retrieval. Appl Intell 44(1):179–195CrossRefGoogle Scholar
  3. 3.
    Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5CrossRefGoogle Scholar
  4. 4.
    Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from national university of singapore. In: Proceedings of the ACM international conference on image and video retrieval. ACM, p 48Google Scholar
  5. 5.
    Yousaf RM, Rehman S, Dawood H, Ping G, Mehmood Z, Azam S, Khan AA (2017) Saliency based object detection and enhancements in static images. In: International Conference on Information Science and Applications. Springer, pp 114–123Google Scholar
  6. 6.
    Sivic J, Zisserman A (2003) Video google: A text retrieval approach to object matching in videos. In: Proceedings of the 9th IEEE International Conference on Computer Vision, 2003. IEEE, pp. 1470–1477Google Scholar
  7. 7.
    Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8Google Scholar
  8. 8.
    Ali N, Bajwa KB, Sablatnig R, Mehmood Zahid (2016) Image retrieval by addition of spatial information based on histograms of triangular regions. Computers & Electrical EngineeringGoogle Scholar
  9. 9.
    Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002MathSciNetCrossRefGoogle Scholar
  10. 10.
    Zhou W, Li H, Yijuan L, Tian Q (2013) Sift match verification by geometric coding for large-scale partial-duplicate web image search. ACM Trans Multimed Comput Commun Appl 9(1):4CrossRefGoogle Scholar
  11. 11.
    Khan R, Barat C, Muselet D, Ducottet C (2012) Spatial orientations of visual word pairs to improve bag-of-visual-words model. In: Proceedings of the British Machine Vision Conference. BMVA Press, pp 89–1Google Scholar
  12. 12.
    Anwar H, Zambanini S, Kampel M, Vondrovec K (2015) Ancient coin classification using reverse motif recognition: Image-based classification of roman republican coins. IEEE Signal Process Mag 32(4):64–74CrossRefGoogle Scholar
  13. 13.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on Computer vision and pattern recognition, vol 2. IEEE, pp 2169-2178Google Scholar
  14. 14.
    Mehmood Z, Anwar SM, Altaf M, Ali N (2017) A novel image retrieval based on rectangular spatial histograms of visual words. Kuwait Journal of Science, KuwaitGoogle Scholar
  15. 15.
    Ashraf R, Bashir K, Mahmood T (2016) Content-based image retrieval by exploring bandletized regions through support vector machines. J Inf Sci Eng 32(2):245–269MathSciNetGoogle Scholar
  16. 16.
    Zhang D, Md MI, Guojun L (2012) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362CrossRefGoogle Scholar
  17. 17.
    Liu Y, Zhang D, Guojun L, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282CrossRefzbMATHGoogle Scholar
  18. 18.
    Das R, Thepade S, Ghosh S (2015) Multi technique amalgamation for enhanced information identification with content based image data. SpringerPlus 4(1):1–26CrossRefGoogle Scholar
  19. 19.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  20. 20.
    Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In: Computer visionECCV 2006. Springer, pp 404-417Google Scholar
  21. 21.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR, vol 1. IEEE, pp 886–893Google Scholar
  22. 22.
    Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767CrossRefGoogle Scholar
  23. 23.
    Leutenegger S, Chli M, Siegwart RY (2011) Brisk: Binary robust invariant scalable keypoints. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, pp 2548–2555Google Scholar
  24. 24.
    Mukherjee D, Wu QMJ, Wang G (2015) A comparative experimental study of image feature detectors and descriptors. Mach Vis Appl 26(4):443–466CrossRefGoogle Scholar
  25. 25.
    Krajnik T, Cristóforis P, Nitsche M, Kusumam K, Duckett T (2015) Image features and seasons revisited. In: 2015 European conference on mobile robots (ECMR). IEEE, pp 1–7Google Scholar
  26. 26.
    Mahmood T, Nawaz T, Ashraf R, Shah M, Khan Z, Irtaza A, Mehmood Z A survey on block based copy move image forgery detection techniques. In: 2015 international conference on emerging technologies (ICET). IEEE, pp 1–6Google Scholar
  27. 27.
    Wang C, Zhang B, Qin Z, Xiong J (2013) Spatial weighting for bag-of-features based image retrieval. In: Integrated uncertainty in knowledge modelling and decision making. Springer pp 91–100Google Scholar
  28. 28.
    Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of eodh and color-sift: Application to image retrieval based on codebook. Signal Process Image Commun 29(4):530–545CrossRefGoogle Scholar
  29. 29.
    Jing Y, Qin Z, Wan T, Xi Z (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364CrossRefGoogle Scholar
  30. 30.
    Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by gaussian mixture models. Neurocomputing 171:673–684CrossRefGoogle Scholar
  31. 31.
    Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348CrossRefGoogle Scholar
  32. 32.
    Yuan X, Yu J, Qin Z, Wan T (2011) A sift-lbp image retrieval model based on bag of features IEEE International Conference on Image ProcessingGoogle Scholar
  33. 33.
    Dubey SR, Singh SK, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Electr Eng 46:288–302CrossRefGoogle Scholar
  34. 34.
    Wan J, Wang D, Hong Hoi SC, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the ACM international conference on multimedia. ACM, 157–166Google Scholar
  35. 35.
    Mehmood Z, Anwar SM, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016Google Scholar
  36. 36.
    Gupta R, Patil H, Mittal A (2010) Robust order-based methods for feature description. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 334–341Google Scholar
  37. 37.
    Liu L, Fieguth WP (2012) Texture classification from random features. IEEE Trans Pattern Anal Mach Intell 34(3):574– 586CrossRefGoogle Scholar
  38. 38.
    Mahmood T, Nawaz T, Mehmood Z, Khan Z, Shah M, Ashraf R (2016) Forensic analysis of copy-move forgery in digital images using the stationary wavelets. In: 2016 6th international conference on innovative computing technology (INTECH). IEEE, pp 578–583Google Scholar
  39. 39.
    Csurka G, Dance C, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision, ECCV. vol 1. Prague, pp 1–2Google Scholar
  40. 40.
    Arthur D, Vassilvitskii S (2007) k-means++: The advantages of careful seeding. In: Proceedings of the 18th annual ACM-SIAM symposium on discrete algorithms. Society for Industrial and Applied Mathematics, pp 1027–1035Google Scholar
  41. 41.
    Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge university press, CambridgeCrossRefzbMATHGoogle Scholar
  42. 42.
    Vedaldi A, Zisserman A Sparse kernel approximations for eff ient classification and detection. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2320–2327Google Scholar
  43. 43.
    Nowak E, Jurie F, Triggs B (2006) Sampling strategies for bag-of-features image classification. In: Computer Vision–ECCV 2006. Springer, pp 490–503Google Scholar
  44. 44.
    Wang JZ, Li J, Wiederhold G (2001) Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Trans Pattern Anal Mach Intell 23(9):947–963CrossRefGoogle Scholar
  45. 45.
    Li R, Bhanu B, Krawiec K (2007) Hybrid coevolutionary algorithms vs. svm algorithms. In: Proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, pp 456–463Google Scholar
  46. 46.
    Xiao J, Hays J, Ehinger KA, Oliva A, Torralba A (2010) Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3485–3492Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Zahid Mehmood
    • 1
    Email author
  • Toqeer Mahmood
    • 2
  • Muhammad Arshad Javid
    • 3
  1. 1.Department of Software EngineeringUniversity of Engineering and TechnologyTaxilaPakistan
  2. 2.Department of Computer EngineeringUniversity of Engineering and TechnologyTaxilaPakistan
  3. 3.Department of Basic SciencesUniversity of Engineering and TechnologyTaxilaPakistan

Personalised recommendations