The Journal of Supercomputing

, Volume 71, Issue 3, pp 909–937

Real-time indexing for large image databases: color and edge directivity descriptor on GPU

  • L. Bampis
  • C. Iakovidou
  • S. A. Chatzichristofis
  • Y. S. Boutalis
  • A. Amanatiadis
Article

Abstract

In this paper, we focus on implementing the extraction of a well-known low-level image descriptor using the multicore power provided by general-purpose graphic processing units (GPGPUs). The color and edge directivity descriptor, which incorporates both color and texture information achieving a successful trade-off between effectiveness and efficiency, is employed and reassessed for parallel execution. We are motivated by the fact that image/frame indexing should be achieved real time, which in our case means that a system should be capable of indexing a frame or an image as it becomes part of a database (ideally, calculating the descriptor as the images are captured). Two strategies are explored to accelerate the method and bypass resource limitations and architectural constrains. An approach that exclusively uses the GPU together with a hybrid implementation that distributes the computations to both available GPU and CPU resources are proposed. The first approach is strongly based on the compute unified device architecture and excels compared to all other solutions when the GPU resources are abundant. The second implementation suggests a hybrid scheme where the extraction process is split in two sequential stages, allowing the input data (images or video frames) to be pipelined through the central and the graphic processing units. Experimental results were conducted on four different combinations of GPU–CPU technologies in order to highlight the strengths and the weaknesses of all implementations. Real-time indexing is obtained over all computational setups for both GPU-only and Hybrid techniques. An impressive 22 times acceleration is recorded for the GPU-only method. The proposed Hybrid implementation outperforms the GPU-only implementation and becomes the preferred solution when a low-cost setup (i.e., more advanced CPU combined with a relatively weak GPU) is employed.

Keywords

GPU Hybrid implementation Image retrieval Feature extraction Database indexing 

References

  1. 1.
    Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60CrossRefGoogle Scholar
  2. 2.
    Wetzel A (1997) Computational aspects of pathology image classification and retrieval. J Supercomput 11(3):279–293CrossRefGoogle Scholar
  3. 3.
    Ren R, Collomosse J, Jose J (2011) A bovw based query generative model. In: Proceedings of the 17th international conference on advances in multimedia modeling. Volume Part I, ser. MMM’11, 2011, pp 118–128Google Scholar
  4. 4.
    Lux M, Chatzichristofis S (2008) Lire: lucene image retrieval: an extensible java cbir library. In: Proceeding of the 16th ACM international conference on multimedia. ACM, 2008, pp 1085–1088Google Scholar
  5. 5.
    Chatzichristofis S, Iakovidou C, Boutalis Y, Marques O (2013) Co.vi.wo.: color visual words based on non-predefined size codebooks. IEEE Trans Cybernet 43(1):192–205CrossRefGoogle Scholar
  6. 6.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. CVPR 2:2169–2178Google Scholar
  7. 7.
    Zagoris K, Chatzichristofis SA, Arampatzis A (2011) Bag-of-visual-words vs global image descriptors on two-stage multimodal retrieval. In: Proceedings of the 34th international ACM SIGIR conference on research and development in information Retrieval, pp 1251–1252Google Scholar
  8. 8.
    Amanatiadis A, Kaburlasos V, Gasteratos A, Papadakis S (2011) Evaluation of shape descriptors for shape-based image retrieval. IET Image Process 5(5):493–499CrossRefGoogle Scholar
  9. 9.
    Sevilla J, Bernabe S, Plaza A (2014) Unmixing-based content retrieval system for remotely sensed hyperspectral imagery on GPUs. J Supercomput, pp 1–12Google Scholar
  10. 10.
    Park IK, Singhal N, Lee MH, Cho S, Kim CW (2011) Design and performance evaluation of image processing algorithms on gpus. IEEE Trans Parallel Distrib Syst 22(1):91–104CrossRefGoogle Scholar
  11. 11.
    Antikainen J, Havel J, Josth R, Herout A, Zemcík P, Hauta-Kasari M, Zemcík P (2011) Nonnegative tensor factorization accelerated using GPGPU. IEEE Trans Parallel Distrib Syst 22(7):1135–1141CrossRefGoogle Scholar
  12. 12.
    Zhu L, Jin H, Zheng R, Feng X (2013) Effective naive bayes nearest neighbor based image classification on GPU. J Supercomput, pp 1–29Google Scholar
  13. 13.
    Risojević V, Babić Z, Dobravec T, Bulić P et al (2013) A GPU implementation of a structural-similarity-based aerial-image classification. J Supercomput 65(2):978–996CrossRefGoogle Scholar
  14. 14.
    van de Sande KEA, Gevers T, Snoek CGM (2011) Empowering visual categorization with the GPU. IEEE Trans Multimed 13(1):60–70CrossRefGoogle Scholar
  15. 15.
    Alvarado R, Tapia JJ, Rolón C (2013) Medical image segmentation with deformable models on graphics processing units. J Supercomput, pp 1–26Google Scholar
  16. 16.
    Song B, Tang W, Nguyen T-D, Hassan MM, Huh EN (2013) An optimized hybrid remote display protocol using GPU-assisted m-JPEG encoding and novel high-motion detection algorithm. J Supercomput 66(3):1729–1748CrossRefGoogle Scholar
  17. 17.
    López MB, Nykänen H, Hannuksela J, Silvén O, Vehviläinen M (2011) Accelerating image recognition on mobile devices using GPGPU. In:Proceedings of SPIE 7872:78720RGoogle Scholar
  18. 18.
    Amanatiadis A, Bampis L, Gasteratos A (2014) Accelerating image super-resolution regression by a hybrid implementation in mobile devices. In: Proceedings IEEE international conference on consumer electronics, pp 335–336Google Scholar
  19. 19.
    Nalpantidis L, Amanatiadis A, Sirakoulis G, Gasteratos A (2011) Efficient hierarchical matching algorithm for processing uncalibrated stereo vision images and its hardware architecture. IET Image Process. 5(5):481–492CrossRefGoogle Scholar
  20. 20.
    Chatzichristofis S, Zagoris K, Boutalis Y, Papamarkos N (2010) Accurate image retrieval based on compact composite descriptors and relevance feedback information. Int J Pattern Recogn Artif Intell 24(2):207–244CrossRefGoogle Scholar
  21. 21.
    Jiang Y, Xu X, Terlecky P, Abdelzaher T, Bar-Noy A, Govindan R (2013) Mediascope: selective on-demand media retrieval from mobile devices. In: Proceedings of the 12th international conference on information processing in sensor networks, ser. IPSN ’13. New York, NY, USA: ACM, 2013, pp 289–300Google Scholar
  22. 22.
    Zha Z-J, Tian Q, Cai J, Wang Z (2013) Interactive social group recommendation for flickr photos. Neurocomputing 105:30–37CrossRefGoogle Scholar
  23. 23.
    van Leuken RH, Pueyo LG, Olivares X, van Zwol R (2009) Visual diversification of image search results. In: WWW. ACM, 2009, pp 341–350Google Scholar
  24. 24.
    Jin X, Gallagher AC, Cao L, Luo J, Han J (2010) The wisdom of social multimedia: using flickr for prediction and forecast. In: ACM Multimedia, 2010, pp 1235–1244Google Scholar
  25. 25.
    Daras P, Semertzidis T, Makris L, Strintzis MG (2010) Similarity content search in content centric networks. In: ACM multimedia, 2010, pp 775–778Google Scholar
  26. 26.
    Iakovidou C, Anagnostopoulos N, Kapoutsis AC, Boutalis YS, Chatzichristofis SA (2014) Searching images with MPEG-7 ( & mpeg-7-like) powered localized descriptors: the SIMPLE answer to effective content based image retrieval. In 2014 12th International workshop on content-based multimedia indexing (CBMI), Klagenfurt, Austria, June 18–20(2014), 2014, pp 1–6. [Online]. doi:10.1109/CBMI.2014.6849821
  27. 27.
    Lux M, Marques O, Schoffmann K, Boszormenyi L, Lajtai G (2010) A novel tool for summarization of arthroscopic videos. Multimed Tools Appl 46(2–3):521–544CrossRefGoogle Scholar
  28. 28.
    Rafailidis D, Manolopoulou S, Daras P (2013) A unified framework for multimodal retrieval. Pattern Recogn 46(12):3358–3370CrossRefGoogle Scholar
  29. 29.
    Piras L, Giacinto G (2012) Synthetic pattern generation for imbalanced learning in image retrieval. Pattern Recogn Lett 33(16):2198–2205CrossRefGoogle Scholar
  30. 30.
    Vallet D, Cantador I, Jose JM (2013) Exploiting semantics on external resources to gather visual examples for video retrieval. Int J Multimed Inf Retriev 2(2):117–130CrossRefGoogle Scholar
  31. 31.
    Daras P, Manolopoulou S, Axenopoulos A (2012) Search and retrieval of rich media objects supporting multiple multimodal queries. IEEE Trans Multimed 14(3–2):734–746CrossRefGoogle Scholar
  32. 32.
    Yu J, Jin X, Han J, Luo J (2011) Collection-based sparse label propagation and its application on social group suggestion from photos. ACM TIST 2(2):12Google Scholar
  33. 33.
    Chatzichristofis S, Boutalis Y (2008) CEDD: color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. LNCS, Computer Vision SystemsGoogle Scholar
  34. 34.
    Wang J, Li J, Wiederhold G (2001) Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Transactions on pattern analysis and machine intelligence, pp 947–963Google Scholar
  35. 35.
    Schaefer G, Stich M (2004) UCID-an uncompressed colour image database. Storage and retrieval methods and applications for multimedia 2004, vol 5307, pp 472–480Google Scholar
  36. 36.
    Chatzichristofis S, Boutalis Y (2010) Content based radiology image retrieval using a fuzzy rule based scalable composite descriptor. Multimed Tools Appl 46:493–519CrossRefGoogle Scholar
  37. 37.
    Chatzichristofis S, Arampatzis A, Boutalis Y (2010) Investigating the behavior of compact composite descriptors in early fusion, late fusion and distributed image retrieval. Radioengineering 19(4):725Google Scholar
  38. 38.
    Chatzichristofis SA, Boutalis YS, Lux M (2010) SpCD–spatial color distribution descriptor. A fuzzy rule based compact composite descriptor appropriate for hand drawn color sketches retrieval. In: ICAART, 2010, pp 58–63Google Scholar
  39. 39.
    Manjunath B, Ohm J, Vasudevan V, Yamada A (2001) Color and texture descriptors. IEEE Trans Circuits Syst video Technol 11(6):703–715CrossRefGoogle Scholar
  40. 40.
    Huang J, Kumar S, Mitra M, Zhu W (2001) Image indexing using color correlograms. US Patent 6,246,790, 12, pp 1–16Google Scholar
  41. 41.
    Thomee B, Bakker EM, Lew MS (21010) Top-surf: a visual words toolkit. In ACM multimedia, 2010, pp 1473–1476Google Scholar
  42. 42.
    Sartori J, Kumar R (2013) Branch and data herding: reducing control and memory divergence for error-tolerant gpu applications. IEEE Trans Multimed 15(2):279–290CrossRefGoogle Scholar
  43. 43.
    van der Laan WJ, Jalba AC, Roerdink JB (2011) Accelerating wavelet lifting on graphics hardware using CUDA. IEEE Trans Parallel Distrib Syst 22(1):132–146CrossRefGoogle Scholar
  44. 44.
    Li R, Saad Y (2013) Gpu-accelerated preconditioned iterative linear solvers. J Supercomput 63(2):443–466CrossRefGoogle Scholar
  45. 45.
    Thibault JC, Senocak I (2012) Accelerating incompressible flow computations with a pthreads-CUDA implementation on small-footprint multi-GPU platforms. J Supercomput 59(2):693–719CrossRefGoogle Scholar
  46. 46.
    Cano A, Luna JM, Ventura S (2013) High performance evaluation of evolutionary-mined association rules on GPUS. J Supercomput 66(3):1438–1461CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • L. Bampis
    • 1
  • C. Iakovidou
    • 1
  • S. A. Chatzichristofis
    • 1
  • Y. S. Boutalis
    • 1
  • A. Amanatiadis
    • 1
  1. 1.Department of Electrical and Computer EngineeringDemocritus University of ThraceXanthiGreece

Personalised recommendations