A fully pipelined and parallel hardware architecture for real-time BRISK salient point extraction

  • Ehsan Azimi
  • Alireza Behrad
  • Mohammad Bagher Ghaznavi-Ghoushchi
  • Jamshid Shanbehzadeh
Original Research Paper
  • 87 Downloads

Abstract

Scale and rotation invariant salient point detection and matching algorithms are variously used in computer vision applications such as image matching, 3D localization and pose estimation. Recently, hardware implementation of image and video processing algorithms has emerged as a viable solution to handle the high computational complexity of applications like 3D pose estimation with several processing stages. The hardware implementation of various stages of theses algorithms can be executed in a pipelined manner to ensure the reality of time. In this paper, a new and fully pipelined hardware architecture is proposed for salient point detection using Binary Robust Invariant Scalable Keypoints (BRISK) algorithm. BRISK algorithm is a binary keypoint extractor that detects salient points by constructing a scale-space pyramid; therefore, its fixed-point hardware implementation in a pipelined manner is challenging because of the required synchronization for various layers in scale domain. The proposed hardware architecture was implemented using Verilog Hardware Description Language, and the functionality of the design was validated through several experiments. The proposed design was synthesized by using an ASIC digital design flow utilizing 180 nm CMOS technology as well as a Virtex-4 FPGA. The design is clocked at 90.91 MHz in ASIC implementation and achieves processing rate of 169.29 frames/s while running on input images with 800 × 600 resolution. The throughput of FPGA implementation is 180.44 frames/s with 96.89 MHz clock frequency for the same input image resolution. Experimental results confirm the efficiency of the proposed hardware architecture in comparison with software implementation.

Keywords

BRISK salient points Feature extraction Application-specific integrated circuit (ASIC) Hardware implementation 

References

  1. 1.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the Fourth Alvey Vision Conference 1988, pp. 147–151Google Scholar
  2. 2.
    Moravec, H.P.: Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover. Stanford University, Stanford (1980)Google Scholar
  3. 3.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 593–600. IEEE (1994)Google Scholar
  4. 4.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  5. 5.
    Yao, L., Feng, H., Zhu, Y., Jiang, Z., Zhao, D., Feng, W.: An architecture of optimised SIFT feature detection for an FPGA implementation of an image matcher. In: International Conference on Field-Programmable Technology (FPT), pp. 30–37. IEEE (2009)Google Scholar
  6. 6.
    Grabner, M., Grabner, H., Bischof, H.: Fast approximated SIFT. In: 7th Asian Conference on Computer Vision (ACCV), Hyderabad, India, pp. 918–927. Springer, Berlin (2006)Google Scholar
  7. 7.
    Ke, Y., Sukthankar, R.: PCA-SIFT: A more distinctive representation for local image descriptors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, Vol. 502, pp. II-506–II-513. IEEE (2004)Google Scholar
  8. 8.
    Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: 9th European Conference on Computer vision (ECCV), Graz, Austria, pp. 404–417. Springer, Berlin (2006)Google Scholar
  9. 9.
    Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: Tenth IEEE International Conference on Computer Vision (ICCV) Beijing, China, pp. 1508–1515. IEEE (2005)Google Scholar
  10. 10.
    Mair, E., Hager, G.D., Burschka, D., Suppa, M., Hirzinger, G.: Adaptive and generic corner detection based on the accelerated segment test. In: 11th European Conference on Computer Vision (ECCV), Heraklion, Crete, Greece, pp. 183–196. Springer, Berlin (2010)Google Scholar
  11. 11.
    Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, pp. 2548–2555. IEEE (2011)Google Scholar
  12. 12.
    Pedre, S., Krajník, T., Todorovich, E., Borensztejn, P.: Accelerating embedded image processing for real time: a case study. J. Real-Time Image Proc. 11(2), 349–374 (2016)CrossRefGoogle Scholar
  13. 13.
    Velez, G., Cortés, A., Nieto, M., Vélez, I., Otaegui, O.: A reconfigurable embedded vision system for advanced driver assistance. J. Real-Time Image Proc. 10(4), 725–739 (2015)CrossRefGoogle Scholar
  14. 14.
    Wang, W., Yan, J., Xu, N., Wang, Y., Hsu, F.-H.: Real-time high-quality stereo vision system in FPGA. IEEE Trans. Circuits Syst. Video Technol. 25(10), 1696–1708 (2015)CrossRefGoogle Scholar
  15. 15.
    Sivanantham, S., Paul, N.N., Iyer, R.S.: Object tracking algorithm implementation for security applications. Far East J. Electron. Commun. 16(1), 1–13 (2016)CrossRefGoogle Scholar
  16. 16.
    Azizabadi, M., Behrad, A., Ghaznavi-Ghoushchi, M.: VLSI implementation of star detection and centroid calculation algorithms for star tracking applications. J. Real-Time Image Proc. 9(1), 127–140 (2014)CrossRefGoogle Scholar
  17. 17.
    Araneda, L., Figueroa, M.: A compact hardware architecture for digital image stabilization using integral projections. Microprocess. Microsyst. 39(8), 987–997 (2015)CrossRefGoogle Scholar
  18. 18.
    Bonato, V., Marques, E., Constantinides, G.: A parallel hardware architecture for scale and rotation invariant feature detection. IEEE Trans. Circuits Syst. Video Technol. 18(12), 1703–1712 (2008)CrossRefGoogle Scholar
  19. 19.
    Huang, F.-C., Huang, S.-Y., Ker, J.-W., Chen, Y.-C.: High-performance SIFT hardware accelerator for real-time image feature extraction. IEEE Trans. Circuits Syst. Video Technol. 22(3), 340–351 (2012)CrossRefGoogle Scholar
  20. 20.
    Suzuki, T., Ikenaga, T.: SIFT-based low complexity keypoint extraction and its real-time hardware implementation for full-HD video. In: 2012 Asia-Pacific Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), California, USA, pp. 1–6. IEEE (2012)Google Scholar
  21. 21.
    Chang, L., Hernández-Palancar, J., Sucar, L.E., Arias-Estrada, M.: FPGA-based detection of SIFT interest keypoints. Mach. Vis. Appl. 24(2), 371–392 (2013)CrossRefGoogle Scholar
  22. 22.
    Zhong, S., Wang, J., Yan, L., Kang, L., Cao, Z.: A real-time embedded architecture for SIFT. J. Syst. Architect. 59(1), 16–29 (2013)CrossRefGoogle Scholar
  23. 23.
    Chiu, L.-C., Chang, T.-S., Chen, J.-Y., Chang, N.Y.-C.: Fast SIFT design for real-time visual feature extraction. IEEE Trans. Image Process. 22(8), 3158–3167 (2013)CrossRefGoogle Scholar
  24. 24.
    Jiang, J., Li, X., Zhang, G.: SIFT hardware implementation for real-time image feature extraction. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1209–1220 (2014)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Sledevič, T., Serackis, A.: SURF algorithm implementation on FPGA. In: 13th Biennial Baltic Electronics Conference (BEC), Tallinn, Estonia, pp. 291–294. IEEE (2012)Google Scholar
  26. 26.
    Wilson, C., Zicari, P., Craciun, S., Gauvin, P., Carlisle, E., George, A., Lam, H.: A power-efficient real-time architecture for SURF feature extraction. In: International Conference on ReConFigurable Computing and FPGAs (ReConFig), pp. 1–8. IEEE (2014)Google Scholar
  27. 27.
    Zhang, W., Liu, L., Yin, S., Zhou, R., Cai, S., Wei, S.: An efficient VLSI architecture of speeded-up robust feature extraction for high resolution and high frame rate video. Sci. China Inf. Sci. 56(7), 1–14 (2013)MathSciNetGoogle Scholar
  28. 28.
    Battezzati, N., Colazzo, S., Maffione, M., Senepa, L.: SURF algorithm in FPGA: a novel architecture for high demanding industrial applications. In: Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, pp. 161–162 (2012)Google Scholar
  29. 29.
    Krajník, T., Šváb, J., Pedre, S., ížek P, P., Peuil, L.: FPGA-based module for SURF extraction. Mach. Vis. Appl. 25(3), 787–800 (2014)CrossRefGoogle Scholar
  30. 30.
    Heo, H., Lee, J.-Y., Lee, K.-Y., Lee, C.-H.: FPGA based implementation of FAST and BRIEF algorithm for object recognition. In: TENCON 2013–2013 IEEE Region 10 Conference (31194), pp. 1–4. IEEE (2013)Google Scholar
  31. 31.
    Park, J.-S., Kim, L.-S.: Hardware accelerator for feature point detection and matching. In: Jaeseok Kim, H.S. (ed.) Algorithm & SoC Design for Automotive Vision Systems, pp. 197–230. Springer, New York (2014)Google Scholar
  32. 32.
    Heinly, J., Dunn, E., Frahm, J.-M.: Comparative evaluation of binary features. In: Computer Vision–ECCV 2012, pp. 759–773. Springer, NewYork (2012)Google Scholar
  33. 33.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)Google Scholar
  34. 34.
    Bekele, D., Teutsch, M., Schuchert, T.: Evaluation of binary keypoint descriptors. In: 20th IEEE International Conference on Image Processing (ICIP), pp. 3652–3656. IEEE (2013)Google Scholar
  35. 35.
    Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: Fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)Google Scholar
  36. 36.
    Canclini, A., Cesana, M., Redondi, A., Tagliasacchi, M., Ascenso, J., Cilla, R.: Evaluation of low-complexity visual feature detectors and descriptors. In: 18th International Conference on Digital Signal Processing (DSP), pp. 1–7. IEEE (2013)Google Scholar
  37. 37.
    Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar
  38. 38.
    Oxford Image Dataset. http://www.robots.ox.ac.uk/~vgg/data/data-aff.html. Accessed Feb 2017
  39. 39.
    Bouris, D., Nikitakis, A., Papaefstathiou, I.: Fast and efficient FPGA-based feature detection employing the SURF algorithm. In: 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 3–10. IEEE (2010)Google Scholar
  40. 40.
    Schaeferling, M., Kiefer, G.: Flex-SURF: A flexible architecture for FPGA-based robust feature extraction for optical tracking systems. In: 2010 International Conference on Reconfigurable Computing and FPGAs (ReConFig), pp. 458–463. IEEE (2010)Google Scholar
  41. 41.
    Lee, S.-S., Jang, S.-J., Kim, J., Hwang, Y., Choi, B.: Memory-efficient SURF architecture for ASIC implementation. Electron. Lett. 50(15), 1058–1059 (2014)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Ehsan Azimi
    • 1
  • Alireza Behrad
    • 2
  • Mohammad Bagher Ghaznavi-Ghoushchi
    • 2
  • Jamshid Shanbehzadeh
    • 3
  1. 1.Department of Computer Engineering, Science and Research BranchIslamic Azad UniversityTehranIran
  2. 2.Electrical Engineering DepartmentShahed UniversityTehranIran
  3. 3.Department of Computer EngineeringKharazmi UniversityTehranIran

Personalised recommendations