Abstract
With the technological improvement in artificial intelligence, particularly deep learning is providing effective outcomes along with hardware platforms such as field-programmable gate arrays (FPGAs) and graphics processing units in various domains. FPGAs with their reconfigurable architectures provide flexibility, better performance and high levels of parallelism. Object detection is one of the prominent areas of research in the fields of computer vision and image processing applications. You Only Look Once (YOLO) is a state-of-the art object detection algorithm which is fast and accurate. However, many applications require accuracy and rapid processing for better results. For such conditions, these algorithms can be implemented on hardware accelerators. This work proposes the implementation of YOLOv4 algorithm on Xilinx® Zynq-7000 System on a chip and is suitable for real-time object detection. The proposed work shows better resource utilization of about 23.2 k (43.6%) of Look-up tables, 45.8 k (43.04%) of Flip-flops, 115 (82.17%) BRAMs and 174 (79%) DSPs achieving at 100 MHz frequency which is more efficient on comparing with other simulation results.
Similar content being viewed by others
References
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., et al.: SSD: Single Shot MultiBox Detector. In: European Conference on Computer Vision, Cham, Switzerland, pp. 21–37 (2016)
He, K., Zhang, X., Ren S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision, Cham, Switzerland, pp. 346–361 (2014)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 580–587(2014)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern. Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 7263–7271 (2017)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection (2020). arXiv:2004.10934
Boutros, A., Yazdanshenas, S., Betz, V.: You cannot improve what you do not measure: FPGA vs. ASIC efficiency gaps for convolutional neural network inference. ACM Trans. Reconfig. Technol. Syst. 11(3), 1–23 (2018)
Lin, T.Y., Maire M., Belongie, S., Hays, J., et al.: Microsoft COCO: common objects in context (2015). arXiv:1405.0312
Shawahna, A., Sait, S.M., El-Maleh, A.: FPGA-based accelerators of deep learning networks for learning and classification: a review. IEEE Access 7, 7823–7859 (2019)
Wang, E., Davis, J., Zhao, R., Ng, H.-C., et al.: Deep neural network approximation for custom hardware where we have been, where we are going. ACM Comput. Surv. 52(2), 1–39 (2019)
Dias, M.A., Ferreira, D.A.P.: Deep learning in reconfigurable hardware: a survey. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Rio de Janeiro, Brazil, pp. 95–98 (2019)
Blaiech, A.G., Khalifa, K.-B., Valderrama, C.V., et al.: A survey and taxonomy of FPGA-based deep learning accelerators. J. Syst. Architect. 98, 331–345 (2019)
HajiRassouliha, A., Taberner, A.J., Nash, M.P., Nielsen, P.M.F.: Suitability of recent hardware accelerators (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms. Signal Process. Image Commun. 68, 101–119 (2018)
Babu, P., Parthasarathy, E.: Reconfigurable FPGA architectures. A survey and applications. J. Inst. Eng. India Ser. B 102, 143–156 (2021)
Tong, K., Wu, Y., Zhou, F.: Recent advances in small object detection based on deep learning: a review. Image Vis. Comput. 97, 103910 (2020)
El-Shafie, A.-H.A., Habib, S.E.: Survey on hardware implementations of visual object trackers. IET Image Process. 13, 863–876 (2019)
Ding, C., Wang, S., Liu, N., Xu, K., et al.: REQ-YOLO: a resource-aware, efficient quantization framework for object detection on FPGAs. In: 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Seaside, CA, USA, pp. 33–42 (2019)
Wang, J., Lin, J., Wang, Z.: Efficient hardware architectures for deep convolutional neural network. IEEE Trans. Circuits Syst. I Regul. Pap. 65(6), 1941–1953 (2018)
Mao, Q.-C., Sun, H.-M., Liu, Y.-B., Jia, R.-S.: Mini-YOLOv3: real-time object detector for embedded applications. IEEE Access 7, 133529–133538 (2019)
Nakahara, H., Yonekawa, H., Tomoya, F., Sato, S.: A lightweight YOLOv2: a binarized CNN with a parallel support vector regression for an FPGA. In: 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA, pp. 31–40 (2018)
Nguyen, D.T., Nguyen, T.N., Kim, H., Lee, H.-J.: A high-throughput and power-efficient FPGA implementation of YOLO CNN for object detection. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 27(8), 1861–1873 (2019)
Yu, Z., Bouganis, C.S.: A parameterisable FPGA-tailored architecture for YOLOv3-Tiny. Proc. International Symposium on Applied Reconfigurable Computing, Cham, Switzerland, pp. 330–344 (2020)
Babu. P., Parthasarathy. E.: FPGA implementation of multi-dimensional kalman filter for object tracking and motion detection. Eng. Sci. Technol. 33, 101084 (2022). https://doi.org/10.1016/j.jestch.2021.101084
Guo, K., Siu, L., Qiu, J., Yao, S., et al.: Angel-eye: a complete design flow for mapping CNN onto customized hardware. In: IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Pittsburgh, PA, USA, pp. 24–29 (2016)
Wei, G., Hou, Y., Cui, Q., Deng, G., et al.: YOLO Acceleration using FPGA architecture. In: IEEE/CIC International Conference on Communications in China (ICCC), Beijing, China, pp. 734–735 (2018)
Zhang, C., Li, P., Sun, G., Guan, Y., et al.: Optimizing FPGA-based accelerator design for deep convolutional neural networks. In: 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA, pp. 161–170 (2018)
Çambay, V.Y., Uçar, A., Arserim, M.A.: Object detection on FPGAs and GPUs by using accelerated deep learning. In: International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey, pp. 1–5 (2019)
Pestana, D., Miranda, P.-R., Lopes, J.-D., Duarte, R.-P., Vestias, M.-P., Neto, H.-C., De Sousa, J.-T.: A Full featured configurable accelerator for object detection with YOLO. IEEE Access 9, 75864–75877 (2021)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement (2018). arXiv:1804.02767
Liu, S., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 8759–8768 (2018)
Lin, T., Dollár, P., Girshick, R., He, K., et al.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 936–944 (2017)
Zhou, D., Fang, J., Song, X., Guan, C., et al.: IoU loss for 2D/3D object detection. In: International Conference on 3D Vision (3DV), Quebec City, QC, Canada, pp. 85–94 (2019)
Xilinx Inc.: Zynq SoC datasheet: overview (2018)
Xilinx Inc.: Zynq-7000 all programmable SoC technical reference manual (2021)
Xilinx Inc.: Xilinx Machine Learning TRD Guide (2019)
Praveenkumar, B., Eswaran, P.: Hardware/software acceleration of image and video processing on Xilinx Zynq platform. Intell. Autom. Soft Comput. 30(3), 1063–1071 (2021)
Praveenkumar, B., Eswaran, P.: Optimized object detection method for FPGA implementation. In 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), IEEE, pp. 72–74 (2021)
Acknowledgements
The authors would like to thank SRM Institute of Science and Technology for providing facilities to carry out the research work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Babu, P., Parthasarathy, E. Hardware acceleration for object detection using YOLOv4 algorithm on Xilinx Zynq platform. J Real-Time Image Proc 19, 931–940 (2022). https://doi.org/10.1007/s11554-022-01234-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-022-01234-y