Journal of Real-Time Image Processing

, Volume 14, Issue 3, pp 535–548 | Cite as

Bio-inspired heterogeneous architecture for real-time pedestrian detection applications

  • Luca MaggianiEmail author
  • Cédric Bourrasset
  • Jean-Charles Quinton
  • François Berry
  • Jocelyn Sérot
Special Issue Paper


Along with the development of powerful processing platforms, heterogeneous architectures are nowadays permitting new design space explorations. In this paper, we propose a novel heterogeneous architecture for reliable pedestrian detection applications. It deploys an efficient Histogram of Oriented Gradient pipeline tightly coupled with a neuro-inspired spatio-temporal filter. By relying on hardware–software co-design principles, our architecture is capable of processing video sequences from real-word dynamic environments in real time. The paper presents the implemented algorithm and details the proposed architecture for executing it, exposing in particular the partitioning decisions made to meet the required performance. A prototype implementation is described and the results obtained are discussed with respect to other state-of-the-art solutions.


Heterogeneous FPGA Real-time DNF Co-design 



This work has been sponsored by the French government research programme “Investissements d’avenir” through the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01), by the European Union through the program Regional competitiveness and employment 2007-2013 (ERDF Auvergne region), and by the Auvergne region.


  1. 1.
    Bauer, S., Köhler, S., Doll, K., Brunsmann, U.: FPGA-GPU architecture for kernel SVM pedestrian detection. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp 61–68 (2010)Google Scholar
  2. 2.
    Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Pedestrian detection at 100 frames per second. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 2903–2910 (2012)Google Scholar
  3. 3.
    Benenson, R., Omran, M., Hosang, J., Schiele, B.: Ten years of pedestrian detection, what have we learned? In: Computer Vision-ECCV 2014 Workshops, Springer, pp 613–627 (2014)Google Scholar
  4. 4.
    Blair, C., Robertson, N.M., Hume, D.: Characterizing a heterogeneous system for person detection in video using histograms of oriented gradients: Power versus speed versus accuracy. IEEE J. Emerg. Select. Topics Circuits Syst. 3(2), 236–247 (2013)CrossRefGoogle Scholar
  5. 5.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition vol 1, pp 886–893 (2005)Google Scholar
  6. 6.
    Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pat. Anal. Mach. Intel. 34(4), 743–761 (2012)CrossRefGoogle Scholar
  7. 7.
    Enzweiler, M., Gavrila, D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. Pat. Anal. Mach. Intel. 31(12), 2179–2195 (2009)CrossRefGoogle Scholar
  8. 8.
    Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comp. Vision 88(2), 303–338 (2010)CrossRefGoogle Scholar
  9. 9.
    Gandhi, T., Trivedi, M.: (2007) Pedestrian protection systems: Issues, survey, and challenges. IEEE Trans. Intel. Transp. Syst. 413–430Google Scholar
  10. 10.
    Gavrila, D.M., Munder, S.: Multi-cue pedestrian detection and tracking from a moving vehicle. Int. J. Comp. Vision 73(1), 41–59 (2007)CrossRefGoogle Scholar
  11. 11.
    Hahnle, M., Saxen, F., Hisung, M., Brunsmann, U., Doll, K.: (2013) FPGA-based real-time pedestrian detection on high-resolution images. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 629–635Google Scholar
  12. 12.
    Happe, M., Lübbers, E., Platzner, M.: A self-adaptive heterogeneous multi-core architecture for embedded real-time video object tracking. J. Real-Time Image Process. 8(1), 95–110 (2013)CrossRefGoogle Scholar
  13. 13.
    Hartmann, C., Yupatova, A., Reichenbach, M., Fey, D., German, R.: A holistic approach for modeling and synthesis of image processing applications for heterogeneous computing architectures (2015)Google Scholar
  14. 14.
    Hong, G.S., Kim, B.G., Hwang, Y.S., Kwon, K.K.: Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform. Multim. Tools Appl. (2015)Google Scholar
  15. 15.
    Hua, C., Makihara, Y., Yagi, Y., Iwasaki, S., Miyagawa, K., Li, B.: Onboard monocular pedestrian detection by combining spatio-temporal HOG with structure from motion algorithm. Mach. Vision Appl. 26(2–3), 161–183 (2015)CrossRefGoogle Scholar
  16. 16.
    Hussein, M., Porikli, F., Davis, L.: A comprehensive evaluation framework and a comparative study for human detectors. IEEE Trans. Intel. Transp. Syst. 10(3), 417–427 (2009)CrossRefGoogle Scholar
  17. 17.
    Inggs, G., Thomas, D.B., Luk, W.: An efficient, automatic approach to high performance heterogeneous computing. arXiv preprint arXiv:150504417 (2015)
  18. 18.
    Joachims, T.: Svmlight: support vector machine. (1999)
  19. 19.
    Kadota, R., Sugano, H., Hiromoto, M., Ochi, H., Miyamoto, R., Nakamura, Y.: Hardware architecture for hog feature extraction. In: Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2009. IIH-MSP ’09, pp 1330–1333 (2009)Google Scholar
  20. 20.
    Kim, S.H., Kim, J.S., Wan, V., Suh, I.H.: Automotive adas camera system configuration using multi-core microcontroller. Tech. rep, SAE Technical Paper (2015)Google Scholar
  21. 21.
    Klaser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: BMVC 2008-19th British Machine Vision Conference, British Machine Vision Association, pp 275–281 (2008)Google Scholar
  22. 22.
    Liu, W., Yu, B., Duan, C., Chai, L., Yuan, H., Zhao, H.: A pedestrian-detection method based on heterogeneous features and ensemble of multi-view-pose parts. IEEE Trans. Intel. Transp. Syst. 16(2), 813–824 (2015)Google Scholar
  23. 23.
    Ma, X., Najjar, W., Roy-Chowdhury, A.: Evaluation and acceleration of high-throughput fixed-point object detection on FPGAs. IEEE Trans. Circuits Syst. Video Technol. 25(6), 1051–1062 (2015)CrossRefGoogle Scholar
  24. 24.
    Machida, T., Naito, T.: GPU and CPU cooperative accelerated pedestrian and vehicle detection. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), IEEE, pp 506–513 (2011)Google Scholar
  25. 25.
    Maggiani, L., Salvadori, C., Petracca, M., Pagano, P., Saletti, R.: Reconfigurable architecture for computing histograms in real-time tailored to FPGA-based smart camera. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), IEEE, pp 1042–1046 (2014)Google Scholar
  26. 26.
    Maggiani, L., Bourrasset, C., Berry, F., Sérot, J., Petracca, M., Salvadori, C.: Extraction core for FPGA-based smart cameras. In: Proceedings of the 9th International Conference on Distributed Smart Camera, ACM, pp 128–133 (2015)Google Scholar
  27. 27.
    Maggiani, L., Bourrasset, C., Petracca, M., Berry, F., Pagano, P., Salvadori, C.: HOG-Dot: a parallel kernel-based gradient extraction for embedded image processing. IEEE Signal Process Lett. 22(11), 2132–2136 (2015)CrossRefGoogle Scholar
  28. 28.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Patt. Recogn. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  29. 29.
    Quinton, J.C., Girau, B.: Predictive neural fields for improved tracking and attentional properties. In: The 2011 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1629–1636 (2011)Google Scholar
  30. 30.
    Reiche, O., Haublein, K., Reichenbach, M., Hannig, F., Teich, J., Fey, D.: Automatic optimization of hardware accelerators for image processing. In: DATE Workshop on Heterogeneous Architectures and Design Methods for Embedded Image Systems, HIS 2015 (2015)Google Scholar
  31. 31.
    Tanabe, J., Toru, S., Yamada, Y., Watanabe, T., Okumura, M., Nishiyama, M., Nomura, T., Oma, K., Sato, N., Banno, M. et al.: 18.2 a 1.9 tops and 564gops/w heterogeneous multicore soc with color-based object classification accelerator for image-recognition applications. In: 2015 IEEE International Solid-State Circuits Conference-(ISSCC), IEEE, pp 1–3 (2015)Google Scholar
  32. 32.
    Vangel, B.C.D., Torres-Huitzil, C., Girau, B.: Randomly spiking dynamic neural fields. ACM J. Emerg. Technol. Comp. Syst. (JETC) 11(4), 37 (2015)Google Scholar
  33. 33.
    Vapnik, V.: Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics). Springer-Verlag New York Inc, Secaucus (1982)Google Scholar
  34. 34.
    World Health Organization (2013) Road traffic injuries.
  35. 35.
    Wu, S., Laganière, R., Payeur, P.: Improving pedestrian detection with selective gradient self-similarity feature. Pat. Recogn. 48(8), 2364–2376 (2015)CrossRefGoogle Scholar
  36. 36.
    Yadav, R., Senthamilarasu, V., Kutty, K., Vaidya, V., Ugale, S.: A review on day-time pedestrian detection. Tech. rep, SAE Technical Paper (2015)Google Scholar
  37. 37.
    Yao, S., Pan, S., Wang, T., Zheng, C., Shen, W., Chong, Y.: A new pedestrian detection method based on combined HOG and LSS features. Neurocomputing 151, 1006–1014 (2015)CrossRefGoogle Scholar
  38. 38.
    Zhang, J., Huang, K., Yu, Y., Tan, T.: (2011) Boosted local structured hog-lbp for object localization. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 1393–1400Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Luca Maggiani
    • 1
    • 2
    Email author
  • Cédric Bourrasset
    • 2
  • Jean-Charles Quinton
    • 2
    • 3
  • François Berry
    • 2
  • Jocelyn Sérot
    • 2
  1. 1.TeCIP Institute, Scuola Superiore Sant’AnnaPisaItaly
  2. 2.Institute Pascal, Universite Blaise PascalClermont FerrandFrance
  3. 3.Laboratory Jean KuntzmannGrenoble Alpes UniversityGrenobleFrance

Personalised recommendations