Skip to main content
SpringerLink
Log in

Affordable person detection in omnidirectional cameras using radial integral channel features

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Omnidirectional cameras cover more ground than perspective cameras, at the expense of resolution. Their comprehensive field of view makes omnidirectional cameras appealing for security and ambient intelligence applications. Person detection is usually a core part of such applications. Conventional methods fail for omnidirectional images due to different image geometry and formation. In this study, we propose a method for person detection in omnidirectional images, which is based on the integral channel features approach. Features are extracted from various channels, such as LUV and gradient magnitude, and classified using boosted decision trees. Features are pixel sums inside annular sectors (doughnut slice shapes) contained by the detection window. We also propose a novel data structure called radial integral image that allows to calculate sums inside annular sectors efficiently. We have shown with experiments that our method outperforms the previous state of the art and uses significantly less computational resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The code is publicly available at https://github.com/barisdemiroz/radial_integral_image.

  2. The code is publicly available at https://github.com/barisdemiroz/adaboost_cpp.

  3. IYTE dataset—available at http://cvrg.iyte.edu.tr.

References

  1. Aghajan, H., Cavallaro, A.: Multi-camera Networks: Principles and Applications. Academic Press, Cambridge (2009)

    Google Scholar 

  2. Agrawal, M., Konolige, K., Blas, M.R.: Censure: center surround extremas for realtime feature detection and matching. In: European Conference on Computer Vision, pp. 102–115. Springer (2008)

  3. Alahi, A., Boursier, Y., Jacques, L., Vandergheynst, P.: Sport players detection and tracking with a mixed network of planar and omnidirectional cameras. In: International Conference on Distributed Smart Cameras (ICDSC), pp. 1–8. IEEE (2009). https://doi.org/10.1109/ICDSC.2009.5289406

  4. Alahi, A., Jacques, L., Boursier, Y., Vandergheynst, P.: Sparsity driven people localization with a heterogeneous network of cameras. J. Math. Imaging Vis. 41(1–2), 39–58 (2011). https://doi.org/10.1007/s10851-010-0258-7

    Article  MathSciNet  MATH  Google Scholar 

  5. Appel, R., Fuchs, T., Dollár, P., Perona, P.: Quickly boosting decision trees–pruning underachieving features early. In: International Conference on Machine Learning, pp. 594–602 (2013)

  6. Arican, Z., Frossard, P.: OmniSIFT: Scale invariant features in omnidirectional images. In: International Conference on Image Processing, pp. 3505–3508. IEEE (2010)

  7. Benenson, R., Mathias, M., Tuytelaars, T., Van Gool, L.: Seeking the strongest rigid detector. In: Computer Vision and Pattern Recognition, pp. 3666–3673 (2013). https://doi.org/10.1109/CVPR.2013.470

  8. Bentley, J.L.: Multidimensional divide-and-conquer. Commun. ACM 23(4), 214–229 (1980). https://doi.org/10.1145/358841.358850

    Article  MathSciNet  MATH  Google Scholar 

  9. Cao, J., Pang, Y., Li, X.: Learning multilayer channel features for pedestrian detection. IEEE Trans. Image Process. 26(7), 3210–3220 (2017). https://doi.org/10.1109/TIP.2017.2694224

    Article  MathSciNet  MATH  Google Scholar 

  10. Cinaroglu, I., Bastanlar, Y.: A direct approach for object detection with catadioptric omnidirectional cameras. Signal Image and Video Process. 10(2), 413–420 (2016). https://doi.org/10.1007/s11760-015-0768-2

    Article  Google Scholar 

  11. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, vol. I, pp. 886–893. IEEE, San Diego (2005). https://doi.org/10.1109/CVPR.2005.177

  12. Delannay, D., Danhier, N., De Vleeschouwer, C.: Detection and recognition of sports (wo) men from multiple views. In: Distributed Smart Cameras, 2009. ICDSC 2009. Third ACM/IEEE International Conference on, pp. 1–7. IEEE (2009)

  13. Demiröz, B.E., Salah, A.A., Akarun, L.: Coupling fall detection and tracking in omnidirectional cameras. In: International Workshop on Human Behavior Understanding (HBU@ECCV), pp. 73–85 (2014)

  14. Dollar, P., Belongie, S., Perona, P.: The fastest pedestrian detector in the west. In: British Machine Vision Conference, pp. 68.1–68.11 (2010). https://doi.org/10.5244/C.24.68

  15. Dollár, P., Tu, Z., Perona, P., Belongie, S.: Integral channel features. In: British Machine Vision Conference, pp. 1–11 (2009). https://doi.org/10.5244/C.23.91

  16. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: a benchmark. In: Computer Vision and Pattern Recognition, pp. 304–311 (2009). https://doi.org/10.1109/CVPRW.2009.5206631

  17. Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 743–761 (2012). https://doi.org/10.1109/TPAMI.2011.155

    Article  Google Scholar 

  18. Ehsan, S., Clark, A.F., ur Rehman, N., McDonald-Maier, K.D.: Integral images: efficient algorithms for their computation and storage in resource-constrained embedded vision systems. Sensors 15(7), 16804–16830 (2015)

    Article  Google Scholar 

  19. Felzenswalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. PAMI 32(9), 1627–1645 (2010). https://doi.org/10.1109/MC.2014.42

    Article  Google Scholar 

  20. Geyer, C., Daniilidis, K.: A unifying theory for central panoramic systems and practical implications. In: European Conference on Computer Vision, pp. 445–461. Springer (2000). https://doi.org/10.1007/3-540-45053-X_29

  21. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  22. Hu, Q., Wang, P., Shen, C., van den Hengel, A., Porikli, F.: Pushing the Limits of Deep CNNs for Pedestrian Detection. IEEE Transactions on Circuits and Systems for Video Technology pp. 1–1 (2017). https://doi.org/10.1109/TCSVT.2017.2648850

  23. Kang, S., Roh, A., Nam, B., Hong, H.: People detection method using graphics processing units for a mobile robot with an omnidirectional camera. Opt. Eng. 50(12), 127204 (2011). https://doi.org/10.1117/1.3660573

    Article  Google Scholar 

  24. Karaimer, H.C., Baştanlar, Y.: Car detection with omnidirectional cameras using Haar-like features and cascaded boosting. In: IEEE Signal Processing and Communications Applications Conference, pp. 301–304 (2014). https://doi.org/10.1109/SIU.2014.6830225

  25. Kobilarov, M., Sukhatme, G., Hyams, J., Batavia, P.: People tracking and following with mobile robot using an omnidirectional camera and a laser. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 557–562 (2006)

  26. Liu, B., Wu, H., Su, W., Zhang, W., Sun, J.: Rotation-invariant object detection using sector-ring hog and boosted random ferns. Vis. Comput. 34(5), 707–719 (2018)

    Article  Google Scholar 

  27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer (2016)

  28. Lourenço, M., Barreto, J.P., Vasconcelos, F.: sRD-SIFT: keypoint detection and matching in images with radial distortion. IEEE Trans. Robot. 28(3), 752–760 (2012)

    Article  Google Scholar 

  29. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  30. Messom, C.H., Barczak, A.L.: Stream processing for fast and efficient rotated haar-like features using rotated integral images. Int. J. Intell. Syst. Technol. Appl. 7(1), 40–57 (2009)

    Google Scholar 

  31. Ouyang, W., Wang, X.: Joint deep learning for pedestrian detection. In: International Conference on Computer Vision, pp. 2056–2063 (2013). https://doi.org/10.1109/ICCV.2013.257

  32. Puig, L., Guerrero, J.J.: Scale space for central catadioptric systems: Towards a generic camera feature extractor. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1599–1606 (2011). https://doi.org/10.1109/ICCV.2011.6126420

  33. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  34. Saito, M., Kitaguchi, K., Kimura, G., Hashimoto, M.: People detection and tracking from fish-eye image based on probabilistic appearance model. SICE 1, 435–440 (2011)

    Google Scholar 

  35. Trichet, R., Bremond, F.: LBP channels for pedestrian detection. In: Winter Conference on Applications of Computer Vision. Lake Tahoe (2018)

  36. Tuzel, O., Porikli, F., Meer, P.: Pedestrian detection via classification on Riemannian manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1713–1727 (2008). https://doi.org/10.1109/TPAMI.2008.75

    Article  Google Scholar 

  37. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. Comput. Vis. Pattern Recognit. 1, 511–518 (2001). https://doi.org/10.1109/CVPR.2001.990517

    Google Scholar 

  38. Wang, M.L., Lin, H.Y.: Object recognition from omnidirectional visual sensing for mobile robot applications. In: IEEE International Conference on Systems, Man and Cybernetics, October 2009, pp. 1941–1946 (2009). https://doi.org/10.1109/ICSMC.2009.5345895

  39. Zhang, L., Lin, L., Liang, X., He, K.: Is faster R-CNN doing well for pedestrian detection? In: Lecture Notes in Computer Science, 9906 LNCS, pp. 443–457 (2016). https://doi.org/10.1007/978-3-319-46475-6_28

  40. Zhang, S., Benenson, R., Omran, M., Hosang, J., Schiele, B.: How far are we from solving pedestrian detection? In: Computer Vision and Pattern Recognition (2016). https://doi.org/10.1109/CVPR.2016.141

  41. Zhu, Q., Avidan, S., Yeh, M.C., Cheng, K.T.: Fast human detection using a cascade of histograms of oriented gradients. Comput. Vis. Pattern Recognit. 2, 1491–1498 (2006). https://doi.org/10.1109/CVPR.2006.119

    Google Scholar 

  42. Zivkovic, Z., Krose, B.: Part based people detection using 2D range data and images. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 214–219 (2007)

Download references

Acknowledgements

The numerical calculations reported in this paper were partially performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources). This study has been funded by the Turkish Ministry of Development under the TAM Project number DPT2007K120610. Part of the work was performed when B. E. Demiröz was with NVIDIA and A. A. Salah was with Nagoya University, Future Value Creation Research Center.

Author information

Authors and Affiliations

  1. Computer Engineering Department, Boğaziçi University, 34342, Istanbul, Turkey

    Barış Evrim Demiröz, Albert Ali Salah & Lale Akarun

  2. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    Albert Ali Salah

  3. Department of Computer Engineering, Izmir Institute of Technology, 35430, Urla, Izmir, Turkey

    Yalin Bastanlar

Authors
  1. Barış Evrim Demiröz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Albert Ali Salah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yalin Bastanlar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lale Akarun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barış Evrim Demiröz.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Demiröz, B.E., Salah, A.A., Bastanlar, Y. et al. Affordable person detection in omnidirectional cameras using radial integral channel features. Machine Vision and Applications 30, 645–655 (2019). https://doi.org/10.1007/s00138-019-01016-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01016-w

Keywords

Search

Navigation