Skip to main content
SpringerLink
Log in
Book cover

Emerging Trends and Advanced Technologies for Computational Intelligence pp 365–380Cite as

Depth and Thermal Image Fusion for Human Detection with Occlusion Handling Under Poor Illumination from Mobile Robot

  • Chapter
  • First Online:

  • 862 Accesses

Part of the book series: Studies in Computational Intelligence ((SCI,volume 647))

Abstract

In this paper we present a vision-based approach to detect multiple persons with occlusion handling from a mobile robot in real-world scenarios under two lighting conditions, good illumination (lighted) and poor illumination (dark). We use depth and thermal information that are fused for occlusion handling. First, a classifier is trained using thermal images of the human upper-body. This classifier is used to obtain the bounding box coordinates of human. The depth image is later fused with the region of interest obtained from the thermal image. Using the initial bounding box, occlusion handling is performed to determine the final position of human in the image. The proposed method significantly improves human detection even in crowded scene and poor illumination.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Xia, L., Chen, C., Aggarwal, J.K.: Human detection using depth information by kinect. In: Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE, pp. 15–22 (2011)

    Google Scholar 

  2. Eng, H., Wang, J., Kam, A.H., Yau, W.A.: Bayesian framework for robust human detection and occlusion handling using human shape model. In: Proceedings of the 17th International Conference on ICPR 2004, vol. 2, pp. 257–260 (2004)

    Google Scholar 

  3. Kamijo, S., Matsushita, Y., Ikeuchi, K., Sakauchi, M.: Occlusion robust tracking utilizing spatio-temporal Markov random field model. In: Proceedings of 15th International Conference on Pattern Recognition ICPR-2000 [Internet], vol. 1, pp. 140–144. IEEE Computer Society (200). http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=905292

  4. Otsuka, K., Mukawa, N.: Multiview occlusion analysis for tracking densely populated objects based on 2-D visual angles. In: Proceedings of 2004 IEEE Computer Society Conference on Computer Vision Pattern Recognition, 2004 CVPR 2004 [Internet], vol. 1, pp. 90–97. IEEE (2004). http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1315018

  5. Ikemura, S., Fujiyoshi, H.: Real-time human detection using relational depth similarity features. Computer Vision—ACCV 2010. Lecture Notes in Computer Science, vol. 6495, pp. 25–38 (2011)

    Google Scholar 

  6. Camps, O., Sznaier, M.: Segmentation for robust tracking in the presence of severe occlusion. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001 CVPR 2001. pp. 483–489 (2001)

    Google Scholar 

  7. Almerfors, A.: Infrared Camera [Internet] (2009). http://www.applegate.co.uk/b2b-news-articles/more-detail-regarding-infrared-camera-0020792.htm?view=NEWS_105027

  8. Zhang, H., Reardon, C., Parker, L.E.: Real-time multiple human perception with color-depth cameras on a mobile robot. IEEE Trans Cybern. 43(5), 1429–41 (2013)

    Article  Google Scholar 

  9. Varshney, P.K.: Multisensor data fusion. Electron Commun. Eng. J. 245–253 (1997)

    Google Scholar 

  10. Hadi, H.S., Rosbi, M., Sheikh, U.U.: A review of infrared spectrum in human detection for surveillance systems. Int. J. Interact Digit Med. 1(3), 13–20 (2013)

    Google Scholar 

  11. Ding, J., Kuo, C., Hong, W.: An efficient image segmentation technique by Graduate Institute of Communication Engineering, National Taiwan University. In: The 21th IPPR Conference on Computer Vision, Graphics and Image Processing (2009). http://djj.ee.ntu.edu.tw/FastS

  12. Alonso, I.P., Llorca, D.F., Sotelo, M.Á., Bergasa, L.M., Member, A., De, Toro P.R., et al.: Combination of feature extraction methods for SVM pedestrian detection. IEEE Trans. Intell. Transp. Syst. 8(2), 292–307 (2007)

    Article  Google Scholar 

  13. Wu, S., Yu, S., Chen, W.: An attempt to pedestrian detection. In: Third Chinese Conference on Intelligent Visual Surveillance (IVS), pp. 9–12 (2011)

    Google Scholar 

  14. Zhang, S., Liu, J.: A self-occlusion detection approach based on range image of vision object 2041–2046 (2011)

    Google Scholar 

  15. Zhang, S., Liu, J.: A self-occlusion detection approach based on depth image using SVM. Int. J. Adv. Robot Syst. 9, 230 (2012). http://www.intechopen.com/journals/international_journal_of_advanced_robotic_systems/a-self-occlusion-detection-approach-based-on-depth-image-using-svm

    Google Scholar 

  16. Mallat, S.G.: A theory for multiresolution signal decomposition? The wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. II(7), 674–693 (1989)

    Article  MATH  Google Scholar 

  17. Oren, M., Papageorgiou, C., Sinha, P., Osuna, E., Poggio, T.: Pedestrian detection using wavelet templates. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 193–199. IEEE Computer Society (1997)

    Google Scholar 

  18. Shimizu, H., Poggio, T.: Direction estimation of pedestrian from multiple §\({}\) tidl images 596–600 (2004)

    Google Scholar 

  19. Koenig, N.: Toward real-time human detection and tracking in diverse environments. In: 2007 ICDL 2007 IEEE 6th International Conference on Development and Learning, pp. 94–98 (2007)

    Google Scholar 

  20. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: European Conference on Computer Vision, pp. 428–441 (2006)

    Google Scholar 

  21. Chang, T.-H., Gong, S., Ong, E.-J.: Tracking multiple people under occlusion using multiple cameras. In: Proceedings of 11th British Machine Vision Conference (2000)

    Google Scholar 

  22. Dockstader, S.L., Tekalp, A.M.: Multiple camera fusion for multi-object tracking. In: Proceedings of IEEE Workshop on Multi-Object Tracking. IEEE Computer Society, pp. 95–102 (2001)

    Google Scholar 

  23. Dockstader, S.L., Tekalp, A.M.: Multiple camera tracking of interacting and occluded human motion. Proc. IEEE 89(10), 1441–1455 (2001)

    Google Scholar 

  24. Cucchiara, R., Grana, C., Tardini, G., Vezzani, R., Emilia, R.: Probabilistic people tracking for occlusion handling. In: Proceedings of the 17th International Conference on ICPR 2004, pp. 132–135 (2004)

    Google Scholar 

  25. Wu, Y., Yu, T., Hua, G.: Tracking appearances with occlusions. In: 2003 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. I-789–I-795. IEEE Computer Society (2003)

    Google Scholar 

  26. Siebel, N.T., Maybank, S.: Fusion of multiple tracking algorithms for robust people tracking. In: 7th European Conference on Computer Vision, Denmark, vol. IV, pp. 373–387 (2002)

    Google Scholar 

  27. Senior, A., Hampapur, A., Tian, Y.-L., Brown, L., Pankanti, S., Bolle, R.: Appearance models for occlusion handling. Image Vis. Comput. 24(11), 1233–1243 (2006)

    Article  Google Scholar 

  28. Nguyen, H.T., Smeulders, A.W.M.: Fast occluded object tracking by a robust appearance filter. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1099–1104 (2004)

    Article  Google Scholar 

  29. Tao, H., Sawhney, H.S., Kumar, R., Rd, W.: Dynamic layer representation with applications to tracking (2000)

    Google Scholar 

  30. Wloka, M,M., Anderson, B.G.: Resolving occlusion in augmented reality. In: Proceedings of the 1995 Symposium on Interactive 3D Graphics, pp. 5–12 (1995)

    Google Scholar 

  31. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of 2001 IEEE Computer Society Conference on Computer Vision Pattern Recognition CVPR 2001, vol. 1, pp. I-511–I-518. IEEE Computer Society (2001)

    Google Scholar 

  32. Shoelson, B.: Cascade trainer: specify ground truth, train a detector [Internet] (2013). http://www.mathworks.com/matlabcentral/fileexchange/39627-cascade-trainer--specify-ground-truth--train-a-detector

  33. Hadi, H.S., Rosbi, M., Sheikh, U.U., Amin, S.H.M.: Improved occlusion handling for human detection from mobile robot. In: Science and Information Conference (SAI), IEEE, pp. 694–698 (2015)

    Google Scholar 

  34. Chai, T., Draxler, R.R.: Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geosci. Model Dev. 7(3), 1247–1250 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Faculty of Electrical Engineering of Universiti Teknologi Malaysia for providing technical support for this research work. The authors are also grateful to the financial aid from Centre for Artificial Intelligence and Robotics (CAIRO) and Computer Vision, Video and Image Processing Research Lab (CvviP) of Universiti Teknologi Malaysia.

Author information

Authors and Affiliations

  1. Control and Mechatronics Engineering Department, Universiti Teknologi Malaysia, 81310, Johore, Malaysia

    Saipol Hadi Hasim, Rosbi Mamat & Shamsuddin Hj. Mohd. Amin

  2. Electronics and Computer Engineering Department, Universiti Teknologi Malaysia, 81310, Johore, Malaysia

    Usman Ullah Sheikh

Authors
  1. Saipol Hadi Hasim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rosbi Mamat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Usman Ullah Sheikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shamsuddin Hj. Mohd. Amin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Saipol Hadi Hasim , Rosbi Mamat , Usman Ullah Sheikh or Shamsuddin Hj. Mohd. Amin .

Editor information

Editors and Affiliations

  1. School of Computer Science, De Montfort University, Leicester, United Kingdom

    Liming Chen

  2. Organization, The Science and Information (SAI), Bradford, West Yorkshire, United Kingdom

    Supriya Kapoor

  3. Organization, The Science and Information (SAI), Bradford, West Yorkshire, United Kingdom

    Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Hasim, S.H., Mamat, R., Sheikh, U.U., Amin, S.H.M. (2016). Depth and Thermal Image Fusion for Human Detection with Occlusion Handling Under Poor Illumination from Mobile Robot. In: Chen, L., Kapoor, S., Bhatia, R. (eds) Emerging Trends and Advanced Technologies for Computational Intelligence. Studies in Computational Intelligence, vol 647. Springer, Cham. https://doi.org/10.1007/978-3-319-33353-3_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-33353-3_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-33351-9

  • Online ISBN: 978-3-319-33353-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation