Skip to main content
SpringerLink
Log in

Efficient Object Detection and Classification of Heat Emitting Objects from Infrared Images Based on Deep Learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Object detection from infrared (IR) images recently attracted attention of researches. There are several techniques that can be performed on images in order to detect objects. Deep learning is an efficient technique among these techniques as it merges the feature extraction in the classification process. This paper presents a deep-learning-based approach that detects whether the image includes a certain object or not. In addition, it considers the scenario of object classification that has not been given attention in the literature for IR images. The importance of multi-object classification is to maintain the ability to discriminate between objects of interest and trivial or discarded objects in the IR images or image sequences of very poor contrast. The suggested deep learning model is based on Convolutional Neural Networks (CNNs). Two scenarios are included in this study. The first scenario is to detect a single object from an IR image. The second one is to detect multiple objects from IR images. Both scenarios have been studied and simulated at different Signal-to-Noise Ratios (SNR) on self-recoded as well as standard IR images. The proposed scenarios have been tested and validated by comparison with the traditional approach based on Histogram of Gradients (HoG) technique that is popularly considered for object detection. Moreover, a comparison with other state-of-the-art methods is presented. Simulation results reveal that the HoG approach may fail with IR images due to the low contrast of these images, while the proposed approach succeeds and achieves an accuracy level of 100 % in both studied scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Arya R, Agrawal RK, Singh N (2018) A novel approach for salient object detection using double-density dual-tree complex wavelet transform in conjunction with superpixel segmentation. Knowledge and Information Systems, https://doi.org/10.1007/s10115-018-1243-5.

    Google Scholar 

  2. Ashiba HI, Awadalla KH, El-Halfawy SM, Abd El-Samie FE (2011) Adaptive Least Squares Interpolation of Infrared Images. Circuits, Systems, and Signal Processing 30(3):543–551

    MathSciNet  MATH  Google Scholar 

  3. Ashiba HI, Mansour HM, Ahmed HM, El-Kordy MF, Dessouky MI, Abd El-Samie FE (2018) Enhancement of infrared images based on efficient histogram processing. Wirel Pers Commun. 99(2):619–636

    Google Scholar 

  4. H. I. Ashiba, H. M. Mansour, H. M. Ahmed, M. F. El-Kordy, M. I. Dessouky, O. Zahran, Fathi E. Abd El-Samie (2018) Enhancement of IR images using histogram processing and the Undecimated additive wavelet transform. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6545-9.

    Google Scholar 

  5. Battiato S et al. (Eds.) (2017) Object detection for crime scene evidence analysis using deep learning,” ICIAP, Part II, LNCS 10485, pp. 14–24, 2017. https://doi.org/10.1007/978-3-319-68548-92.

  6. Biswas SK, Milanfar P (2017) Linear support tensor machine with LSK channels: pedestrian detection in thermal infrared images. IEEE Trans Image Process J 26(9):4229–4241

    MathSciNet  MATH  Google Scholar 

  7. Cheng G, Zhou P, and Han J (2016) Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans Geosci Remote Sens, http://www.ieee.org/publications_standards/publications/rights/index.html.

  8. Farsaei AA, Mokhtari-Koushyar F, Seyed-Talebi SMJ, Kavehvash Z, Shabany M (2016) Improved two-dimensional millimeter-wave imaging for concealed weapon detection through partial fourier sampling. Journal of Infrared, Millimeter, and Terahertz Waves 37(3):267–280

    Google Scholar 

  9. Fendri E, Boukhriss RR, Hammami M (2017) Fusion of thermal infrared and visible spectra for robust moving object detection. Pattern Analysis and Applications 20(4):907–926

    MathSciNet  Google Scholar 

  10. Geng X and Kang B-H (Eds.) (2018) Multi-object detection based on deep learning in real classrooms. PRICAI 2018, LNAI 11013, pp. 352–359. https://doi.org/10.1007/978-3-319-97310-4_40.

    Google Scholar 

  11. Gundogdu E, Koc A, Aydın Alatan A (Sept. 2016) Object classification in infrared images using deep learning representations. In Proceedings of IEEE International Conference on Image Processing (ICIP) 2016(25-28):1066–1070

    Google Scholar 

  12. Han X, Zhong Y and Zhang L (2017) An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Remote Sensing Journal. Vol (9) ; https://doi.org/10.3390/rs9070666.

    Google Scholar 

  13. Hong G-S, Kim B-G, Hwang Y-S, Kwon K-K (2016) Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform. Multimedia Tools and Applications 75(23):15229–15245

    Google Scholar 

  14. Hong R et al. (Eds.) (2018) Research on multitask deep learning network for semantic segmentation and object detection. PCM, LNCS 11166, pp. 708–718. https://doi.org/10.1007/978-3-030-00764-5_65.

    Google Scholar 

  15. https://keras.io/ The python deep learning library

  16. https://www.tensorflow.org/ An end-to-end open source machine learning platform

  17. Hu X, Huang Y, Duan Q, Ci W, Dai J, Yang H (2018) Abnormal event detection in crowded scenes using histogram of oriented contextual gradient descriptor. EURASIP J Adv Signal Process 2018:54. https://doi.org/10.1186/s13634-018-0574-4

    Google Scholar 

  18. IEEE OTCBVS WS Series Bench, Davis J, Sharma V (2007) Background-subtraction using contour-based fusion of thermal and visible imagery. Computer Vision and Image Understanding 106(2-3):162–182

    Google Scholar 

  19. Khare M, Srivastava RK, Jeon M (2018) Shadow detection and removal for moving objects using Daubechies complex wavelet transform. Multimedia Tools and Applications 77(2):2391–2421

    Google Scholar 

  20. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Google Scholar 

  21. Lee CP (2004) Mine detection techniques using multiple sensors. M. Sc. Thesis, Electrical and Computer Engineering the University of Tennessee at Knoxville

  22. Lindeberg T (June 2013) Scale selection properties of generalized scale-space interest point detectors. Journal of Mathematical Imaging and Vision 46(2):177–210

    MathSciNet  MATH  Google Scholar 

  23. Liu D et al. (Eds.) (2017) Deep salient object detection via hierarchical network learning. ICONIP 2017, Part III, LNCS 10636, pp. 319–329. https://doi.org/10.1007/978-3-319-70090-8_33.

    Google Scholar 

  24. Liu Q, Lu X, He Z, Zhang C, Chen W-S (2017) Deep convolutional neural networks for thermal infrared object tracking. Journal of Knowledge-Based Systems 134:189–198

    Google Scholar 

  25. Liu F, Han P, Wang Y, Li X, Lu B, Shao X (2018) Super-resolution reconstruction of infrared images based on classified dictionary learning. Infrared Phys Technol 90:146–155

    Google Scholar 

  26. Nishanth K, Karthik G (2015) Identification of diabetic maculopathy stages using fundus images “, J Mol Image Dynamic, published at 28 July 2015.

  27. Pang S, del Coz JJ, Yu Z, Luaces O, Díez J (2018) Deep learning and preference learning for object tracking: a combined approach. Neural Process Lett 47:859–876. https://doi.org/10.1007/s11063-017-9720-5

    Google Scholar 

  28. Qiu M (Ed.) (2018) An object detection algorithm for deep learning based on batch normalization. SmartCom 2017, LNCS 10699, pp. 438–448. https://doi.org/10.1007/978-3-319-73830-7_43.

    Google Scholar 

  29. Ranzato MA, Huang FJ, Boureau YL, & LeCun Y (2007) Unsupervised learning of invariant feature hierarchies with applications to object recognition. In Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on (pp. 1-8). IEEE.

  30. Revathi AR, Kumar D (2017) An efficient system for anomaly detection using deep learning classifier. SIViP 11:291–299. https://doi.org/10.1007/s11760-016-0935-0

    Google Scholar 

  31. Salazar AM, de Diego IM, Conde C, Pardos EC (2016) Evaluation of keypoint descriptors applied in the pedestrian detection in low quality images. IEEE Lat Am Trans 14(3):1401–1407

    Google Scholar 

  32. Shapiro LG and Stockman GC (2001) Computer vision, Prentice Hall

  33. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  34. Suard F, Rakotomamonjy A, Bensrhair A, and Broggi A (2006) Pedestrian detection using infrared images and histograms of oriented gradients. In Proceedings of International Journal on Intelligent Vehicles Symposium, June 13-15, 2006, Tokyo, Japan, pp. 206–212.

  35. Subudhi BN, Ghosh S, Nanda PK, Ghosh A (2017) Moving object detection using spatio-temporal multilayer compound Markov Random Field and histogram thresholding based change detection. Multimedia Tools and Applications 76(11):13511–13543

    Google Scholar 

  36. Sun S et al. (eds.) Deep learning and machine learning for object detection in remote sensing images. Signal and Information Processing, Networking and Computers, Lecture Notes in Electrical Engineering 473, https://doi.org/10.1007/978-981-10-7521-6_30.

    Google Scholar 

  37. Villamizar M, Andrade-Cetto J, Sanfeliu A, Moreno-Noguer F (2018) Boosted random ferns for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(2):272–288

    Google Scholar 

  38. Wang M-S and Zhang Z-R (2018) FPGA implementation of HOG based multi-scale pedestrian detection. In of Proceedings of IEEE International Conference on Applied System Innovation

  39. Wang Y, Luo X, Ding L, Wu J (2018) Object tracking via dense SIFT features and low-rank representation. Soft Comput. https://doi.org/10.1007/s00500-018-3571-5.

    Google Scholar 

  40. Zhang L, Zhang Y (2017) Airport detection and aircraft recognition based on two-layer saliency model in high spatial resolution remote-sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10(4):1511–1524

    MathSciNet  Google Scholar 

  41. Zhang H, Luo C, Wang Q, Kitchin M, Parmley A, Monge-Alvarez J, Casaseca-de-la-Higuera P (2018) A novel infrared video surveillance system using deep learning based techniques. Multimed Tools Appl 77:26657–26676

    Google Scholar 

  42. Zhao Y, Chen Q, Sui X, Guohua G (2015) A novel infrared image super-resolution method based on sparse representation. Infrared Phys Technol 71:506–513

    Google Scholar 

Download references

Acknowledgements

This research was funded by the Deanship of Scientific Research at Princess Nourah bint Abdulrahman University through Fast-track Research Funding Program.

Author information

Authors and Affiliations

  1. Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia

    Abeer D. Algarni

Authors
  1. Abeer D. Algarni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abeer D. Algarni.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Algarni, A.D. Efficient Object Detection and Classification of Heat Emitting Objects from Infrared Images Based on Deep Learning. Multimed Tools Appl 79, 13403–13426 (2020). https://doi.org/10.1007/s11042-020-08616-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08616-z

Keywords

Search

Navigation