Skip to main content
SpringerLink
Log in

High-Performance Detection of Concealed Forbidden Objects on Human Body with Deep Neural Networks Based on Passive Millimeter Wave and Visible Imagery

  • Published:
Journal of Infrared, Millimeter, and Terahertz Waves Aims and scope Submit manuscript

A Publisher Correction to this article was published on 22 September 2022

This article has been updated

Abstract

In this paper, a high-performance detection algorithm of concealed forbidden objects on human body is presented based on deep neural networks (DNN) and complementary advantages of passive millimeter wave imagery (PMMWI) and visible imagery (VI). With well capacity of penetrability, PMMWI can effectively reveal suspected forbidden objects concealed on human body without harm of ionizing radiation compared with conventional X-ray methods. However, due to its current limited imaging capability, the resolution of PMMWI is still unsatisfactory and easy to result in false alarms. Therefore, by complementarity of superiorities, VI is employed to overcome the deficiency of confusions. In this way, massive image samples of PMMWI and VI are simultaneously acquired and manually annotated as necessary training datasets to carry out deep learning on DNN models so as to achieve high-performance human body profile segmentation on both PMMWI and VI. Then, high-precision region registration of human body profiles is implemented between PMMWI and VI to localize and confirm high confident suspected targets and remove false alarm regions as well. According to the principle of synthetic integration and global optimization, a high performance detection algorithm system is constructed, analyzed, and assessed. A series of comprehensive experiment results demonstrate the outstanding performance of our proposed detection algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Change history

References

  1. Maqueda I G, De La Blanca N P, Molina R, et al. Fast millimeter wave threat detection algorithm[C]. European Signal Processing Conference (EUSIPCO), 2015, pp:599–603.

  2. Wang S, Ye Z, Wang Y. Real–time dangerous objects detection in millimeter wave images[C]. Tenth International Conference on Digital Image Processing (ICDIP 2018). International Society for Optics and Photonics, 2018, 10806: 108060Z, pp:1–7.

  3. Yeom S., Lee D S, Jang Y, et al. Real–time concealed–object detection and recognition with passive millimeter wave imaging[J], Optics Express, 2012, 20(9): pp. 9371—-9381,.

  4. Meng Y, Qing A, Lin C, et al. Passive millimeter wave imaging system for public security check[C]. Applied Computational Electromagnetics Society Symposium (ACES), 2017: 1–2.

  5. Appleby R, Anderton R N. Millimeter–wave and submillimeter–wave imaging for security and surveillance[J]. Proceedings of the IEEE, 2007, 95(8): 1683–1690.

  6. Yeom S, Lee D S, Son J Y, et al. Concealed object detection using passive millimeter wave imaging[C]. Universal Communication Symposium. IEEE, 2010:383–386.

  7. Tapia S L, Molina R, de la Blanca N P. Detection and localization of objects in Passive Millimeter Wave Images[C]. European Signal Processing Conference, 2016: 2101–2105.

  8. Heinz E, May T, Born D, et al. Passive 350 ghz video imaging systems for security applications[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2015, 36(10): 879–895.

  9. Accardo J, Chaudhry M A. Radiation exposure and privacy concerns surrounding full–body scanners in airports[J]. Journal of Radiation Research and Applied Sciences, 2014, 7(2): 198–200.

  10. Du K, Zhang L, Chen W, et al. Concealed objects detection based on FWT in active millimeter–wave images[C]. Seventh International Conference on Electronics and Information Engineering, 2017, 10322: 103221O. pp:1–7.

  11. Fang G, Gao X, Li C, et al. A novel active millimeter–wave imaging scheme suitable for fast personal screening[C]. International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW–THz), 2013, pp:1–2.

  12. Dong S L, Chen X G, Yu W Y, et al. Indoor Passive Millimeter–Wave Imaging for Concealed Object Detection[J]. Advanced Materials Research, 2013, 760–762.

  13. Li X, Li S, Zhao G, et al. Multi–polarized millimeter–wave imaging for concealed weapon detection[C]. IEEE International Conference on Microwave and Millimeter Wave Technology (ICMMT), 2016, 2: 892–894.

  14. Babacan S D, Luessi M, Spinoulas L, et al. Compressive passive millimeter–wave imaging[C]. IEEE International Conference on Image Processing (ICIP), 2011: 2705–2708.

  15. AlSaafin W, Villena S, Vega M, et al. Compressive sensing super resolution from multiple observations with application to passive millimeter wave images[J]. Digital Signal Processing, 2016, 50: 180–190.

  16. Kowalski M, Kastek M. Comparative studies of passive imaging in terahertz and mid–wavelength infrared ranges for object detection[J]. IEEE Transactions on Information Forensics and Security, 2016, 11(9): 2028–2035.

  17. Mosavi M R, Bisjerdi M H, Rezai–Rad G. Optimal Target–Oriented Fusion of Passive Millimeter Wave Images with Visible Images Based on Contourlet Transform[J]. Wireless Personal Communications, 2017, 95(4): 4643–4666.

  18. Kowalski M, Palka N, Piszczek M, et al. Hidden Object Detection System Based on Fusion of THz and VIS Images[J]. Acta Physica Polonica, A., 2013, 124(3):1–4.

  19. Gonzalez–Valdes B, Alvarez Y, Mantzavinos S, et al. Improving security screening: A comparison of multistatic radar configurations for human body imaging[J]. IEEE Antennas and Propagation Magazine, 2016, 58(4): 35–47.

  20. Farsaei A A, Mokhtari–Koushyar F, Seyed–Talebi S M J, et al. Improved two–dimensional millimeter–wave imaging for concealed weapon detection through partial Fourier sampling[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2016, 37(3): 267–280.

  21. Yu W, Chen X, Wu L. Segmentation of concealed objects in passive millimeter–wave images based on the Gaussian mixture model[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2015, 36(4): 400–421.

  22. Xiao Z, Lu X, Yan J, et al. Automatic detection of concealed pistols using passive millimeter wave imaging[C]. Imaging Systems and Techniques (IST), 2015 IEEE International Conference on. IEEE, 2015: 1–4.

  23. López–Tapia S, Molina R, de la Blanca N P. Deep CNNs for Object Detection using Passive Millimeter Sensors[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017: 1–11.

  24. Yang J, Wright J, Huang T S, et al. Image super–resolution via sparse representation[J]. IEEE transactions on image processing, 2010, 19(11): 2861–2873.

  25. Carreira J, Sminchisescu C. Cpmc: Automatic object segmentation using constrained parametric min cuts[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2011 (7): 1312–1328.

  26. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]. Advances in neural information processing systems. 2012: 1097–1105.

  27. Peng C, Zhang X, Yu G, et al. Large kernel matters——improve semantic segmentation by global convolutional network[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 1743–1751.

  28. Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation[C]. Proceedings of the IEEE international conference on computer vision. 2015: 1520–1528.

  29. Farabet C, Couprie C, Najman L, et al. Learning hierarchical features for scene labeling[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(8): 1915–1929.

  30. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3431–3440.

  31. Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder–Decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017 (12): 2481–2495.

  32. Zheng S, Jayasumana S, Romera–Paredes B, et al. Conditional random fields as recurrent neural networks[C]. Proceedings of the IEEE international conference on computer vision. 2015: 1529– 1537.

  33. Chen L C, Papandreou G, Kokkinos I, et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[J], Computer Science, 2014: pp. 357–361.

  34. Ronneberger O, Fischer P, Brox T. U–net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer–assisted intervention. Springer, Cham, 2015: 234–241.

  35. Ciresan D, Giusti A, Gambardella L M, et al. Deep neural networks segment neuronal membranes in electron microscopy images[C]. Advances in neural information processing systems. 2012: 2843–2851.

  36. Zhang Y, Li Y, Chen J. Detection of concealed objects in passive millimeter wave imaging based on CS theory[C]. Microwave Conference Proceedings (APMC), 2013 Asia–Pacific. IEEE, 2013: 981–983.

  37. Torralba A, Russell B C, Yuen J. Labelme: Online image annotation and applications[J]. Proceedings of the IEEE, 2010, 98(8): 1467–1484.

  38. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770–778.

  39. Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 2261–2269.

  40. Su W, Wang Z. Widening residual skipped network for semantic segmentation[J]. IET Image Processing, 2017, 11(10): 880–887.

  41. Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.

Download references

Funding

This work was supported in part by the National Nature Science Foundation of China (Grant Nos. 61731001 and U1435220).

Author information

Authors and Affiliations

  1. School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China

    Lin Guo & Shiyin Qin

Authors
  1. Lin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shiyin Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiyin Qin.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, L., Qin, S. High-Performance Detection of Concealed Forbidden Objects on Human Body with Deep Neural Networks Based on Passive Millimeter Wave and Visible Imagery. J Infrared Milli Terahz Waves 40, 314–347 (2019). https://doi.org/10.1007/s10762-018-0558-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10762-018-0558-3

Keywords

Search

Navigation