Skip to main content
SpringerLink
Log in

Handling occlusion in prohibited item detection from X-ray images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Prohibited item detection from X-ray images determines whether any prohibited items are present in baggage, and great progress has recently been made in this field with the development of deep learning. Nevertheless, the appearance of an occluded item interacts with the cover, which is different from occlusions encountered in conventional object detection. We design three mechanisms to handle this challenge on the assumption that the occluded part is still partially observed. First, we propose a scale interaction module in which the features in neighboring scales interact one or more times to enhance the model’s perception ability. Then, we design a cross-image weakly supervised semantic analysis model utilizing the coattention mechanism to perceive similar and different targets, breaking through the information bottleneck of the isolated detection of a single image. Finally, we introduce a multitask learning module to simultaneously optimize the model at the global level and pixel level. We evaluate our approach on the publicly available security inspection X-ray (SIXray) dataset, the occluded prohibited items X-ray (OPIXray) dataset, and the HIXray dataset, and the results show that our approach is competitive with other X-ray baggage inspection approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. In: Asian conference on computer vision, pp 622–637

  2. Akcay S, Kundegorski ME, Willcocks CG et al (2018) Using deep convolutional neural network architectures for object classification and detection within x-ray baggage security imagery. IEEE Trans Inf Forens Security 13(9):2203–2215

    Article  Google Scholar 

  3. Cui Y, Oztan B (2019) Automated firearms detection in cargo x-ray images using retinanet. In: Anomaly detection and imaging with X-Rays (ADIX) IV, p 109990P

  4. Fu J, Liu J, Tian H et al (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154

  5. Ghiasi G, Cui Y, Srinivas A et al (2021) Simple copy-paste is a strong data augmentation method for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2918–2928

  6. Hacohen G, Weinshall D (2019) On the power of curriculum learning in training deep networks. In: International conference on machine learning, pp 2535–2544

  7. Hassan T, Akcay S, Bennamoun M et al (2020) Cascaded structure tensor framework for robust identification of heavily occluded baggage items from x-ray scans. arXiv preprint arXiv:2004.06780

  8. He K, Zhang X, Ren S, et al (2016) Identity mappings in deep residual networks. In: Proceedings of the european conference on computer vision, pp 630–645

  9. Jain DK et al (2019) An evaluation of deep learning based object detection strategies for threat object detection in baggage security imagery. Pattern Recogn Lett 120:112–119

    Article  Google Scholar 

  10. Lin TY, Goyal P, Girshick R, et al (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  11. Liu JJ, Hou Q, Cheng MM et al (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3917–3926

  12. Liu Z, Lin Y, Cao Y, et al (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022

  13. Miao C, Xie L, Wan F, et al (2019) Sixray: A large-scale security inspection x-ray benchmark for prohibited item discovery in overlapping images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2119–2128

  14. Qiao S, Chen LC, Yuille A (2021) Detectors: Detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10213–10224

  15. Ren S, He K, Girshick R et al (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Proc Adv Neural Inf Process Syst 28:91–99

    Google Scholar 

  16. Tao R, Wei Y, Jiang X, Li H, Qin H, Wang J, Ma Y, Zhang L, Liu X (2021) Towards real-world x-ray security inspection: A high-quality benchmark and lateral inhibition module for prohibited items detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10923–10932

  17. Tian Y, Chen T, Cheng G et al (2022) Global context assisted structure-aware vehicle retrieval. IEEE Trans Intell Transp Syst 22(12):1–10

    Google Scholar 

  18. Tian Y, Cheng G, Gelernter J et al (2020) Joint temporal context exploitation and active learning for video segmentation. Pattern Recogn 100:107158

    Article  Google Scholar 

  19. Tian Y, Gelernter J, Wang X et al (2018) Lane marking detection via deep convolutional neural network. Neurocomputing 280:46–55

    Article  Google Scholar 

  20. Tian Y, Gelernter J, Wang X et al (2019) Traffic sign detection using a multi-scale recurrent attention network. IEEE Trans Intell Transp Syst 20(12):4466–4475

    Article  Google Scholar 

  21. Tian Y, Hu W, Jiang H et al (2019) Densely connected attentional pyramid residual network for human pose estimation. Neurocomputing 347:13–23

    Article  Google Scholar 

  22. Tian Y, Wang H, Wang X (2017) Object localization via evaluation multi-task learning. Neurocomputing 253:34–41

    Article  Google Scholar 

  23. Tian Y, Wang X, Wu J et al (2019) Multi-scale hierarchical residual network for dense captioning. J Artif Intell Res 64:181–196

    Article  MathSciNet  Google Scholar 

  24. Tian Y, Zhang Y, Xu H et al (2022) 3d tooth instance segmentation learning objectness and affinity in point cloud. ACM Trans Multimed Comput Commun Appl 18:202–211

    Article  Google Scholar 

  25. Tian Y, Zhang Y, Zhou D et al (2020) Triple attention network for video segmentation. Neurocomputing 417:202–211

    Article  Google Scholar 

  26. Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. In: Proceedings of the advances in neural information processing systems, pp 5998–6008

  27. Wang CY, Bochkovskiy A, Liao HYM (2021) Scaled-yolov4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13029–13038

  28. Wang X, Girshick R, Gupta A, et al. (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  29. Wei Y, Tao R, Wu Z, et al. (2020) Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module. In: Proceedings of the ACM international conference on multimedia, pp 138–146

  30. Yang J, Li C, Zhang P, et al. (2020) Focal self-attention for local-global interactions in vision transformers. In: Proceedings of the advances in neural information processing systems, pp 138–146

  31. Yu J, Lin Z, Yang J, et al. (2019) Free-form image inpainting with gated convolution. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4471–4480

Download references

Acknowledgements

The authors would like to thank AJE (www.aje.com) for its linguistic assistance during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Zhejiang Gongshang University, Hangzhou, 310018, China

    Dongsheng Liu, Yan Tian, Zhaocheng Xu & Guotang Jian

Authors
  1. Dongsheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaocheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guotang Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Tian.

Ethics declarations

Conflict of interest

All authors declare that they have no conflicts of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China under Grant 61972351 and 62111530300, in part by the Public Welfare Technology Research Project of Zhejiang Province under Grant LGF19G010002 and LGF20G010002, in part by the Science and Technology Program of Zhejiang Province (Key Research and Development Plan) under Grant 2022C01005, and in part by the Special Project for Basic Business Expenses of Zhejiang Provincial Colleges and Universities under Grant JRK22003.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Tian, Y., Xu, Z. et al. Handling occlusion in prohibited item detection from X-ray images. Neural Comput & Applic 34, 20285–20298 (2022). https://doi.org/10.1007/s00521-022-07578-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07578-7

Keywords

Search

Navigation