Skip to main content
SpringerLink
Log in

Automatic Detection of Threat Objects in X-ray Baggage Inspection Using Mask Region-Based Convolutional Neural Network and Deformable Convolutional Network

  • X-RAY METHODS
  • Published:
Russian Journal of Nondestructive Testing Aims and scope Submit manuscript

Abstract

In X-ray baggage inspection, due to the increase in the types of dangerous goods and the composition of the baggage contents, it is not suitable to clearly display various baggage and dangerous goods on the image acquired by the X-ray inspection system. In recent years, many deep learning-based object detection and recognition technologies have appeared, and object detection algorithms are also developing at a rapid pace along with the development of deep learning technologies. Object detection models commonly use convolution networks to process input values, but the convolution network has a problem of using a common filter regardless of the size of the object. This study aims to improve the accuracy of the threat objects detection algorithm in X-ray baggage images by applying deformable convolutional networks that make filters corresponding to objects using offsets. The proposed method was applied to Mask R-CNN(mask region based convolutional neural network) using GDXray and SIXray, which are X-ray security inspection image data sets, and the experimental results show that the proposed model performance can be improved compared to the existing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

REFERENCES

  1. Zentai, G., X-ray imaging for homeland security, IEEE Int. Workshop Imag. Syst. Techniques (IST2008, (Chania, 2008). https://doi.org/10.1109/IST.2008.4659929

  2. Akcay, S., Kundegorski, M.E., Devereux, M., and Breckon, T.P., Transfer learning using convolutional neural networks for object classification within X-ray baggage security imagery, IEEE Int. Conf. Image Process. (ICIP) (Phoenix, 2016), pp. 1057–1061. https://doi.org/10.1109/ICIP.2016.7532519

  3. Schwaninger, A., Bolfing, A., Halbherr, T., Helman, S., Belyavin, A., and Hay, L., The impact of image based factors and training on threat detection performance in X-ray screening, Proc. Int. Conf. Res. Air Transp. (Fairfax, 2008), pp. 317–324. https://doi.org/10.13140/RG.2.1.1299.3526

  4. Michel, S., Koller, S.M., de Ruiter, J.C., Moerland, R., Hogervorst, M., and Schwaninger, A., Computer-based training increases efficiency in X-ray image interpretation by aviation security screeners, Proc. 41st Annu. IEEE Int. Carnahan Conf. Secur. Technol. (Ottawa, 2007), pp. 201–206. https://doi.org/10.1109/CCST.2007.4373490

  5. Turcsany, D., Mouton, A., and Breckon, T.P., Improving feature-based object recognition for X-ray baggage security screening using primed visual words, IEEE Int. Conf. Industr. Technol. (ICIT) (Capetown, 2013), pp. 1140–1145. https://doi.org/10.1109/ICIT.2013.6505833

  6. Bastan, M., Yousefi, M.R., and Breuel, T.M., Visual words on baggage X-ray images, Comput. Anal. Imag. Patterns 14th Int. Conf. (CAIP 2011) (Seville, 2011), pp. 360–368. https://doi.org/10.1007/978-3-642-23672-3_44

  7. Kundegorski, M.E., Akçay, S., Devereux, M., Mouton, A., and Breckon, T.P., On using feature descriptors as visual words for object detection within X-ray baggage security screening, Int. Conf. Imag. Crime Detection & Prevention (Madrid, 2016). https://doi.org/10.1049/ic.2016.0080

  8. Bastan, M., Byeon, W., and Breuel, T.M., Object recognition in multi-view dual energy X-ray images, Proc. BMVC (Berlin, 2013), pp. 130–131. https://doi.org/10.1007/978-3-642-32717-9_15

  9. Baştan, M., Multi-view object detection in dual-energy X-ray images, Mach. Vision Appl., 2015, vol. 26, no. 7, pp. 1045–1060. https://doi.org/10.1007/s00138-015-0706-x

    Article  Google Scholar 

  10. Mery, D., Riffo, V., Zuccar, I., and Pieringer, C., Automated X-ray object recognition using an efficient search algorithm in multiple views, Conf. Comput. Vision Pattern Recognit. Workshops (Portland, 2013), pp. 368–374. https://doi.org/10.1109/CVPRW.2013.62

  11. Mery, D., Svec, E., Arias, M., Riffo, V., Saavedra, J.M., and Banerjee, S., Modern computer vision techniques for X-ray testing in baggage inspection, IEEE Trans. Syst. Man Cyberne. Syst., 2017, vol. 47, no. 4, pp. 682–692. https://doi.org/10.1109/TSMC.2016.2628381

    Article  Google Scholar 

  12. Akcay, S., Kundegorski, M.E., Willcocks, C.G., and Breckon, T.P., Using deep convolutional neural network architectures for object classification and detection within X-ray baggage security imagery, IEEE Trans. Inf. Forensics Secur., 2018, vol. 13, no. 9, pp. 2203–2215. https://doi.org/10.1109/TIFS.2018.2812196

    Article  Google Scholar 

  13. Girshick, R., Donahue, J., Darrell, T., and Malik, J., Region-based convolutional networks for accurate object detection and segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 2016, vol. 38, no. 1, pp. 142–158. https://doi.org/10.1109/TPAMI.2015.2437384

    Article  Google Scholar 

  14. Ren, S., He, K., Girshick, R., and Sun, J., Faster R-CNN: Towards real-time object detection with region proposal networks, Proc. Adv. Neural Inf. Proces. Syst. (Montreal, 2015), pp. 91–99. https://doi.org/10.1109/TPAMI.2016.2577031

  15. Dai, J., Li, Y., He, K., and Sun, J., R-FCN:Object detection via region-based fully convolutional networks, Proc. Adv. Neural Inf. Process. Syst. (2016), pp. 379–387. https://doi.org/10.48550/arXiv.1605.06409

  16. Redmon, J. and Farhadi, A., YOLO9000: Better, faster, stronger, 2017 IEEE Conf. Comput. Vision Pattern Recognit. (CVPR) (Honolulu, 2017), pp. 7263–7271. https://doi.org/10.1109/CVPR.2017.690

  17. He, K., Zhang, X., Ren, S., and Sun, J., Deep residual learning for image recognition, Conf. Comput. Vision Pattern Recognit. (Las Vegas, 2016). https://doi.org/10.1109/CVPR.2016.90

  18. Krizhevskii, A., Sutskever, I., and Hinton, G.E., Imagenet classification with deep convolutional neural networks, Adv. Neural Inform. Proces. Syst. (2012), pp. 1097–1105. https://doi.org/10.1145/3065386

  19. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A., Inception-v4, inception-resnet and the impact of residual connections on learning, Proc. 31st AAAI Conf. Artif. Intell., (2017), pp. 4278–4284. https://doi.org/10.48550/arXiv.1602.07261

  20. Kim, J. and Ri, J., Generative adversarial networks and faster-region convolutional neural networks based object detection in X-ray baggage security imagery, OSA Continuum, 2020, vol. 3, no. 12, pp. 3604–3614. https://doi.org/10.1364/OSAC.412523

    Article  Google Scholar 

  21. Xu, M., Zhang, H., and Yang, J., Prohibited item detection in airport X- ray security images via attention mechanism based CNN, Chin. Conf. Pattern Recognit. & Comput. Vision (Guangzhou, 2018), pp. 429–439. https://doi.org/10.1007/978-3-030-03335-4_37

  22. An, J., Zhang, H., Zhu, Y., and Yang, J., Semantic segmentation for prohibited items in baggage inspection, Proc. Int. Conf. Intell. Sci. Big Data Eng. (ISCIDE) (Nanjing, 2019), pp. 495–505. https://doi.org/10.1007/978-3-030-36189-1_41

  23. Gu, B., Chen, G., and Luo, L., Automatic and robust object detection in x-ray baggage inspection using deep convolutional neural networks, IEEE Trans. Ind. Electron., 2021, vol. 68, no. 10, pp. 10248–10257. https://doi.org/10.1109/TIE.2020.3026285

    Article  Google Scholar 

  24. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y., Deformable convolutional networks, Proc. IEEE Int. Conf. Comput. Vision (2017), pp. 764–773. https://doi.org/10.1109/ICCV.2017.89

  25. Wang, J., Chen, Y., Chakraborty, R., Yu, S.X., Orthogonal convolutional neural networks, Preprint, 2019. arXiv:1911.12207. https://doi.org/10.48550/arXiv.1911.12207

  26. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A., You only look once: Unified, real-time object detection, Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) (Las Vegas, 2016), pp. J779–788. https://doi.org/10.1109/CVPR.2016.91

  27. He, K., Gkioxari, G., Dollár, P., and Girshick, R., Mask r-cnn, Proc. IEEE Int. Conf. Comput. Vision (Venice, 2017), pp. 2961–2969. https://doi.org/10.1109/ICCV.2017.322

  28. Ren, Y., Zhu, C., and Xiao, S., Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images, Remote Sens., 2018, vol. 10, no. 9, pp. 1470–1470. https://doi.org/10.3390/rs10091470

    Article  Google Scholar 

  29. Mery, D., Riffo, V., Zscherpel, U., Mondragón, G., Lillo, I., Zuccar, I., Lobel, H., and Carrasco, M., GDXray: The database of X-ray images for nondestructive testing, J. Nondestr. Eval., 2015, vol. 34, no. 4, p. 42. https://doi.org/10.1007/s10921-015-0315-7

    Article  Google Scholar 

  30. Miao, C., Xie, L., Wan, F., Su, C., Liu, H., Jiao, J., and Ye, Q., SIXray: A large-scale security inspection X-ray benchmark for prohibited item discovery in overlapping images, IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR) (Long Beach, 2019), pp. 2119–2128. https://doi.org/10.1109/CVPR.2019.00222

  31. Taimur, H., Muhammad, S., Samet, A., Salman, K., Mohammed, B., Ernesto, D., and Naoufel, W., Meta-transfer learning driven tensor-shot detector for the autonomous localization and recognition of concealed baggage threats, Sensors, 2020, vol. 20, no. 22, p. 6450. https://doi.org/10.3390/s20226450

    Article  Google Scholar 

  32. Hassan, T., Khan, S., Akcay, S., Bennamoun, M., and Werghi, N., Cascaded structure tensor framework for robust identification of heavily occluded baggage items from multi-vendor X-ray scans, IEEE Conf. Comput. Vision Pattern Recognit. (CVPR 2019) (Long Beach, 2019). https://doi.org/10.48550/arXiv.1912.04251

  33. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., and Zisserman, A., The pascal visual object classes challenge, machine learning challenges, evaluating predictive uncertainty, visual object classification and recognizing textual entailment, MLCW 2005 (Southampton, 2005), pp. 117–176. https://doi.org/10.1007/11736790_8

  34. Henderson, P. and Ferrari, V., End-to-end training of object class detectors for mean average precision, Asian Conf. Comput. Vision (Cham, 2016), pp. 198–213. https://doi.org/10.1007/978-3-319-54193-8_13

  35. Tan, P.N., Steinbach, M., and Kumar, V., Introduction to Data Mining, London: Pearson, 2005.

    Google Scholar 

  36. Kingma, D.P. and Ba, J., Adam: A method for stochastic optimization, Proc. Int. Conf. Learn. Representations (ICLR) (2015). https://doi.org/10.48550/arXiv.1412.6980

Download references

Author information

Authors and Affiliations

  1. University of Science, Department of Control Science, 999093, Pyongyang, North Korea

    Jongchol Kim, Jinmyong Ri & Hyok Jo

Authors
  1. Jongchol Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinmyong Ri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hyok Jo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jongchol Kim.

Ethics declarations

The authors declare that they have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J., Ri, J. & Jo, H. Automatic Detection of Threat Objects in X-ray Baggage Inspection Using Mask Region-Based Convolutional Neural Network and Deformable Convolutional Network. Russ J Nondestruct Test 58, 1175–1184 (2022). https://doi.org/10.1134/S1061830922600733

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1061830922600733

Keywords:

Search

Navigation