Abstract
In the field of security, baggage-screening with X-rays is used as nondestructive testing for threat object detection. This is a common protocol when inspecting passenger baggage particularly at airports. Unfortunately, the accuracy of such human inspection is around 80–90%, under optimal operator conditions. For this reason, it is quite necessary to assist human inspectors with the aid of computer vision algorithms. This work proposes a deep learning-based methodology designed to detect threat objects in (single spectrum) X-ray baggage scan images. For this purpose, our proposed framework simulates a large number of X-ray images, using a combination of PGGAN (Karras et al. in International conference on learning representations, 2018. https://openreview.net/forum?id=Hk99zCeAb) and superimposition (Mery and Katsaggelos in 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), 2017.https://doi.org/10.1109/CVPRW.2017.37) strategies, that are used to train state-of-the-art detection models such as YOLO (Redmon et al. in You only look once: unified, real-time object detection. CoRR abs/1506.02640, 2015. http://arxiv.org/ abs/1506.02640), SSD (Liu et al. in SSD: single shot multibox detector. CoRR abs/1512.02325, 2015. http://arxiv. org/abs/1512.02325) and RetinaNet (Lin et al. in Focal loss for dense object detection. CoRR abs/1708.02002, 2017. http://arxiv.org/abs/1708.02002). Our method has been tested on real X-ray images in the detection of four categories of threat objects: guns, knives, razor blades and shuriken (ninja stars). In our experiments, YOLOv3 (Redmon and Farhadi in Yolov3: An incremental improvement. CoRR abs/1804.02767, 2018. http://arxiv.org/abs/1804.02767) obtained the best mean average precision (mAP) with 96.3% for guns, 76.2% for knives, 86.9% for razor blades and 93.7% for shuriken, while the average mAP for all threat objects was 80.0%. We believe the effectiveness of our method in the detection of threat objects makes its use in checkpoints possible. Moreover, our methodology is scalable and can be easily extended to detect other categories automatically.
Similar content being viewed by others
Notes
Razor blades and shuriken are not present in Subset 2.
Code and datasets can be downloaded from https://github.com/dlsaavedra/Detector_GDXray.
References
Akcay S, Kundegorski ME, Willcocks CG, Breckon TP (2018) Using deep convolutional neural network architectures for object classification and detection within X-ray baggage security imagery. IEEE Trans Inf Forensics Secur 13(9):2203–2215. https://doi.org/10.1109/TIFS.2018.2812196
Alcorn MA, Li Q, Gong Z, Wang C, Mai L, Ku WS, Nguyen A (2019) Strike (with) a pose: neural networks are easily fooled by strange poses of familiar objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4845–4854)
Anh HN (2018) keras-yolo2. https://github.com/experiencor/keras-yolo2
Anh HN (2018) keras-yolo3. https://github.com/experiencor/keras-yolo3
Baştan M (2015) Multi-view object detection in dual-energy X-ray images. Mach Vis Appl 26(7):1045–1060. https://doi.org/10.1007/s00138-015-0706-x
Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: Leonardis A, Bischof H, Pinz A (eds) Computer vision—ECCV 2006. Springer, Berlin, pp 404–417
Bolfing A, Halbherr T, Schwaninger A (2008) How image based factors and human factors contribute to threat detection performance in X-ray aviation security screening. In: Holzinger A (ed) HCI and usability for education and work. Springer, Berlin, pp 419–438
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: CVPR09
Dhiraj Jain DK (2019) An evaluation of deep learning based object detection strategies for threat object detection in baggage security imagery. Pattern Recognit Lett 120:112–119. https://doi.org/10.1016/j.patrec.2019.01.014
Everingham M, Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338. https://doi.org/10.1007/s11263-009-0275-4
Ferrari P (2018) ssd keras. https://github.com/pierluigiferrari/ssd_keras
Fizyr: keras-retinanet. https://github.com/fizyr/keras-retinanet (2018)
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. arXiv e-prints arXiv:1406.2661
Goodfellow I, Bengio Y, Courville A (2016) Deep learning, vol 1. MIT Press, Cambridge, p 2
Harris C, Stephens M (1988) A combined corner and edge detector. In: In Proc. of fourth Alvey vision conference, pp. 147–151
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge
He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR abs/1512.03385. http://arxiv.org/abs/1512.03385
Kanazawa A (2014) Locally scale-invariant convolutional neural networks. In: NeurIPS workshops
Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of GANs for improved quality, stability, and variation. In: international conference on learning representations. https://openreview.net/forum?id=Hk99zCeAb
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates Inc., New York, pp 1097–1105
Lin CC, Shiou FJ (2018) Object recognition based on foreground detection using X-ray imaging. J Chin Inst Eng 41(5):395–402. https://doi.org/10.1080/02533839.2018.1482235.
Lin T, Dollár P, Girshick RB, He K, Hariharan B, Belongie SJ (2016) Feature pyramid networks for object detection. CoRR abs/1612.03144. http://arxiv.org/abs/1612.03144
Lin T, Goyal P, Girshick RB, He K, Dollár P (2017) Focal loss for dense object detection. CoRR abs/1708.02002. http://arxiv.org/abs/1708.02002
Lin T, Maire M, Belongie SJ, Bourdev LD, Girshick RB, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014). http://arxiv.org/abs/1405.0312
Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C, Berg AC (2015) SSD: single shot multibox detector. CoRR abs/1512.02325. http://arxiv.org/abs/1512.02325
Mathanker S (2013) X-ray applications in food and agriculture: a review. Trans ASABE (American Society of Agricultural and Biological Engineers) 56:1227–1239. https://doi.org/10.13031/trans.56.9785
McBee MP, Awan OA, Colucci AT, Ghobadi CW, Kadom N, Kansagra AP, Tridandapani S, Auffermann WF (2018) Deep learning in radiology. Acad. Radiol 25(11):1472–1480
Mery D (2015) Computer vision for X-ray testing. Springer, Berlin
Mery D (2015) Inspection of complex objects using multiple-X-ray views. IEEE/ASME Trans Mechatron 20(1):338–347. https://doi.org/10.1109/TMECH.2014.2311032
Mery D, Katsaggelos AK (2017) A logarithmic X-ray imaging model for baggage inspection: Simulation and object detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 251–259 . 10.1109/CVPRW.2017.37
Mery D, Riffo V, Zscherpel U, Mondragón G, Lillo I, Zuccar I, Lobel H, Carrasco M (2015) Gdxray: the database of X-ray images for nondestructive testing. J Nondestruct Eval 34:42. https://doi.org/10.1007/s10921-015-0315-7
Mery D, Riffo V, Zuccar I, Pieringer C (2013) Automated X-ray object recognition using an efficient search algorithm in multiple views. In: 2013 IEEE conference on computer vision and pattern recognition workshops, pp. 368–374 . 10.1109/CVPRW.2013.62
Mery D, Svec E, Arias M, Riffo V, Saavedra JM, Banerjee S (2017) Modern computer vision techniques for X-ray testing in baggage inspection. IEEE Trans Syst Man Cybern Syst 47(4):682–692. https://doi.org/10.1109/TSMC.2016.2628381
Miao C, Xie L, Wan F, Su C, Liu H, Jiao J, Ye Q (2019) Sixray : A large-scale security inspection X-ray benchmark for prohibited item discovery in overlapping images. CoRR abs/1901.00303. http://arxiv.org/abs/1901.00303
Michel S, Koller SM, de Ruiter JC, Moerland R, Hogervorst M, Schwaninger A (2007) Computer-based training increases efficiency in X-ray image interpretation by aviation security screeners. In: 2007 41st Annual IEEE international Carnahan conference on security technology, pp. 201–206. 10.1109/CCST.2007.4373490
Mikhaylichenko A, Demyanenko Y, Grushko E (2016) Automatic detection of bone contours in X-ray images. In: AIST (Supplement), pp. 212–223
Mikolajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem. In 2018 International interdisciplinary PhD workshop (IIPhDW) (pp. 117-122). IEEE
Nercessian S, Panetta K, Agaian S (2008) Automatic detection of potential threat objects in X-ray luggage scan images. In: 2008 IEEE conference on technologies for Homeland Security, pp. 504–509 . 10.1109/THS.2008.4534504
Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621
Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: International conference on learning representations (ICLR)
Ramani R, Vanitha S, Valarmathy S (2013) The pre-processing techniques for breast cancer detection in mammography images. Int J Image Gr Signal Process 5:47–54. https://doi.org/10.5815/ijigsp.2013.05.06
Redmon J, Divvala SK, Girshick RB, Farhadi A (2015) You only look once: unified, real-time object detection. CoRR abs/1506.02640. http://arxiv.org/abs/1506.02640
Redmon J, Farhadi A (2016) YOLO9000: better, faster, stronger. CoRR abs/1612.08242. http://arxiv.org/abs/1612.08242
Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. CoRR abs/1804.02767. http://arxiv.org/abs/1804.02767
Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 . http://arxiv.org/abs/1506.01497
RichardWebster B, Anthony SE, Scheirer WJ (2018) Psyphy: a psychophysics driven evaluation framework for visual recognition. IEEE Trans Pattern Anal Mach Intell 41(9):2280–2286
Riffo V, Mery D (2016) Automated detection of threat objects using adapted implicit shape model. IEEE Trans Syst Man Cybern Syst 46(4):472–482. https://doi.org/10.1109/TSMC.2015.2439233
Ruder S (2016) An overview of gradient descent optimization algorithms. CoRR abs/1609.04747 . http://arxiv.org/abs/1609.04747
Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):60
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 . http://arxiv.org/abs/1409.1556
Steitz JO, Saeedan F, Roth S (2018) Multi-view X-ray R-CNN. In: Proceedings of the German conference on pattern recognition (GCPR), LNCS 11269, pp. 153–158
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9
Szeliski R (2010) Computer vision: algorithms and applications. Springer, Berlin
Turcsany D, Mouton A, Breckon TP (2013) Improving feature-based object recognition for X-ray baggage security screening using primed visualwords. In: 2013 IEEE International conference on industrial technology (ICIT), pp. 1140–1145. IEEE
Wu N, Phang J, Park J, Shen Y, Huang Z, Zorin M, Jastrzebski S, Fevry T, Katsnelson J, Kim E, Wolfson S, Parikh U, Gaddam S, Lin LLY, Ho K, Weinstein JD, Reig B, Gao Y, Toth H, Pysarenko K, Lewin A, Lee J, Airola, K, Mema E, Chung S, Hwang E, Samreen N, Kim SG, Heacock L, Moy L, Cho K, Geras KJ (2019) Deep neural networks improve radiologists’ performance in breast cancer screening. arXiv preprint arXiv:1903.08297
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? CoRR abs/1411.1792. http://arxiv.org/abs/1411.1792
Zentai G (2008) X-ray imaging for homeland security. In: 2008 IEEE international workshop on imaging systems and techniques, pp. 1–6 . 10.1109/IST.2008.4659929
Zhao Z, Zheng P, Xu S, Wu X (2018) Object detection with deep learning: a review. CoRR abs/1807.05511. http://arxiv.org/abs/1807.05511
Zou L, Yusuke T, Hitoshi I (2020) Dangerous objects detection of X-ray images using convolution neural network. In: Yang CN, Peng SL, Jain LC (eds) Security with intelligent computing and big-data services. Springer, Cham, pp 714–728
Funding
This work was supported in part by Fondecyt Grants 1161314 and 1191131 from CONICYT—Chile.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Saavedra, D., Banerjee, S. & Mery, D. Detection of threat objects in baggage inspection with X-ray images using deep learning. Neural Comput & Applic 33, 7803–7819 (2021). https://doi.org/10.1007/s00521-020-05521-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05521-2