Threat Objects Detection in X-ray Images Using an Active Vision Approach

  • Vladimir Riffo
  • Sebastian Flores
  • Domingo Mery


X-ray testing for baggage inspection has been increasingly used at airports, reducing the risk of terrorist crimes and attacks. Nevertheless, this task is still being carried out by human inspectors and with limited technological support. The technology that is being used is not always effective, as it depends mainly on the position of the object of interest, occlusion, and the accumulated experience of the inspector. Due to this problem, we have developed an approach that inspects X-ray images using active vision in order to automatically detect objects that represent a threat. Our method includes three steps: detection of potential threat objects in single views based on the similarity of features and spatial distribution; estimation of the best-next-view using Q-learning; and elimination of false alarms based on multiple view constraints. We tested our algorithm on X-ray images that included handguns and razor blades. In the detection of handguns we registered good results for recall and precision (Re = 67%, Pr = 83%) along with a high performance in the detection of razor blades (Re = 82%, Pr = 100%) taking into consideration 360 inspections in each case. Our results indicate that non-destructive inspection actively using X-ray images, leads to more effective object detection in complex environments, and helps to offset certain levels of occlusion and the internal disorder of baggage.


X-ray testing Threat objects detection Active vision X-ray images Computer vision 



This work was supported in part by DIUDA Grant No. 22277 from Universidad de Atacama, and in part by Fondecyt Grant No. 1161314 from CONICYT, Chile.


  1. 1.
    Riffo, V., Mery, D.: Active X-ray testing of complex objects. Insight Non-Destr. Test. Cond. Monit. 54(1), 28–35 (2012)CrossRefGoogle Scholar
  2. 2.
    Riffo, V., Mery, D.: Automated detection of threat objects using adapted implicit shape model. IEEE Trans. Syst. Man Cybern. 46(4), 472–482 (2016)CrossRefGoogle Scholar
  3. 3.
    Leibe, B., Schiele, B.: Interleaved object categorization and segmentation. In: Proceedings of the British Machine Vision Conference, pp. 78.1–78.10. BMVA Press (2003)Google Scholar
  4. 4.
    Mery, D., Riffo, V., Zscherpel, U., Mondragón, G., Lillo, I., Zuccar, I., Lobel, H., Carrasco, M.: GDXray: the database of X-ray images for nondestructive testing. J. Nondestr. Eval. 34(4), 42 (2015)CrossRefGoogle Scholar
  5. 5.
    Newman, T., Jain, A.: A survey of automated visual inspection. Comput. Vis. Image Underst. 61(2), 231–262 (1995)CrossRefGoogle Scholar
  6. 6.
    Hardmeier, D., Hofer, F., Schwaninger, A.: The role of recurrent cbt for increasing aviation security screeners’ visual knowledge and abilities needed in X-ray screening. In: Proceedings of the 4th International Aviation Security Technology Symposium, pp. 338–342, Washington, DC (2006)Google Scholar
  7. 7.
    Schwaninger, A., Hardmeler, D., Hofer, F.: Aviation security screeners visual abilities visual knowledge measurement. IEEE Aerosp. Electron. Syst. Mag. 20(6), 29–35 (2005)Google Scholar
  8. 8.
    Wales, A., Anderson, C., Jones, K., Schwaninger, A., Horne, J.: Evaluating the two-component inspection model in a simplified luggage search task. Behav. Res. Methods 41(3), 937 (2009)CrossRefGoogle Scholar
  9. 9.
    Bolfing, A., Halbherr, T., Schwaninger, A.: How image based factors and human factors contribute to threat detection performance in X-ray aviation security screening. HCI and Usability for Education and Work. Lecture Notes in Computer Science, vol. 5298, pp. 419–438. Springer, Berlin (2008)Google Scholar
  10. 10.
    Michel, S., Koller, S.M., Schwaninger, A.: Relationship between level of detection performance and amount of recurrent computer-based training. In: Security Technology, 2008. ICCST 2008. 42nd Annual IEEE International Carnahan Conference on, pp. 299–304. IEEE (2008)Google Scholar
  11. 11.
    Schwaninger, A., Bolfing, A., Halbherr, T., Helman, S., Belyavin, A., Hay, L.: The impact of image based factors and training on threat detection performance in X-ray screening. In: Proceedings of the 3rd International Conference on Research in Air Transportation, ICRAT 2008, pp. 317–324 (2008)Google Scholar
  12. 12.
    Zentai, G.: X-ray imaging for homeland security. Imaging systems and techniques. IEEE International Workshop on IST 2008, pp. 1–6 (2008)Google Scholar
  13. 13.
    Mery, D.: Computer Vision for X-Ray Testing. Springer, New York (2015)CrossRefzbMATHGoogle Scholar
  14. 14.
    Mery, D., Svec, E., Arias, M.: Object recognition in X-ray testing using adaptive sparse representations. J. Nondestr. Eval. 35(3), 45 (2016)CrossRefGoogle Scholar
  15. 15.
    Mery, D., Svec, E., Arias, M., Riffo, V., Saavedra, J., Banerjee, S.: Modern computer vision techniques for X-ray testing in baggage inspection. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2016)Google Scholar
  16. 16.
    Uroukov, I., Speller, R.: A preliminary approach to intelligent X-ray imaging for baggage inspection at airports. Signal Process. Res. 4, 1–11 (2015)CrossRefGoogle Scholar
  17. 17.
    Baştan, M., Yousefi, M., Breuel, T.: Visual words on baggage X-ray images. In: Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 6854, pp. 360–368. Springer, Berlin (2011)Google Scholar
  18. 18.
    Turcsany, D., Mouton, A., Breckon, T.: Improving feature-based object recognition for X-ray baggage security screening using primed visualwords. In: IEEE International Conference on Industrial Technology (ICIT), 2013, pp. 1140–1145 (2013)Google Scholar
  19. 19.
    Zhang, N., Zhu, J.: A study of X-ray machine image local semantic features extraction model based on bag-of-words for airport security. Int. J. Smart Sens. Intell. Syst. 8(1), 45–64 (2015)Google Scholar
  20. 20.
    Wang, Y., Yang, X., Wu, W., Su, B., Jeon, G.: An X-ray inspection system for illegal object classification based on computer vision. Int. J. Secur. Appl. 10(10), 155–168 (2016)Google Scholar
  21. 21.
    Akşay, S., Kundegorski, M.E., Devereux, M., Breckon, T.P.: Transfer learning using convolutional neural networks for object classification within X-ray baggage security imagery. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1057–1061 (2016)Google Scholar
  22. 22.
    Mery, D.: Inspection of complex objects using multiple-X-ray views. IEEE/ASME Trans. Mech. 20(1), 338–347 (2015)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Mery, D., Riffo, V., Zuccar, I., Pieringer, C.: Automated X-ray object recognition using an efficient search algorithm in multiple views. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 368–374 (2013)Google Scholar
  24. 24.
    Franzel, T., Schmidt, U., Roth, S.: Object detection in multi-view X-ray images. In: Pattern Recognition. Lecture Notes in Computer Science, vol. 7476, pp. 144–154. Springer, Berlin (2012)Google Scholar
  25. 25.
    Mouton, A., Flitton, G.T., Bizot, S.: An evaluation of image denoising techniques applied to CT baggage screening imagery. In: IEEE International Conference on Industrial Technology (ICIT 2013). IEEE (2013)Google Scholar
  26. 26.
    Flitton, G., Breckon, T.P., Megherbi, N.: A comparison of 3D interest point descriptors with application to airport baggage object detection in complex CT imagery. Pattern Recogn. 46(9), 2420–2436 (2013)CrossRefGoogle Scholar
  27. 27.
    Megherbi, N., Han, J., Breckon, T.P., Flitton, G.T.: A comparison of classification approaches for threat detection in CT based baggage screening. In: 19th IEEE International Conference on Image Processing (ICIP), 2012, pp. 3109–3112. IEEE (2012)Google Scholar
  28. 28.
    Flitton, G., Mouton, A., Breckon, T.P.: Object classification in 3D baggage security computed tomography imagery using visual codebooks. Pattern Recogn. 48(8), 2489–2499 (2015)CrossRefGoogle Scholar
  29. 29.
    Mouton, A., Breckon, T.P.: Materials-based 3D segmentation of unknown objects from dual-energy computed tomography imagery in baggage security screening. Pattern Recogn. 48(6), 1961–1978 (2015)CrossRefGoogle Scholar
  30. 30.
    von Bastian, C., Schwaninger, A., Michel, S.: Do Multi-view X-ray Systems Improve X-ray Image Interpretation in Airport Security Screening?, vol. 52. GRIN Verlag, Munich (2010)Google Scholar
  31. 31.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  32. 32.
    Watkins, C.J.C.H.: Learning from delayed rewards. Ph.D. thesis, University of Cambridge England (1989)Google Scholar
  33. 33.
    Mery, D., Filbert, D.: Automated flaw detection in aluminum castings based on the tracking of potential defects in a radioscopic image sequence. IEEE Trans. Robot. Autom. 18(6), 890–901 (2002)CrossRefGoogle Scholar
  34. 34.
    Mery, D.: Explicit geometric model of a radioscopic imaging system. NDT E Int. 36(8), 587–599 (2003)Google Scholar
  35. 35.
    Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  36. 36.
    Vedaldi, A., Fulkerson, B.: VLfeat: an open and portable library of computer vision algorithms. In: MM ’10: Proceedings of the international conference on Multimedia, pp. 1469–1472. New York (2010)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Departamento de Ingeniería Informática y Ciencias de la ComputaciónUniversidad de AtacamaCopiapóChile
  2. 2.Department of Computer SciencePontificia Universidad Católica de ChileSantiagoChile

Personalised recommendations