Skip to main content
SpringerLink
Log in

Multi-view object detection in dual-energy X-ray images

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Automatic inspection of X-ray scans at security checkpoints can improve the public security. X-ray images are different from photographic images. They are transparent. They contain much less texture. They may be highly cluttered. Objects may undergo in- and out-of-plane rotations. On the other hand, scale and illumination change is less of an issue. More importantly, X-ray imaging provides extra information which are usually not available in regular images: dual-energy imaging, which provides material information about the objects; and multi-view imaging, which provides multiple images of objects from different viewing angles. Such peculiarities of X-ray images should be leveraged for high-performance object recognition systems to be deployed on X-ray scanners. To this end, we first present an extensive evaluation of standard local features for object detection on a large X-ray image dataset in a structured learning framework. Then, we propose two dense sampling methods as keypoint detector for textureless objects and extend the SPIN color descriptor to utilize the material information. Finally, we propose a multi-view branch-and-bound search algorithm for multi-view object detection. Through extensive experiments on three object categories, we show that object detection performance on X-ray images improves substantially with the help of extended features and multiple views.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. An, S., Peursum, P., Liu, W., Venkatesh, S.: Efficient algorithms for subwindow search in object detection and localization. In: Computer Vision and Pattern Recognition (CVPR), pp. 264–271 (2009)

  3. Bastan, M., Byeon, W., Breuel, T.: Object recognition in multi-view dual energy X-ray images. In: British Machine Vision Conference (BMVC) (2013)

  4. Bastan, M., Yousefi, M., Breuel, T.: Visual words on baggage X-ray images. In: International Conference on Computer Analysis of Images and Patterns (CAIP), pp. 360–368 (2011)

  5. Blaschko, M., Lampert, C.: Learning to localize objects with structured output regression. In: European Conference on Computer Vision (ECCV) (2008)

  6. Chen, Z., Zheng, Y., Abidi, B., Page, D., Abidi, M.: A combinational approach to the fusion. In: CVPR Workshops, De-noising and Enhancement of Dual-Energy X-Ray Luggage Images (2005)

  7. Erhan, D., Szegedy, C., Toshev, A., Anguelov, D.: Scalable object detection using deep neural networks. In: Computer Vision and Pattern Recognition (CVPR) (2014)

  8. Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge–a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2014)

    Article  Google Scholar 

  9. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)

    Article  Google Scholar 

  10. Flitton, G., Breckon, T., Megherbi, N.: A comparison of 3D interest point descriptors with application to airport baggage object detection in complex CT imagery. Pattern Recognit. 46(9), 2420–2436 (2013)

    Article  Google Scholar 

  11. Franzel, T., Schmidt, U., Roth, S.: Object detection in multi-view X-ray images. In: DAGM, pp. 144–154. Springer, Berlin (2012)

  12. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Computer Vision and Pattern Recognition (CVPR) (2014)

  13. Joachims, T.: SVM-Struct: Support Vector Machine for Complex Outputs (2013). http://svmlight.joachims.org/svm_struct.html

  14. Joachims, T., Finley, T., Yu, C.N.J.: Cutting-plane training of structural SVMs. Mach. Learn. 77(1), 27–59 (2009)

    Article  MATH  Google Scholar 

  15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  16. Lampert, C., Blaschko, M., Hofmann, T.: Efficient subwindow search: a branch and bound framework for object localization. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2129–2142 (2009)

    Article  Google Scholar 

  17. Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)

    Article  Google Scholar 

  18. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation (2015)

  19. Mery, D., Riffo, V., Zuccar, I., Pieringer, C.: Automated X-ray object recognition using an efficient search algorithm in multiple views. In: CVPR Workshop on Perception Beyond the Visible Spectrum (PBVS) (2013)

  20. Mikolajczyk, K.: Feature Detectors and Descriptors: The State Of The Art and Beyond (2014). URL:http://kahlan.eps.surrey.ac.uk/featurespace

  21. Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)

    Article  Google Scholar 

  22. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  23. Nowozin, S., Lampert, C.H.: Structured learning and prediction in computer vision. Found. Trends Comput. Graph. Vis. 6(3–4), 185–365 (2011)

    MATH  Google Scholar 

  24. Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: Cnn features off-the-shelf: an astounding baseline for recognition. In: CVPR Workshop (2014)

  25. Rebuffel, V., Dinten, J.: Dual-energy X-ray imaging: benefits and limits. Insight-Non-Destr. Test. Cond. Monit. 49(10), 589–594 (2007)

    Article  Google Scholar 

  26. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  27. Schmidt-Hackenberg, L., Yousefi, M., Breuel, T.: Visual cortex inspired features for object detection in X-ray images. In: International Conference on Pattern Recognition (ICPR), pp. 2573–2576 (2012)

  28. Singh, S., Singh, M.: Explosives detection systems (EDS) for aviation security: a review. Signal Process. 83(1), 31–55 (2003)

    Article  MATH  Google Scholar 

  29. Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: International Conference on Machine Learning (2004)

  30. Turcsany, D., Mouton, A., Breckon, T.: Improving Feature-based Object Recognition for X-Ray Baggage Security Screening Using Primed Visual Words. In: International Conference on Industrial Technology (ICIT), pp. 1140–1145 (2013)

  31. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2008)

    Article  Google Scholar 

  32. Van De Weijer, J., Gevers, T., Smeulders, A.W.: Robust photometric invariant features from the color tensor. IEEE Trans. Image Process. 15(1), 118–127 (2006)

    Article  Google Scholar 

  33. von Bastian, C.C., Schwaninger, A., Michel, S.: Do Multi-View X-Ray Systems Improve X-Ray Image Interpretation in Airport Security Screening?, vol. 52. GRIN Publishing GmbH, Germany (2008)

Download references

Acknowledgments

The major part of this work was done when the author was a post-doctoral researcher at the Image Understanding and Pattern Recognition Group (IUPR) of Technical University of Kaiserslautern, Germany; as part of the SICURA project, which was supported by the Bundesministerium für Bildung und Forschung of Germany with ID FKZ 13N11125 (2010–2013). The X-ray data were recorded for the SICURA project in collaboration with Smiths–Heimann (http://www.smithsdetection.com) a manufacturer of X-ray machines and one of the partners in the SICURA project. We are thankful to the project partners and members of the IUPR research group.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Turgut Özal University, Ankara, Turkey

    Muhammet Baştan

Authors
  1. Muhammet Baştan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammet Baştan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baştan, M. Multi-view object detection in dual-energy X-ray images. Machine Vision and Applications 26, 1045–1060 (2015). https://doi.org/10.1007/s00138-015-0706-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0706-x

Keywords

Search

Navigation