Advertisement

Machine Vision and Applications

, Volume 28, Issue 3–4, pp 409–420 | Cite as

Removal of specular reflections from image sequences using feature correspondences

  • Syed. M. Z. Abbas Shah
  • Stephen Marshall
  • Paul Murray
Original Paper

Abstract

The presence of specular highlights can hide underlying features of a scene within an image and can be problematic in many application scenarios. In particular, this poses a significant challenge for applications where image stitching is used to create a single static image of a scene from inspection footage of pipes, gas tubes, train tracks and concrete structures. Furthermore, they can hide small defects in the images causing them to be missed during inspection. We present a method which exploits additional information in neighbouring frames from video footage to reduce specularity from each frame. The technique first automatically determines frames which contain overlapping regions before the relationship that exists between them is exploited in order to suppress the effects of specular reflections. This results in an image that is free from specular highlights provided there is at least one frame present in the sequence where a given pixel is present in a diffuse form. The method is shown to work well on greyscale as well as colour images and effectively reduces specularity and significantly improves the quality of the stitched image, even in the presence of noise. While applied to the challenge of reducing specularity in inspection videos, the method improves upon the state-of-the-art in specularity removal, and its applications are wide-ranging as a general purpose pre-processing tool.

Keywords

Image Projection Specular Reflection Removal Visual Inspection Non-Destructive Evaluation 

Notes

Acknowledgements

The authors would like to thank Rahul Summan and Francesco Guarato for useful discussions about NDE applications and for their guidance for data acquisition.

Supplementary material

138_2017_826_MOESM1_ESM.rar (4.6 mb)
Supplementary material 1 (rar 4686 KB)
138_2017_826_MOESM2_ESM.avi (2 mb)
Supplementary material 2 (avi 2012 KB)
138_2017_826_MOESM3_ESM.avi (2.3 mb)
Supplementary material 3 (avi 2338 KB)

References

  1. 1.
    Dobie, G., Summan, R., Macleod, C., Pierce, S.G.: NDT & E International Visual odometry and image mosaicing for NDE. NDT E Int. 57, 17–25 (2013)CrossRefGoogle Scholar
  2. 2.
    Hansen, P., Alismail, H., Browning, B., Rander, P.: Stereo visual odometry for pipe mapping. In: IEEE International Conference Intelligent Robots and System, pp. 4020–4025 (2011)Google Scholar
  3. 3.
    Zhou, F., Zou, R., Qiu, Y., Gao, H.: Automated visual inspection of angle cocks during train operation. Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit. 228, 794–806 (2014)CrossRefGoogle Scholar
  4. 4.
    Kumar, S.S., Bharatkumar, B.H., Ramesh, G., Krishnamoorthy, T.S.: Integration of NDT in rapid screening of concrete structures. In: Güneş, O., Akkaya, Y. (eds.) Nondestructive testing of materials and structures, pp. 1259–1264. Springer, New York (2013)Google Scholar
  5. 5.
    Dickson, P., Li, J., Zhu, Z., Hanson, A.R., Riseman, E.M., Sabrin, H., Schultz, H., Whitten, G.: Mosaic generation for under vehicle inspection. In: Proceedings Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002), pp. 251–256. IEEE (2002)Google Scholar
  6. 6.
    Metni, N., Hamel, T.: A UAV for bridge inspection: visual servoing control law with orientation limits. Autom. Constr. 17, 3–10 (2007)CrossRefGoogle Scholar
  7. 7.
    Jensen, A.M., Baumann, M., Chen, Y.: Low-cost multispectral aerial imaging using autonomous runway-free small flying wing vehicles. In: IEEE International Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008, pp. V–506. IEEE (2008)Google Scholar
  8. 8.
    Kim, A., Eustice, R.: Pose-graph visual SLAM with geometric model selection for autonomous underwater ship hull inspection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, pp. 1559–1565. IEEE (2009)Google Scholar
  9. 9.
    Ridao, P., Carreras, M., Ribas, D., Garcia, R.: Visual inspection of hydroelectric dams using an autonomous underwater vehicle. J. Field Robot. 27, 759–778 (2010)CrossRefGoogle Scholar
  10. 10.
    Scotti, F.: Computational intelligence techniques for reflections identification in iris biometric images. In: Proceedings 2007 IEEE International Conference on Computational Intelligence Measurement Systems and Applications CIMSA, pp. 84–88 (2007)Google Scholar
  11. 11.
    Madooei, A., Drew, M.S.: Detecting specular highlights in dermatological images. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4357–4360. IEEE (2015)Google Scholar
  12. 12.
    Alsaleh, S.M., Aviles, A.I., Sobrevilla, P., Casals, A., Hahn, J.K.: Automatic and Robust Single-Camera Specular Highlight Removal in Cardiac Images, pp. 675–678 (2015)Google Scholar
  13. 13.
    Cui, Z., Zhang, D., Wang, K., Zhang, H., Li, N., Zuo, W.: Weighted nuclear norm minimization based tongue specular reflection removal. Math. Probl. Eng. 2015, 1–15 (2015). doi: 10.1155/2015/979415 Google Scholar
  14. 14.
    Conte, D., Foggia, P., Percannella, G., Tufano, F., Vento, M.: Reflection removal for people detection in video surveillance applications. In: Image Analysis and Processing - ICIAP 2011, Pt I. 6978, pp. 178–186 (2011)Google Scholar
  15. 15.
    Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Comput. Graph. Forum 30, 2208–2230 (2011)CrossRefGoogle Scholar
  16. 16.
    Yilmaz, O., Doerschner, K.: Detection and localization of specular surfaces using image motion cues. Mach. Vis. Appl. 25, 1333–1349 (2014)CrossRefGoogle Scholar
  17. 17.
    Huo, Y., Yang, F., Li, C.: HDR image generation from LDR image with highlight removal. In: 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–5. IEEE (2015)Google Scholar
  18. 18.
    Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 178–193 (2005)CrossRefGoogle Scholar
  19. 19.
    Shen, H.-L., Cai, Q.-Y.: Simple and efficient method for specularity removal in an image. Appl. Opt. 48, 2711–9 (2009)CrossRefGoogle Scholar
  20. 20.
    Kim, H., Jin, H., Hadap, S., Kweon, I.: Specular reflection separation using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1460–1467 (2013)Google Scholar
  21. 21.
    Koirala, P., Hauta-Kasari, M., Parkkinen, J.: Highlight removal from single image. In: Advanced Concepts for Intelligent Vision Systems, pp. 176–187. Springer, Berlin (2009)Google Scholar
  22. 22.
    Yu, D., Han, J., Jin, X., Han, J.: Efficient highlight removal of metal surfaces. Signal Process. 103, 367–379 (2014)CrossRefGoogle Scholar
  23. 23.
    Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1304–1311 (2015)CrossRefGoogle Scholar
  24. 24.
    He, Y., Khanna, N., Boushey, C.J., Delp, E.J.: Specular highlight removal for image-based dietary assessment. In: Proceedings 2012 IEEE International Conference on Multimedia Expo Workshops. ICMEW 2012, pp. 424–428 (2012)Google Scholar
  25. 25.
    Liu, Y., Yuan, Z., Zheng, N., Wu, Y.: Saturation-preserving specular reflection separation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3725–3733. IEEE (2015)Google Scholar
  26. 26.
    Lin, S., Li, Y., Kang, S., Tong, X., Shum, H.: Diffuse-specular separation and depth recovery from image sequences. In: Computer Vision—ECCV 2002, pp. 210–224 (2002)Google Scholar
  27. 27.
    Feris, R., Raskar, R., Tan, K.H., Turk, M.: Specular reflection reduction with multi-flash imaging. In: Brazilian Symposium on Computer Graphics Image Processing, pp. 316–321 (2004)Google Scholar
  28. 28.
    Iwata, S., Ogata, K., Sakaino, S., Tsuji, T.: Specular reflection removal with high-speed camera for video imaging. In: Industrial Electronics Society, IECON 2015-41st Annual Conference of the IEEE, pp. 1735–1740. IEEE (2015)Google Scholar
  29. 29.
    Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends\(\textregistered \) Comput. Graph. Vis. 3, 177–280 (2008)Google Scholar
  30. 30.
    Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Computer vision–ECCV 2006, pp. 404–417. Springer, Berlin (2006)Google Scholar
  31. 31.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  32. 32.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: Lecture Notes Computer Science (including Subseries Lecture Notes Artificial Intelligence, Lecture Notes Bioinformatics), 6314 LNCS, pp. 778–792 (2010)Google Scholar
  33. 33.
    Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: Fast Retina Keypoint. In: Proceedings IEEE International Conference on Computer Vision Pattern Recognition, pp. 510–517 (2012)Google Scholar
  34. 34.
    Leutenegger, S., Chli, M., Siegwart, R.: BRISK: Binary Robust Invariance Scalable Keypoints. In:Proceedings International Conference on Computer Vision, pp. 2548–2555 (2011)Google Scholar
  35. 35.
    Lindeberg, T.: Image matching using generalized scale-space interest points. J. Math. Imaging Vis. 52, 3–36 (2015)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Khan, N.Y., McCane, B., Wyvill, G.: SIFT and SURF performance evaluation against various image deformations on benchmark dataset. In: 2011 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 501–506. IEEE (2011)Google Scholar
  37. 37.
    Juan, L., Gwun, O.: A comparison of sift, pca-sift and surf. Int. J. Image Process. 3, 143–152 (2009)Google Scholar
  38. 38.
    Kashif, M., Deserno, T.M., Haak, D., Jonas, S.: Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment. Comput. Biol. Med. 68, 67–75 (2016)CrossRefGoogle Scholar
  39. 39.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)CrossRefGoogle Scholar
  40. 40.
    Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Proceedings IEEE Computer Society Conference on Computer Vision Pattern Recognition, pp. 1000–1006 (1997)Google Scholar
  41. 41.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2003)MATHGoogle Scholar
  43. 43.
    Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11, 23–27 (1975)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Syed. M. Z. Abbas Shah
    • 1
  • Stephen Marshall
    • 2
  • Paul Murray
    • 2
  1. 1.Department of Electronics EngineeringMehran University of Engineering and TechnologyJamshoroPakistan
  2. 2.Department of Electronic and Electrical EngineeringUniversity of StrathclydeGlasgowUnited Kingdom

Personalised recommendations