Skip to main content
SpringerLink
Log in

Removal of specular reflections from image sequences using feature correspondences

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The presence of specular highlights can hide underlying features of a scene within an image and can be problematic in many application scenarios. In particular, this poses a significant challenge for applications where image stitching is used to create a single static image of a scene from inspection footage of pipes, gas tubes, train tracks and concrete structures. Furthermore, they can hide small defects in the images causing them to be missed during inspection. We present a method which exploits additional information in neighbouring frames from video footage to reduce specularity from each frame. The technique first automatically determines frames which contain overlapping regions before the relationship that exists between them is exploited in order to suppress the effects of specular reflections. This results in an image that is free from specular highlights provided there is at least one frame present in the sequence where a given pixel is present in a diffuse form. The method is shown to work well on greyscale as well as colour images and effectively reduces specularity and significantly improves the quality of the stitched image, even in the presence of noise. While applied to the challenge of reducing specularity in inspection videos, the method improves upon the state-of-the-art in specularity removal, and its applications are wide-ranging as a general purpose pre-processing tool.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Dobie, G., Summan, R., Macleod, C., Pierce, S.G.: NDT & E International Visual odometry and image mosaicing for NDE. NDT E Int. 57, 17–25 (2013)

    Article  Google Scholar 

  2. Hansen, P., Alismail, H., Browning, B., Rander, P.: Stereo visual odometry for pipe mapping. In: IEEE International Conference Intelligent Robots and System, pp. 4020–4025 (2011)

  3. Zhou, F., Zou, R., Qiu, Y., Gao, H.: Automated visual inspection of angle cocks during train operation. Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit. 228, 794–806 (2014)

    Article  Google Scholar 

  4. Kumar, S.S., Bharatkumar, B.H., Ramesh, G., Krishnamoorthy, T.S.: Integration of NDT in rapid screening of concrete structures. In: Güneş, O., Akkaya, Y. (eds.) Nondestructive testing of materials and structures, pp. 1259–1264. Springer, New York (2013)

  5. Dickson, P., Li, J., Zhu, Z., Hanson, A.R., Riseman, E.M., Sabrin, H., Schultz, H., Whitten, G.: Mosaic generation for under vehicle inspection. In: Proceedings Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002), pp. 251–256. IEEE (2002)

  6. Metni, N., Hamel, T.: A UAV for bridge inspection: visual servoing control law with orientation limits. Autom. Constr. 17, 3–10 (2007)

    Article  Google Scholar 

  7. Jensen, A.M., Baumann, M., Chen, Y.: Low-cost multispectral aerial imaging using autonomous runway-free small flying wing vehicles. In: IEEE International Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008, pp. V–506. IEEE (2008)

  8. Kim, A., Eustice, R.: Pose-graph visual SLAM with geometric model selection for autonomous underwater ship hull inspection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, pp. 1559–1565. IEEE (2009)

  9. Ridao, P., Carreras, M., Ribas, D., Garcia, R.: Visual inspection of hydroelectric dams using an autonomous underwater vehicle. J. Field Robot. 27, 759–778 (2010)

    Article  Google Scholar 

  10. Scotti, F.: Computational intelligence techniques for reflections identification in iris biometric images. In: Proceedings 2007 IEEE International Conference on Computational Intelligence Measurement Systems and Applications CIMSA, pp. 84–88 (2007)

  11. Madooei, A., Drew, M.S.: Detecting specular highlights in dermatological images. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4357–4360. IEEE (2015)

  12. Alsaleh, S.M., Aviles, A.I., Sobrevilla, P., Casals, A., Hahn, J.K.: Automatic and Robust Single-Camera Specular Highlight Removal in Cardiac Images, pp. 675–678 (2015)

  13. Cui, Z., Zhang, D., Wang, K., Zhang, H., Li, N., Zuo, W.: Weighted nuclear norm minimization based tongue specular reflection removal. Math. Probl. Eng. 2015, 1–15 (2015). doi:10.1155/2015/979415

    Google Scholar 

  14. Conte, D., Foggia, P., Percannella, G., Tufano, F., Vento, M.: Reflection removal for people detection in video surveillance applications. In: Image Analysis and Processing - ICIAP 2011, Pt I. 6978, pp. 178–186 (2011)

  15. Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Comput. Graph. Forum 30, 2208–2230 (2011)

    Article  Google Scholar 

  16. Yilmaz, O., Doerschner, K.: Detection and localization of specular surfaces using image motion cues. Mach. Vis. Appl. 25, 1333–1349 (2014)

    Article  Google Scholar 

  17. Huo, Y., Yang, F., Li, C.: HDR image generation from LDR image with highlight removal. In: 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–5. IEEE (2015)

  18. Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 178–193 (2005)

    Article  Google Scholar 

  19. Shen, H.-L., Cai, Q.-Y.: Simple and efficient method for specularity removal in an image. Appl. Opt. 48, 2711–9 (2009)

    Article  Google Scholar 

  20. Kim, H., Jin, H., Hadap, S., Kweon, I.: Specular reflection separation using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1460–1467 (2013)

  21. Koirala, P., Hauta-Kasari, M., Parkkinen, J.: Highlight removal from single image. In: Advanced Concepts for Intelligent Vision Systems, pp. 176–187. Springer, Berlin (2009)

  22. Yu, D., Han, J., Jin, X., Han, J.: Efficient highlight removal of metal surfaces. Signal Process. 103, 367–379 (2014)

    Article  Google Scholar 

  23. Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1304–1311 (2015)

    Article  Google Scholar 

  24. He, Y., Khanna, N., Boushey, C.J., Delp, E.J.: Specular highlight removal for image-based dietary assessment. In: Proceedings 2012 IEEE International Conference on Multimedia Expo Workshops. ICMEW 2012, pp. 424–428 (2012)

  25. Liu, Y., Yuan, Z., Zheng, N., Wu, Y.: Saturation-preserving specular reflection separation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3725–3733. IEEE (2015)

  26. Lin, S., Li, Y., Kang, S., Tong, X., Shum, H.: Diffuse-specular separation and depth recovery from image sequences. In: Computer Vision—ECCV 2002, pp. 210–224 (2002)

  27. Feris, R., Raskar, R., Tan, K.H., Turk, M.: Specular reflection reduction with multi-flash imaging. In: Brazilian Symposium on Computer Graphics Image Processing, pp. 316–321 (2004)

  28. Iwata, S., Ogata, K., Sakaino, S., Tsuji, T.: Specular reflection removal with high-speed camera for video imaging. In: Industrial Electronics Society, IECON 2015-41st Annual Conference of the IEEE, pp. 1735–1740. IEEE (2015)

  29. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends\(\textregistered \) Comput. Graph. Vis. 3, 177–280 (2008)

  30. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Computer vision–ECCV 2006, pp. 404–417. Springer, Berlin (2006)

  31. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  32. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: Lecture Notes Computer Science (including Subseries Lecture Notes Artificial Intelligence, Lecture Notes Bioinformatics), 6314 LNCS, pp. 778–792 (2010)

  33. Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: Fast Retina Keypoint. In: Proceedings IEEE International Conference on Computer Vision Pattern Recognition, pp. 510–517 (2012)

  34. Leutenegger, S., Chli, M., Siegwart, R.: BRISK: Binary Robust Invariance Scalable Keypoints. In:Proceedings International Conference on Computer Vision, pp. 2548–2555 (2011)

  35. Lindeberg, T.: Image matching using generalized scale-space interest points. J. Math. Imaging Vis. 52, 3–36 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  36. Khan, N.Y., McCane, B., Wyvill, G.: SIFT and SURF performance evaluation against various image deformations on benchmark dataset. In: 2011 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 501–506. IEEE (2011)

  37. Juan, L., Gwun, O.: A comparison of sift, pca-sift and surf. Int. J. Image Process. 3, 143–152 (2009)

    Google Scholar 

  38. Kashif, M., Deserno, T.M., Haak, D., Jonas, S.: Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment. Comput. Biol. Med. 68, 67–75 (2016)

    Article  Google Scholar 

  39. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)

    Article  Google Scholar 

  40. Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Proceedings IEEE Computer Society Conference on Computer Vision Pattern Recognition, pp. 1000–1006 (1997)

  41. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  42. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  43. Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11, 23–27 (1975)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Rahul Summan and Francesco Guarato for useful discussions about NDE applications and for their guidance for data acquisition.

Author information

Authors and Affiliations

  1. Department of Electronics Engineering, Mehran University of Engineering and Technology, Jamshoro, Sindh, Pakistan

    Syed. M. Z. Abbas Shah

  2. Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow, United Kingdom

    Stephen Marshall & Paul Murray

Authors
  1. Syed. M. Z. Abbas Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Marshall
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Murray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Syed. M. Z. Abbas Shah.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shah, S.M.Z.A., Marshall, S. & Murray, P. Removal of specular reflections from image sequences using feature correspondences. Machine Vision and Applications 28, 409–420 (2017). https://doi.org/10.1007/s00138-017-0826-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0826-6

Keywords

Search

Navigation