Advertisement

Signal, Image and Video Processing

, Volume 12, Issue 6, pp 1173–1180 | Cite as

Image de-fencing using histograms of oriented gradients

  • Madiha Khalid
  • Muhammad Murtaza Yousaf
  • Kashif Murtaza
  • Syed Mansoor Sarwar
Original Paper
  • 174 Downloads

Abstract

Image de-fencing is often used by digital photographers to remove regular or near-regular fence-like patterns from an image. The goal of image de-fencing is to remove a fence object from an image in such a seamless way that it appears as if the fence never existed in the image. This task is mainly challenging due to a wide range intra-class variation of fence, complexity of background, and common occlusions. We present a novel image de-fencing technique to automatically detect fences of regular and irregular patterns in an image. We use a data-driven approach that detects a fence using encoded images as feature descriptors. We use a variant of the histograms of oriented gradients (HOG) descriptor for feature representation. We modify the conventional HOG descriptor to represent each pixel rather than representing a full patch. We evaluated our algorithm on 41 different images obtained from various sources on the Internet based on a well-defined selection criteria. Our evaluation shows that the proposed algorithm is capable of detecting a fence object in a given image with more than 98% accuracy and 87% precision.

Keywords

Image de-fencing Histograms of oriented gradients Supervised learning and classification Object detection Image inpainting Fence removal 

Supplementary material

11760_2018_1266_MOESM1_ESM.jpg (55 kb)
Supplementary material 1 (jpg 54 KB)
11760_2018_1266_MOESM2_ESM.jpg (121 kb)
Supplementary material 2 (jpg 120 KB)

References

  1. 1.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)Google Scholar
  2. 2.
    Liu, Y., Belkina, T., Hays, J.H., Lublinerman, R.: Image de-fencing. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)Google Scholar
  3. 3.
    Hays, J., Leordeanu, M., Efros, A.A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 522–535 (2006)Google Scholar
  4. 4.
    Park, M., Brocklehurst, K., Collins, R.T., Liu, Y.: Image de-fencing revisited. In: Proceedings of Asian Conference on Computer Vision (ACCV), pp. 422–434 (2011)Google Scholar
  5. 5.
    Farid, M.S., Mahmood, A., Grangetto, M.: Image de-fencing framework with hybrid inpainting algorithm. Signal Image Video Process. 10(7), 1193–1201 (2016)CrossRefGoogle Scholar
  6. 6.
    Qin Zou, Y., Cao, Q.L., Mao, Q., Wang, S.: Automatic inpainting by removing fence-like structures in rgbd images. Mach. Vis. Appl. 25(7), 1841–1858 (2014)CrossRefGoogle Scholar
  7. 7.
    Kumar, V., Mukherjee, J., Mandal, S.K.D.: Image defencing via signal demixing. In: Proceedings of 10th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 1–8 (2016)Google Scholar
  8. 8.
    Khalid, M., Yousaf, M.M.: Parallel image de-fencing: technique, analysis and performance evaluation. In: Advanced Computer and Communication Engineering Technology, pp. 979–988. Springer (2016)Google Scholar
  9. 9.
    Zhang, Q., Yuan, Y., Lu, X.: Image de-fencing with hyperspectral camera. In Proceedings of the International Conference on Computer, Information and Telecommunication Systems (CITS), pp. 1–5 (2016)Google Scholar
  10. 10.
    Jonna, S., Satapathy, S., Sahay, R.R.: Stereo image de-fencing using smartphones. In: Proceedings of the International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1792–1796 (2017)Google Scholar
  11. 11.
    Xue, T., Rubinstein, M., Liu, C., Freeman, W.T.: A computational approach for obstruction-free photography. ACM Trans. Graph. 34(4), 1–11 (2015)CrossRefGoogle Scholar
  12. 12.
    Liu, W., Mu, Y., Yan, S.: Video de-fencing. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1111–1121 (2014)CrossRefGoogle Scholar
  13. 13.
    Khasare, V.S., Sahay, R.R., Kankanhalli, M.S.: Seeing through the fence: image de-fencing using a video sequence. In: IEEE Proceedings of International Conference on Image Processing (ICIP), pp. 1351–1355 (2013)Google Scholar
  14. 14.
    Negi, C.S., Mandal, K., Sahay, R.R., Kankanhalli, M.S.: Super-resolution de-fencing: simultaneous fence removal and high-resolution image recovery using videos. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2014)Google Scholar
  15. 15.
    Yi, R., Wang, J., Tan, P.: Automatic fence segmentation in videos of dynamic scenes. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 705–713 (2016)Google Scholar
  16. 16.
    Jonna, S., Nakka, K.K., Sahay, R.R.: My camera can see through fences: a deep learning approach for image de-fencing. In: Asian Conference on Pattern Recognition (ACPR), pp. 261–265 (2015)Google Scholar
  17. 17.
    Jonna, S., Nakka, K.K., Sahay, R.R.: Deep learning based fence segmentation and removal from an image using a video sequence. In: Computer Vision—ECCV Workshops, pp. 836–851. Springer, 2016Google Scholar
  18. 18.
    Jonna, S., Voleti, V.S., Sahay, R.R., Kankanhalli, M.S.: A multimodal approach for image de-fencing and depth inpainting. In: International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6 (2015)Google Scholar
  19. 19.
    Jonna, S., Nakka, K.K., Khasare, V.S., Sahay, R.R., Kankanhalli, M.S.: Detection and removal of fence occlusions in an image using a video of the static/dynamic scene. J. Opt. Soc. Am. 33(10), 1917–1930 (2016)CrossRefGoogle Scholar
  20. 20.
    Yamashita, A., Matsui, A., Kaneko, T.: Fence removal from multi-focus images. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 4532–4535 (2010)Google Scholar
  21. 21.
    Li, Y., Wang, Y., Piao, Y.: Extraction of thin occlusions from digital images. In: Proceedings of SPIE 10255, Selected Papers of the Chinese Society for Optical Engineering Conferences (2017)Google Scholar
  22. 22.
    Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  23. 23.
    Mirkamali, S.S., Nagabhushan, P.: Object removal by depth-wise image inpainting. Signal Image Video Process. 9(8), 1785–1794 (2015)CrossRefGoogle Scholar
  24. 24.
    Lee, J., Lee, D.K., Park, R.H.: Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans. Consum. Electron. 58(2), 553–561 (2012)CrossRefGoogle Scholar
  25. 25.
    Yamashita, A., Tsurumi, F., Kaneko, T., Asama, H.: Automatic removal of foreground occluder from multi-focus images. In: IEEE International Conference on Robotics and Automation, pp. 5410–5416 (2012)Google Scholar
  26. 26.
    Kumar, V., Mukherjee, J., Mandal, S.K.D.: Combinatorial exemplar-based image inpainting. In: International Workshop on Combinatorial Image Analysis, pp. 284–298 (2015)Google Scholar
  27. 27.
    Yamauchi, H., Haber, J., Seidel, H.P.: Image restoration using multiresolution texture synthesis and image inpainting. In: Proceedings of Computer Graphics International, pp. 120–125 (2003)Google Scholar
  28. 28.
    Wei, Y., Liu, S.: Domain-based structure-aware image inpainting. Signal Image Video Process. 10(5), 911–919 (2016)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Farid, M.S., Khan, H., Mahmood, A.: Image inpainting based on pyramids. In: Proceedings of IEEE International Conference on Signal Processing (ICSP), pp. 711–715 (2010)Google Scholar
  30. 30.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 886–893 (2005)Google Scholar
  31. 31.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2), 91–110 (2004)CrossRefGoogle Scholar
  32. 32.
    Belongie, S., Malik, J., Puzicha, J.: Matching shapes. In: The 8th ICCV, pp. 454–461 (2001)Google Scholar
  33. 33.
    Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. Technical report, International Workshop on Automatic Face and Gesture Recognition. IEEE Computer Society (1995)Google Scholar
  34. 34.
    Serra, J.: Image Analysis and Mathematical Morphology. Academic Press Inc., Orlando (1983)Google Scholar
  35. 35.
    Dalal, N.: Finding people in images and videos. Ph.D. thesis, Institut National Polytechnique de Grenoble/INRIA (2006)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Madiha Khalid
    • 1
  • Muhammad Murtaza Yousaf
    • 1
  • Kashif Murtaza
    • 1
  • Syed Mansoor Sarwar
    • 1
  1. 1.Punjab University College of Information TechnologyUniversity of the PunjabLahorePakistan

Personalised recommendations