Skip to main content
SpringerLink
Log in

Detection and Restoration of Image from Multi-Color Fence Occlusions

  • APPLIED PROBLEMS
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

In Image De-fencing, segmentation and restoration of occluded fence region from the images are very challenging, when the desired object is occluded with an undesired object. This undesired object may be scattered in the whole image region. Existing algorithms can only detect single colored fences at a time from the digital images. This paper presents a multi-colored fence detection algorithm. Multi-threshold segmentation technique is used to segment the fence in the image. The segmented mask is amended by using morphological operations. To restore the fence occluded area in an image hybrid inpainting technique is used. Obtained results after experimentation are compared with the start-of-art image de-fencing technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

Similar content being viewed by others

REFERENCES

  1. Y. Liu, T. Belkina, J. H. Hays, R. Lublinerman, “Image de-fencing,” in Proc. 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008) (Anchorage, AK, USA), pp. 1–8.

  2. A. Criminisi, P. Perez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process. 13 (9), 1200–1212 (2004).

    Article  Google Scholar 

  3. M. Park, K. Brocklehurst, R. T. Collins, and Y. Liu, “Image de-fencing revisited,” in Computer Vision – ACCV 2010, Proc. 10th Asian Conf. on Computer Vision, Part IV, Ed. by R. Kimmel, R. Klette, and A. Sugimoto, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2011), Vol. 6495, pp. 422–434 (2011).

  4. R. Hettiarachchi, J. Peters, and N. Bruce, “Fence-like quasi-periodic texture detection in images,” Theory Appl. Math. Comput. Sci. 4 (2), 123–139 (2014).

    MATH  Google Scholar 

  5. M. Khalid and M. M. Yousaf, “Parallel image de-fencing: Technique, analysis and performance evaluation,” in Advanced Computer and Communication Engineering Technology, Ed. by H. Sulaiman, M. Othman, et al., Lecture Notes in Electrical Engineering (Springer, Cham, 2016), Vol. 362, pp. 979–988.

  6. J. Yang, J. Wang, L. Liu, and C. Hou, “RIFO: Restoring images with fence occlusions,” in Proc. 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP) (Xiamen, China, 2015), pp. 1–6.

  7. A. Yamashita, A. Matsui, and T. Kaneko, “Fence removal from multi-focus images,” in Proc. 2010 20th Int. Conf. on Pattern Recognition (ICPR 2010) (Istanbul, Turkey, 2011), IEEE, pp. 4532–4535.

  8. A. Yamashita, F. Tsurumi, T. Kaneko, and H. Asama, “Automatic removal of foreground occluder from multi-focus images,” in Proc. 2012 IEEE Int. Conf. on Robotics and Automation (ICRA 2012) (Saint Paul, MN, USA), pp. 5410–5416.

  9. Y. Li, Y. Wang, and Y. Piao, “Extraction of thin occlusions from digital images,” in Selected Papers of the Chinese Society for Optical Engineering Conferences held October and November 2016, Ed. by Y. Lv, J. Le, H. Chen, et al., Proc. SPIE 10255, 1025540 (2017).

  10. M. S. Farid, A. Mahmood, and M. Grangetto, “Image de-fencing framework with hybrid inpainting algorithm,” Signal, Image Video Process. (SIViP) 10 (7), 1193–1201 (2016).

    Article  Google Scholar 

  11. Q. Zou, Y. Cao, Q. Li, Q. Mao, and S. Wang, “Automatic inpainting by removing fence-like structures in RGBD images,” Mach. Vision Appl. 25 (7), 1841–1858 (2014).

    Article  Google Scholar 

  12. M. Leordeanu and M. Hebert, “A spectral technique for correspondence problems using pairwise constraints,” in Proc. Tenth IEEE Int. Conf. on Computer Vision (ICCV ’05) (Beijing, China, 2005), Vol. 2, pp. 1482–1489.

  13. T. Leung and J. Malik, “Detecting, localizing and grouping repeated scene elements from an image,” in Computer Vision — ECCV ’96, Proc. 4th European Conference on Computer Vision, Vol. 1, Ed. by B. Buxton and R. Cipolla, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 1996), Vol. 1064, pp. 546–555.

  14. Y. Liu, R. Collins, and Y. Tsin, “A computational model for periodic pattern perception based on frieze and wallpaper groups,” IEEE Trans. Pattern Anal. Mach. Intell. 26 (3), 354–371 (2004).

    Article  Google Scholar 

  15. A. Lobay, and D. A. Forsyth, “Shape from texture without boundaries,” Int. J. Comput. Vision 67 (1), 71–91 (2006).

    Article  Google Scholar 

  16. T. Tuytelaars, A. Turina, and L. Van Gool, “Non-combinatorial detection of regular repetitions under perspective skew,” IEEE Trans. Pattern Anal. Mach. Intell. 25 (4), 418–432 (2003).

    Article  Google Scholar 

  17. M. S. Farid, M. Lucenteforte, and M. Grangetto, “DOST: a distributed object segmentation tool,” Multimed. Tools Appl. 77 (16), 20839–20862 (2018).

    Article  Google Scholar 

  18. M. Khalid, M. M. Yousaf, K. Murtaza, and S. M. Sarwar, “Image de-fencing using histograms of oriented gradients,” Signal, Image Video Process. (SIViP) 12 (6), 1173–1180 (2018).

    Article  Google Scholar 

  19. V. Kumar, J. Mukherjee, and S. K. D. Mandal, “Image defencing via signal demixing,” in Proc. 10th Indian Conf. on Computer Vision, Graphics and Image Processing (ICVGIP’16) (Guwahati, India, 2016), Article No. 11, pp. 1–8.

  20. M. Li., “Example-based learning using heuristic orthogonal matching pursuit teaching mechanism with auxiliary coefficient representation for the problem of de-fencing and its affiliated applications,” Appl. Intell. 48 (9), 2884–2893 (2018).

    Article  Google Scholar 

  21. Adobe Photoshop: Lasso tool. Available at http://helpx.adobe.com/photoshop/using/selecting-lasso-tools.html

  22. Mortensen, E., Barrett, W.: “Intelligent scissors for image composition,” in Proc. 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95) (Los Angeles, CA, USA, 1995) (ACM, New York, 1995), pp. 191–198.

  23. E. Mortensen, B. Morse, W. Barrett, and J. Udupa, “Adaptive boundary detection using ‘live-wire’ two dimensional dynamic programming,” in Proc. Computers in Cardiology (Durham, NC, USA, 1992), IEEE, pp. 635–638.

  24. A. Berman, A. Dadourian, and P. Vlahos, “Method for removing from an image the background surrounding a selected object,” (October 17 2000) US Patent 6134346 (2010). https://patents.google.com/patent/US6134346

  25. Y. Y. Boykov and M.-P. Jolly, “Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images,” in Proc. Eighth IEEE Int. Conf. on Computer Vision (ICCV 2001) (Vancouver, Canada, 2001), Vol. l, pp, 105–112.

  26. V. Gulshan, C. Rother, A. Criminisi, A. Blake, and A. Zisserman, “Geodesic star convexity for interactive image segmentation,” in Proc. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010) (San Francisco, CA, USA, 2010), pp. 3129–3136.

  27. C. Rother, V. Kolmogorov, and A. Blake, “GrabCut”: Interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph. 23 (3), 309–314 (2004).

    Article  Google Scholar 

  28. K. He, J. Sun, X. Tang, “Fast matting using large kernel matting Laplacian matrices,” in Proc. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010) (San Francisco, CA, USA, 2010), pp. 2165–2172.

  29. A. Levin, D. Lischinski, Y. Weiss, “A closed form solution to natural image matting,” in Proc. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006) (New York, USA, 2006), pp. 61–68.

  30. Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, “Lazy snapping,” ACM Trans. Graph. 23 (3), 303–308 (2004).

    Article  Google Scholar 

  31. M. Ashikhmin, “Synthesizing natural textures,” in Proc. 2001 Symposium on Interactive 3D Graphics (I3D’01) (Research Triangle Park, NC, USA, 2001) (ACM, New York, 2001), pp. 217–226.

  32. L. Demanet, B. Song, and T. Chan, “Image inpainting by correspondence maps: A deterministic approach,” in UCLA CAM R, Technical Report (2003).

  33. J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17 (1), 53–69 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  34. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’00) (New Orleans, LA, USA, 2000) (ACM /Addison-Wesley, New York, 2000), pp. 417–424.

  35. D. Tschumperlé, “Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE’s,” Int. J. Comput. Vision 68 (1), 65–82 (2006).

    Article  Google Scholar 

  36. J. Cai, R. Chan, and Z. Shen, “A framelet-based image inpainting algorithm,” Appl. Comput. Harmon. Anal. 24 (2), 131–149 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  37. T. F. Chan, and J. Shen, “Nontexture inpainting by curvature-driven diffusions,” J. Vis. Commun. Image Rep. 12 (4), 436–449 (2001).

    Article  Google Scholar 

  38. A. Criminisi, P. Perez, and K. Toyama, “Object removal by exemplarbased inpainting,” in Proc. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2003), (Madison, WI, USA, 2003), Vol. 2, pp. 721–728.

  39. K. Zhang, X. Gao, D. Tao, and X. Li, “Single image super-resolution with non-local means and steering kernel regression,” IEEE Trans. Image Process. 21 (11), 4544–4556 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  40. V. K. Alilou and F. Yaghmaee, “Application of GRNN neural network in non-texture image inpainting and restoration,” Pattern Recogn. Lett. 62, 24–31 (2015).

    Article  Google Scholar 

  41. M. Melgosa, “Testing CIELAB-based color-difference formulas,” Color Res. Appl. 25 (1), 49–55 (2000).

    Article  Google Scholar 

  42. B.-Y. Wong, K.-T. Shih, C.-K. Liang, and H. H. Chen, “Single image realism assessment and recoloring by color compatibility,” IEEE Trans. Multimed. 14 (3), 760–769 (2012).

    Article  Google Scholar 

  43. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd. ed. (Prentice Hall, Upper Saddle River, NJ, 2002).

    Google Scholar 

  44. M. Vralakshmamma and T. Venkateswarlu, “Image de-fencing using wavelet based inpainting technique,” Int. J. Imaging Rob. 18 (4), 49–63 (2018).

    Google Scholar 

  45. J. Serra, Image Analysis and Mathematical Morphology (Academic Press, Orlando, FL, 1983).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. S.V University College of Engineering, Tirupati, A.P., India

    M. Varalakshmamma & T. Venkateswarlu

Authors
  1. M. Varalakshmamma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Venkateswarlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to M. Varalakshmamma or T. Venkateswarlu.

Additional information

Mrs. M. Varalakshmamma received the B.Tech. and M.Tech. Degrees in E.C.E from Jawaharlal Nehru Technological University Anantapur, Ananthapuramu, India in 2011 and 2014 respectively. Currently she is pursuing PhD, Degree at S.V. University College of Engineering, Tirupati. She had few publications in international journals. Her research areas include Image inpainting and image de-fencing.

Dr. T. Venkateswarlu received the B.Tech. and M.Tech. Degrees in E.C.E from S.V. University College of Engineering Tirupati, India in 1979 and 1981 respectively. He received the PhD degree in Electrical Engineering from Indian Institute of Technology, Madras in 1990. Currently he is working in department of E.C.E, SVUCE, Tirupati. During 1986–1989 he was a QIP research Scholar at the department of Electrical Engineering, Indian Institute of Technology, Madras. He had many publications in IEEE transactions and various international reputed journals. His teaching and research interest are in the areas of digital systems, communications, and image processing and multidimensional digital filters.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Varalakshmamma, M., Venkateswarlu, T. Detection and Restoration of Image from Multi-Color Fence Occlusions. Pattern Recognit. Image Anal. 29, 546–558 (2019). https://doi.org/10.1134/S1054661819030209

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661819030209

Keywords:

Search

Navigation