Advertisement

Iterative Application of Autoencoders for Video Inpainting and Fingerprint Denoising

  • Le Manh Quan
  • Yong-Guk KimEmail author
Conference paper
Part of the The Springer Series on Challenges in Machine Learning book series (SSCML)

Abstract

The sparse autoencoder stacked inside deep neural network has been a powerful tool for image inpainting. We propose a new method for video inpainting as well as fingerprint denoising based on the Iterative Application of Autoencoders (IAA). Instead of using either a single sparse autoencoder or denoising autoencoder, multiple autoencoders are concatenated in an iterative manner until a desired output is acquired from the last stage. This method allows us to reduce loss via iteration and reuse a well-defined network. Results from two public challenges on video inpainting and fingerprint denoising suggest that performance is excellent and it can be a useful approach for image inpainting in general. Our codes are available online.

Notes

Acknowledgements

This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2016-0-00312) supervised by the IITP (Institute for information and communications Technology Promotion).

References

  1. 1.
    Burlin, Charles, Yoann Le Calonnec, and Louis Duperier. Deep Image Inpainting. (2017).Google Scholar
  2. 2.
    Demir, Ugur, and Gozde Unal.: Patch-Based Image Inpainting with Generative Adversarial Networks. arXiv preprint arXiv:1803.07422 (2018).Google Scholar
  3. 3.
    Dong, Chao, Chen Change Loy, Kaiming He, and Xiaoou Tang : Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, no. 2, pp. 295–307, (2016).Google Scholar
  4. 4.
    Drori, Iddo, Daniel Cohen-Or, and Hezy Yeshurun.: Fragment-based image completion. In ACM Transactions on graphics (TOG), vol. 22, no. 3, pp. 303–312. ACM, (2003).Google Scholar
  5. 5.
    Efros, Alexei A., and William T. Freeman.: Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341–346. ACM, (2001).Google Scholar
  6. 6.
    Girshick, Ross, Jeff Donahue, Trevor Darrell, and Jitendra Malik.: Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587. (2014).Google Scholar
  7. 7.
    Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.: Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680. (2014).Google Scholar
  8. 8.
    Hays, James, and Alexei A. Efros.: Scene completion using millions of photographs. In ACM Transactions on Graphics (TOG), vol. 26, no. 3, p. 4. ACM, (2007).Google Scholar
  9. 9.
    He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.: Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778. (2016).Google Scholar
  10. 10.
    Igehy, Homan, and Lucas Pereira.: Image replacement through texture synthesis. In Image Processing, 1997. Proceedings., International Conference on, vol. 3, pp. 186–189. IEEE, (1997).Google Scholar
  11. 11.
    Iizuka, Satoshi, Edgar Simo-Serra, and Hiroshi Ishikawa.: Globally and locally consistent image completion. ACM Transactions on Graphics (TOG) 36, no. 4,107, (2014).Google Scholar
  12. 12.
    Jain, Viren, and Sebastian Seung.:Natural image denoising with convolutional networks. In Advances in Neural Information Processing Systems, pp. 769–776. (2009)Google Scholar
  13. 13.
    Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton.: Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105. (2012).Google Scholar
  14. 14.
    Kim, Jiwon, Jung Kwon Lee, and Kyoung Mu Lee.: Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654. (2016).Google Scholar
  15. 15.
    Ledig, Christian, Lucas Theis, Ferenc Huszr, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P. Aitken et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In CVPR, vol. 2, no. 3, p. 4. (2017).Google Scholar
  16. 16.
    Mao, Xiao-Jiao, Chunhua Shen, and Yu-Bin Yang.: Image restoration using convolutional auto-encoders with symmetric skip connections. arXiv preprint arXiv:1606.08921 (2016).Google Scholar
  17. 17.
    Pathak, Deepak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros.: Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544. (2016).Google Scholar
  18. 18.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z: Rethinking the inception architecture for computer vision. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826 (2016).Google Scholar
  19. 19.
    Xie, Junyuan, Linli Xu, and Enhong Chen.: Image denoising and inpainting with deep neural networks. In Advances in neural information processing systems, pp. 341–349. (2012).Google Scholar
  20. 20.
    Xu, Li, Jimmy SJ Ren, Ce Liu, and Jiaya Jia.: Deep convolutional neural network for image deconvolution.In Advances in Neural Information Processing Systems, pp. 1790–1798. (2014).Google Scholar
  21. 21.
    Yang, Chao, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li.: High-resolution image inpainting using multi-scale neural patch synthesis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, no. 2, p. 3. (2017).Google Scholar
  22. 22.
    Yeh, Raymond A., Chen Chen, Teck-Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnson, and Minh N. Do.: Semantic Image Inpainting with Deep Generative Models. In CVPR, vol. 2, no. 3, p. 4. (2017).Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer EngineeringSejong UniversitySeoulSouth Korea

Personalised recommendations