Skip to main content
SpringerLink
Log in

A lightweight image inpainting model for removing unwanted objects from residential real estate’s indoor scenes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To enhance the appeal of residential real estate listings and captivate online customers, clean and visually convincing indoor scenes are highly desirable. In this research, we introduce an innovative image inpainting model designed to seamlessly replace undesirable elements within images of indoor residential spaces with realistic and coherent alternatives. While Generative Adversarial Networks (GANs) have demonstrated remarkable potential for removing unwanted objects, they can be resource-intensive and face difficulties in consistently producing high-quality outcomes, particularly when unwanted objects are scattered throughout the images. To empower small- and medium-sized businesses with a competitive edge, we present a novel GAN model that is resource-efficient and requires minimal training time using arbitrary mask generation and a novel half-perceptual loss function. Our GAN model achieves compelling results in removing unwanted elements from indoor scenes, demonstrating the capability to train within a single day using a single GPU, all while minimizing the need for extensive post-processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

The datasets generated and/or analyzed during the current study are from https://unsplash.com, https://www.pexels.com, https://pixabay.com, and the LSUN dataset [23]. They are available from the corresponding author upon reasonable request.

References

  1. The total value of global real estate (2021) Savills. Retrieved March 27, 2022, from https://www.savills.com/impacts/market-trends/the-total-value-of-global-real-estate.html

  2. Zhang S, Lee D, Singh PV, Srinivasan K (2022) What makes a good image? Airbnb demand analytics leveraging interpretable image features. Manag Sci 68(8):5644–5666

    Article  Google Scholar 

  3. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial nets. In Neural Inform Process Syst (NIPS). pp 2672–2680

    Google Scholar 

  4. Gui J, Sun Z, Wen Y, Tao D, Ye J (2021) A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Trans Knowl Data Eng 35(4):3313–3332

    Article  Google Scholar 

  5. Eilertsen G, Tsirikoglou A, Lundström CF, Unger J (2021) Ensembles of GANs for synthetic training data generation. In Int Conf Learn Representations (ICLR) Workshop on Synthetic Data Generation: Quality, Privacy, Bias. https://iclr.cc/virtual/2021/3699

    Google Scholar 

  6. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In Int Conf Learn Representations (ICLR)

    Google Scholar 

  7. Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of GANs for improved quality, stability, and variation. In Int Conf Learn Representations (ICLR). pp 1–26

    Google Scholar 

  8. Jin Y, Zhang J, Li M, Tian Y, Zhu H, Fang Z (2017) Towards the automatic anime characters creation with generative adversarial networks. Arxiv abs/1708.05509

  9. Angkawinijwong S, Chootai P, Chuenphitthayavut K, Narksenee T, Siriborvornratanakul S (2023) Generative adversarial network for DOTA2 character generation. J Appl Sci Emerg Technol (JASET) 22(2):1

    Article  Google Scholar 

  10. Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 5967–5976

    Google Scholar 

  11. Zhu J, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision (ICCV). pp 2242–2251

    Google Scholar 

  12. Nazeri K, Ng E, Joseph T, Qureshi F, Ebrahimi M (2019) EdgeConnect: structure guided image inpainting using edge prediction. In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

    Google Scholar 

  13. Yu J, Lin ZL, Yang J, Shen X, Lu X, Huang TS (2018) Generative image inpainting with contextual attention. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5505–5514

  14. Yu J, Lin ZL, Yang J, Shen X, Lu X, Huang TS (2019) Free-form image inpainting with gated convolution. In IEEE/CVF International Conference on Computer Vision (ICCV), pp 4470–4479

  15. Ruzic T, Pižurica A (2015) Context-aware patch-based image inpainting using Markov random field modelling. IEEE Trans Image Process 24(1):444–456

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  16. Amrani N, Serra-Sagristà J, Peter P, Weickert J (2017) Diffusion-based inpainting for coding remote-sensing data. IEEE Geosci Remote Sens Lett 14(8):1203–1207

    Article  ADS  Google Scholar 

  17. Yan Z, Li X, Li M, Zuo W, Shan S (2018) Shift-Net: Image inpainting via deep feature rearrangement. In European Conference on Computer Vision (ECCV), pp 3–19

  18. Pathak D, Krähenbühl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2536–2544

  19. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph (TOG) 36(4):1–14

    Article  Google Scholar 

  20. Wang T, Ouyang H, Chen Q (2021) Image inpainting with external-internal learning and monochromic bottleneck. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5116–5125

  21. Liu G, Reda FA, Shih KJ, Wang TC, Tao A, Catanzaro B (2018) Image inpainting for irregular holes using partial convolutions. In European Conference on Computer Vision (ECCV), pp 85–100

  22. Zhao S, Cui J, Sheng Y, Dong Y, Liang X, Chang EI, Xu Y (2021) Large scale image completion via co-modulated generative adversarial networks. In International Conference on Learning Representations (ICLR). pp 1–25

    Google Scholar 

  23. Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J (2015) LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365 [cs.CV]

  24. Tseng CW, Lin H, Lai S (2017) General deep image completion with lightweight conditional generative adversarial networks. In British Machine Vision Conference (BMVC)

    Google Scholar 

  25. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR). pp 1–14

    Google Scholar 

  26. Clevert D, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network learning by exponential linear units (ELUs). In International Conference on Learning Representations (ICLR)

    Google Scholar 

  27. Gentile C, Warmuth MK (1998) Linear hinge loss and average margin. In Neural Information Processing Systems (NIPS), pp 225–231

  28. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57

    Article  Google Scholar 

  29. Horé A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In International Conference on Pattern Recognition (ICPR), pp 2366–2369

  30. Chong MJ, Forsyth DA (2020) Effectively unbiased fid and inception score and where to find them. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 6069–6078

  31. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision (ECCV), pp 694–711

  32. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2414–2423

  33. Suvorov R, Logacheva E, Mashikhin A, Remizova A, Ashukha A, Silvestrov A, Kong N, Goka H, Park K, Lempitsky VS (2022) Resolution-robust large mask inpainting with fourier convolutions. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp 3172–3182

  34. Mao X, Li Q, Xie H, Lau RY, Wang Z, Smolley SP (2017) Least squares generative adversarial networks. In IEEE International Conference on Computer Vision (ICCV), pp 2813–2821

  35. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 6230–6239

Download references

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Authors and Affiliations

  1. Graduate School of Applied Statistics, National Institute of Development Administration, 148 SeriThai Road, Klong-Chan, Bangkapi, Bangkok, 10240, Thailand

    Srun Sompoppokasest & Thitirat Siriborvornratanakul

Authors
  1. Srun Sompoppokasest
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thitirat Siriborvornratanakul
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to this article equally.

Corresponding author

Correspondence to Thitirat Siriborvornratanakul.

Ethics declarations

Conflicts of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sompoppokasest, S., Siriborvornratanakul, T. A lightweight image inpainting model for removing unwanted objects from residential real estate’s indoor scenes. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18605-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18605-1

Keywords

Search

Navigation