Skip to main content
SpringerLink
Log in

Image manipulation localization using reconstruction attention

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the development of image manipulation techniques and the widespread use of image editing tools, it is effortless to forge images without leaving an apparent manual trail. Existing methods achieve manipulation localization by detecting the anomalies in images. However, we argue that these methods mainly focus on tampering traces, and the learned features are representations of specific types of tampering in the training dataset, which are less effective in detecting tampering across diverse datasets. In this paper, we propose a novel Network using image Reconstruction and Reconstruction Attention (RRA-Net) that incorporates the anomaly detection idea for image tampering localization. By reconstructing the pristine part of images, we can learn the normal latent representation and mine the essential dissimilarities between the real and tampered regions. Furthermore, we use the reconstructed attention to guide the model in characterizing the tampered regions, thus facilitating manipulation localization. We conducted a series of experiments, and the results demonstrate that our RRA-Net outperforms the State-of-the-art (SOTA) methods and achieves good robustness against different post-processing attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data that supports the findings of this research are available in this article.

References

  1. Enfield S (2022) How Many Photos Will be Taken in 2022? https://news.mylio.com/how-many-photos-taken-in-2022/. Accessed 21 Feb 2023

  2. Sutcu Y, Coskun B, Sencar HT, Memon N (2007) Tamper detection based on regularity of wavelet transform coefficients. In: 2007 IEEE International Conference on Image Processing. IEEE Computer Society, San Antonio, TX, pp 397–1400

  3. Niu Y, Tondi B, Zhao Y, Ni R, Barni M (2021) Image splicing detection, localization and attribution via JPEG primary quantization matrix estimation and clustering. IEEE Trans Inf Forensics Secur 16:5397–5412

    Article  Google Scholar 

  4. Mahdian B, Saic S (2009) Using noise inconsistencies for blind image forensics. Image Vis Comput 27:1497–1503

    Article  Google Scholar 

  5. Verma V, Singh D, Khanna N (2024) Block-level double JPEG compression detection for image forgery localization. Multimed Tools Appl 83: 9949–9971

  6. Iseed SY, Mahmoud KW (2023) Forensic approach for distinguishing between source and destination regions in copy-move forgery. Multimed Tools Appl 82:31181–31210

    Article  Google Scholar 

  7. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, San Antonio, TX, pp 1053–1061

  8. Bappy JH, Simons C, Nataraj L, Manjunath BS, Roy-Chowdhury AK (2019) Hybrid LSTM and encoder–decoder architecture for detection of image forgeries. IEEE Trans Image Process 28:3286–3300

    Article  MathSciNet  Google Scholar 

  9. Wu Y, AbdAlmageed W, Natarajan P (2019) ManTra-Net: manipulation tracing net-work for detection and localization of image forgeries with anomalous features. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Long Beach, CA, USA, pp 9535–9544

  10. Hu X, Zhang Z, Jiang Z, Chaudhuri S, Yang Z, Nevatia R (2020) SPAN: spatial pyramid attention network for image manipulation localization. In: Computer Vision – ECCV 2020. Springer, Glasgow, UK, pp 312–328

  11. Yang C, Li H, Lin F, Jiang B, Zhao H (2020) Constrained R-Cnn: a general image manipulation detection model. In: 2020 IEEE International Conference on Multimedia and Expo (ICME). IEEE Computer Society, London, UK, pp 1–6

  12. Bi X, Zhang Z, Liu Y, Xiao B, Li W (2021) Multi-task wavelet corrected network for image splicing forgery detection and localization. In: 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE Computer Society, Shenzhen, China, pp 3539–3553

  13. Chen X, Dong C, Ji J, Cao J, Li X (2021) Image manipulation detection by multi-view multi-scale supervision. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, ELECTR NETWORK, pp 14165–14173

  14. Li S, Xu S, Ma W, Zong Q (2021) Image manipulation localization using attentional cross-domain CNN features. IEEE Trans Neural Netw Learn Syst 34: 5614–5628

  15. Sharma P, Kumar M, Sharma HK (2023) A generalized novel image forgery detection method using generative adversarial network. Multimed Tools Appl 1: 1–32

  16. Zhang Y, Tan Q, Qi S, Xue M (2023) PRNU-based Image forgery localization with deep multi-scale fusion. ACM Trans Multimed Comput Commun Appl 19:1551–6857

    Article  Google Scholar 

  17. Lin X, Wang S, Deng J, Ying Fu, Bai X, Chen X, Xiaolei Qu, Tang W (2023) Image manipulation detection by multiple tampering traces and edge artifact enhancement. Pattern Recogn 133:109026

    Article  Google Scholar 

  18. Chen J, Liao X, Wang W, Qian Z, Qin Z, Wang Y (2023) SNIS: a signal noise sepa-ration-based network for post-processed image forgery detection. IEEE Trans Circuits Syst Video Technol 33:935–951

    Article  Google Scholar 

  19. Wu H, Zhou J, Tian J, Liu J, Qiao Y (2022) Robust image forgery detection against transmission over online social networks. IEEE Trans Inf Forensics Secur 17:443–456

    Article  Google Scholar 

  20. Hsu Y, Chang S (2006) Detecting image splicing using geometry invariants and camera characteristics consistency. In: 2006 IEEE International Conference on Multimedia and Expo. IEEE Computer Society, Toronto, ON, Canada, pp 549–552

  21. Dong J, Wang W, Tan T (2013) CASIA image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing. IEEE, Beijing, China, pp 422–426

  22. Wen B, Zhu Y, Subramanian R, Ng T-T, Shen X, Winkler S (2016) COVERAGE — A novel database for copy-move forgery detection. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, Phoenix, AZ, pp 161–165

  23. Guan H, Kozak M, Robertson E, Lee Y, Yates AN, Delgado A, Zhou D, Kheyrkhah T, Smith J, Fiscus J (2019) MFC datasets: large-scale benchmark datasets for media forensic challenge evaluation. In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE Computer Society, Waikoloa, HI, pp 63–72

  24. Novozámský A, Mahdian B, Saic S (2020) IMD2020: A Large-Scale Annotated Dataset Tailored for Detecting Manipulated Images. In: 2020 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE Computer Society, Snowmass, CO, pp 71–80

  25. Ferrara P, Bianchi T, De Rosa A, Piva A (2012) Image forgery localization via fine-grained analysis of CFA artifacts. IEEE Trans Inf Forensics Secur 7:1566–1577

    Article  Google Scholar 

  26. Dong J, Wang W, Tan T, Shi YQ (2009) Run-length and edge statistics based approach for image splicing detection. In: 7th International Workshop, IWDW 2008. Springer, Busan, Korea, pp 76–87

  27. Zhang Z, Zhang Y, Zhou Z, Luo J (2018) Boundary-based image forgery detection by fast shallow CNN. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE Computer Society, Beijing, China, pp 2658–2663

  28. Amerini I, Uricchio T, Ballan L, Caldelli R (2017) Localization of JPEG double compression through multi-domain convolutional neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, Long Beach, CA, USA, pp 1865–1871

  29. Wang J, Wang H, Li J, Luo X, Shi Y-Q, Jha SK (2020) Detecting double JPEG compressed color images with the same quantization matrix in spherical coordinates. IEEE Trans Circuits Syst Video Technol 30:2736–2749

    Article  Google Scholar 

  30. Kwon M-J, Yu I-J, Nam S-H, Lee H-K (2021) CAT-Net: compression artifact tracing network for detection and localization of image splicing. In: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE Computer Society, Waikoloa, HI, USA, pp 375–384

  31. Li W, Li X, Ni R, Zhao Y (2022) Quantization step estimation for JPEG image forensics. IEEE Trans Circuits Syst Video Technol 32:4816–4827

    Article  Google Scholar 

  32. Wang H, Wang J, Luo X, Zheng Y, Ma B, Sun J, Jha SKR (2022) Detecting aligned double JPEG compressed color image with same quantization matrix based on the stability of image. IEEE Trans Circuits Syst Video Technol 32:4065–4080

    Article  Google Scholar 

  33. Kumar M, Srivastava S (2019) Image authentication by assessing manipulations using illumination. Multimed Tools Appl 78:12451–12463

    Article  Google Scholar 

  34. Kumar M, Srivastava S, Uddin N (2019) Forgery detection using multiple light sources for synthetic images. Aust J Forensic Sci 51:243–250

    Article  Google Scholar 

  35. Gou H, Swaminathan A, Wu M (2007) Noise features for image tampering detection and steganalysis. In: 2007 IEEE International Conference on Image Processing. IEEE Computer Society, San Antonio, TX, pp 2893–2896

  36. Pan X, Zhang X, Lyu S (2012) Exposing image splicing with inconsistent local noise variances. In: 2012 IEEE International Conference on Computational Photography (ICCP). IEEE Computer Society, Seattle, WA, USA, pp 1–10

  37. Lyu S, Pan X, Zhang X (2014) Exposing region splicing forgeries with blind local noise estimation. Int J Comput Vision 110:202–221

    Article  Google Scholar 

  38. Bergmann P, Löwe S, Fauser M, Sattlegger D, Steger C (2019) Improving unsupervised defect segmentation by applying structural similarity to autoencoders. In: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. SciTePress, Prague, Czech, pp 372–380

  39. Zhang Y, Wang J, Chen Y, Yu H, Qin T (2022) Adaptive memory networks with self-supervised learning for unsupervised anomaly detection. IEEE Trans Knowl Data Eng 35: 12068–12080

  40. Cozzolino D, Verdoliva L (2016) Single-image splicing localization through autoencoder-er-based anomaly detection. In: 2016 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE Computer Society, Abu Dhabi, United Arab Emirates, pp 1–6

  41. Jia S, Zhang Y (2018) Saliency-based deep convolutional neural network for no-reference image quality assessment. Multimed Tools Appl 77:14859–14872

    Article  Google Scholar 

  42. Liu Z, Mao H, Wu C-Y, Feichtenhofer C, Darrell T, Xie S (2022) A ConvNet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, New Orleans, LA, pp 11976–11986

  43. Yang J, Xu R, Qi Z, Shi Y (2021) Visual anomaly detection for images: a survey. In: 8th International Conference on Information Technology and Quantitative Management (ITQM). Elsevier, Chengdu, China, pp 471–478

  44. Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Honolulu, HI, USA, pp 2117–2125

  45. Kirillov A, Girshick R, He K, Dollar P (2019) Panoptic feature pyramid networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Long Beach, CA, USA, pp 6399–6408

  46. Li H, Huang J (2019) Localization of deep inpainting using high-pass fully convolutional network. In: IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Seoul, South Korea, pp 8301–8310

  47. Zhou P, Chen B-C, Han X, Najibi M, Shrivastava A, Lim S-N, Davis L (2020) Generate, segment, and refine: towards generic manipulation segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, New York, NY, pp 13058–13065

  48. Dong C, Chen X, Hu R, Cao J, Li X (2022) MVSS-Net: multi-view multi-scale supervised networks for image manipulation detection. IEEE Trans Pattern Anal Mach Intell 45:3539–3553

    Google Scholar 

  49. Gao Z, Chen S, Guo Y, Guan W, Nie J, Liu A (2022) Generic image manipulation localization through the lens of multi-scale spatial inconsistence. In: Proceedings of the 30th ACM International Conference on Multimedia. Association for Computing Machinery, Lisbon, Portugal, pp 6146–6154

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (62272331, 61972269) and the Sichuan Science and Technology Program (2022YFG0320).

Author information

Authors and Affiliations

  1. School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China

    Sijiang Meng, Hongxia Wang, Yang Zhou, Qiang Zeng & Rui Zhang

Authors
  1. Sijiang Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiang Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongxia Wang.

Ethics declarations

Conflicts of interest

All authors declare that they have no financial or personal relationships that could have influenced the research presented in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, S., Wang, H., Zhou, Y. et al. Image manipulation localization using reconstruction attention. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19014-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19014-0

Keywords

Search

Navigation