Skip to main content
SpringerLink
Log in

Joint manipulation trace attention network and adaptive fusion mechanism for image splicing forgery localization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Splicing forgery, which manipulates images by copying regions from donor images and pasting them to host images, is one of the common types of image forgery in life, where the copied regions include object regions or background regions. In order to accurately detect these forgery regions, the most mainstream approach is to use an encoder-decoder network architecture that extracts enough manipulation traces to determine whether each pixel of the input image has been spliced or not. However, due to the limited receptive field of such networks, only local manipulation traces can be learned, and therefore some large object area forgery and background forgery cannot be well localized. To address these issues, in this paper, an end-to-end splicing detection framework is proposed, which includes localization network L-Net, manipulation traces attention network MTA-Net, and adaptive multi-scale fusion module. The localization network L-Net is designed as an encoder-decoder network to extract local manipulation traces for each pixel and implement localization of splicing areas. MTA-Net uses the proposed content-remove convolutional layer (CRCL) to suppress image content information that would hinder the network from learning to manipulate traces, and then uses subsequent convolutional layers to extract features to discriminate whether the input image is a spliced image or not. In this process, the regions in the feature map of the convolutional layers with large activation values are the ones that contain global manipulation traces. These global manipulation traces are fused with the local manipulation traces learned by L-Net through the proposed adaptive multi-scale fusion module (AMSFM), thus allowing L-Net to effectively handle object forgery and background region forgery images of various sizes. Ablation experiments showed an increase of 4.6% and 3.9% in F1-score and MCC after the introduction of MTA-Net and AMSFM, respectively The splicing region detection performance on three standard datasets, CASIA, COLUMB, and CARVALHO, shows that the proposed method outperforms the state-of-the-art methods for both object forgery and background forgery, and is more robust to post-processing methods such as JPEG compression and noise addition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bappy JH, Roy-Chowdhury AK, Bunk J, Nataraj L, Manjunath B (2017) Exploiting spatial structure for localizing manipulated image regions. In: Proceedings of the IEEE international conference on computer vision, pp 4970–4979

  2. Bayar B, Stamm MC (2018) Constrained convolutional neural networks: a new approach towards general purpose image manipulation detection. IEEE Trans Inform Forens Secur 13(11):2691–2706

    Article  Google Scholar 

  3. Bondi L, Lameri S, Güera D., Bestagini P, Delp EJ, Tubaro S (2017) Tampering detection and localization through clustering of camera-based cnn features. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1855–1864. IEEE

  4. Chen H, Chang C, Shi Z, Lyu Y (2021) Hybrid features and semantic reinforcement network for image forgery detection. Multimed Syst:1–12

  5. Chen X, Dong C, Ji J, Cao J, Li X (2021) Image manipulation detection by multi-view multi-scale supervision. arXiv:2104.06832

  6. De Carvalho TJ, Riess C, Angelopoulou E, Pedrini H, de Rezende Rocha A (2013) Exposing digital image forgeries by illumination color classification. IEEE Trans Inform Forens Secur 8(7):1182–1194

    Article  Google Scholar 

  7. Ding H, Chen L, Tao Q, Fu Z, Dong L, Cui X (2021) Dcu-net: a dual-channel u-shaped network for image splicing forgery detection. Neural Comput Appl:1–17

  8. Dong J, Wang W, Tan T (2010) Casia image tampering detection evaluation database 2010. doi 10:422–426

    Google Scholar 

  9. Dong J, Wang W, Tan T (2013) Casia image tampering detection evaluation database. In: 2013 IEEE China summit and international conference on signal and information processing, pp 422–426. IEEE

  10. Dong S, Wang P, Abbas K (2021) A survey on deep learning and its applications. Comput Sci Rev 100379:40

    MathSciNet  MATH  Google Scholar 

  11. Farid H (2009) Exposing digital forgeries from jpeg ghosts. IEEE Trans Inform Forens Secur 4(1):154–160

    Article  Google Scholar 

  12. Ferrara P, Bianchi T, De Rosa A, Piva A (2012) Image forgery localization via fine-grained analysis of cfa artifacts. IEEE Trans Inform Forens Secur 7(5):1566–1577

    Article  Google Scholar 

  13. Goljan M, Fridrich J (2015) Cfa-aware features for steganalysis of color images. In: Media watermarking, security, and forensics 2015, vol 9409, p 94090v. International society for optics and photonics

  14. He K, Gkioxari G, Dollár P., Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  15. Hsu YF, Chang SF (2006) Detecting image splicing using geometry invariants and camera characteristics consistency. In: 2006 IEEE international conference on multimedia and expo, pp 549–552 IEEE

  16. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  17. Hu X, Zhang Z, Jiang Z, Chaudhuri S, Yang Z, Nevatia R (2020) Span: spatial pyramid attention network for image manipulation localization. In: European conference on computer vision, pp 312–328. Springer

  18. Huh M, Liu A, Owens A, Efros AA (2018) Fighting fake news: image splice detection via learned self-consistency. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 101–117

  19. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  20. Krawetz N, Solutions HF (2007) A picture’s worth. Hacker Factor Solution 6(2):2

    Google Scholar 

  21. Li H, Chen X, Zhuang P, Li B (2021) Image tampering localization using unified two-stream features enhanced with channel and spatial attention. In: Chinese conference on pattern recognition and computer vision (PRCV), pp 610–622. Springer

  22. Li W, Yuan Y, Yu N (2009) Passive detection of doctored jpeg image via block artifact grid extraction. Signal Process 89(9):1821–1829

    Article  Google Scholar 

  23. Liu S, Huang D et al (2018) Receptive field block net for accurate and fast object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 385–400

  24. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  25. Mahdian B, Saic S (2009) Using noise inconsistencies for blind image forensics. Image Vis Comput 27(10):1497–1503

    Article  Google Scholar 

  26. Pan X, Zhang X, Lyu S (2012) Exposing image splicing with inconsistent local noise variances. In: 2012 IEEE International conference on computational photography (ICCP), pp 1–10. IEEE

  27. Pannu HS, Ahuja S, Dang N, Soni S, Malhi AK (2020) Deep learning based image classification for intestinal hemorrhage. Multimed Tools Appl 79(29):21941–21966

    Article  Google Scholar 

  28. Pham NT, Lee JW, Kwon GR, Park CS (2019) Hybrid image-retrieval method for image-splicing validation. Symmetry 11(1):83

    Article  Google Scholar 

  29. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241

  30. Salloum R, Ren Y, Kuo CCJ (2018) Image splicing localization using a multi-task fully convolutional network (mfcn). J Vis Commun Image Represent 51:201–209

    Article  Google Scholar 

  31. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv:1602.07261

  32. Wang X, Wang H, Niu S, Zhang J (2019) Detection and localization of image forgeries using improved mask regional convolutional neural network. Math Bioscience Eng

  33. Wu Y, He K (2018) Group normalization. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  34. Xiao B, Wei Y, Bi X, Li W, Ma J (2020) Image splicing forgery detection combining coarse to refined convolutional neural network and adaptive clustering. Inf Sci 511:172–191

    Article  MathSciNet  Google Scholar 

  35. Yang C, Li H, Lin F, Jiang B, Zhao H (2020) Constrained r-cnn: a general image manipulation detection model. In: 2020 IEEE International conference on multimedia and expo (ICME), pp 1–6. IEEE

  36. Yang C, Wang Z, Shen H, Li H, Jiang B (2021) Multi-modality image manipulation detection. In: 2021 IEEE International conference on multimedia and expo (ICME), pp 1–6. IEEE

  37. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122

  38. Yu L, Liu N, Zhou W, Dong S, Fan Y, Abbas K (2021) Weber’s law based multi-level convolution correlation features for image retrieval. Multimed Tools Appl 80(13):19157–19177

    Article  Google Scholar 

  39. Zampoglou M, Papadopoulos S, Kompatsiaris Y (2017) Large-scale evaluation of splicing localization algorithms for web images. Multimed Tools Appl 76(4):4801–4834

    Article  Google Scholar 

  40. Zhang Y, Goh J, Win LL, Thing VL (2016) Image region forgery detection: a deep learning approach. SG-CRC 2016:1–11

    Google Scholar 

  41. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

  42. Zhou P, Han X, Morariu VI, Davis LS (2017) Two-stream neural networks for tampered face detection. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1831–1839. IEEE

  43. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1053–1061

Download references

Funding

This work is supported by National Natural Science Foundation of Guangdong [Grant No. 2021A1515012020], and Guangzhou science and technology plan project [Grant No.202002030298].

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou, China

    Yuanlu Wu, Yan Wo & Guoqiang Han

Authors
  1. Yuanlu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Wo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoqiang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Wo.

Ethics declarations

Conflict of Interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Wo, Y. & Han, G. Joint manipulation trace attention network and adaptive fusion mechanism for image splicing forgery localization. Multimed Tools Appl 81, 38757–38780 (2022). https://doi.org/10.1007/s11042-022-13151-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13151-0

Keywords

Search

Navigation