Skip to main content
SpringerLink
Log in

Nested Attention U-Net: A Splicing Detection Method for Satellite Images

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12666))

Included in the following conference series:

Abstract

Satellite imagery is becoming increasingly available due to a large number of commercial satellite companies. Many fields use satellite images, including meteorology, forestry, natural disaster analysis, and agriculture. These images can be changed or tampered with image manipulation tools causing issues in applications using these images. Manipulation detection techniques designed for images captured by “consumer cameras” tend to fail when used on satellite images. In this paper we propose a supervised method, known as Nested Attention U-Net, to detect spliced areas in the satellite images. We introduce three datasets of manipulated satellite images that contain objects generated by a generative adversarial network (GAN). We test our approach and compare it to existing supervised splicing detection and segmentation techniques and show that our proposed approach performs well in detection and localization.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Davari, A.A., Christlein, V., Vesal, S., Maier, A., Riess, C.: GMM supervectors for limited training data in hyperspectral remote sensing image classification. In: Felsberg, M., Heyden, A., Krüger, N. (eds.) CAIP 2017. LNCS, vol. 10425, pp. 296–306. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-64698-5_25

    Chapter  Google Scholar 

  2. Shimoni, M., Borghys, D., Heremans, R., Perneel, C., Acheroy, M.: Land-cover classification using fused PolSAR and PolInSAR features. In: Proceedings of the European Conference on Synthetic Aperture Radar, Friedrichshafen, Germany, pp. 1–4, June 2008

    Google Scholar 

  3. Lebedev, V., et al.: Precipitation nowcasting with satellite imagery. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK (2019)

    Google Scholar 

  4. Sahoo, I., Guinness, J., Reich, B.: Estimating atmospheric motion winds from satellite image data using space-time drift models. arXiv:1902.09653, February 2019

  5. Efremova, N., Zausaev, D., Antipov, G.: Prediction of soil moisture content based on satellite data and sequence-to-sequence networks. In: Proceedings of the Conference on Neural Information Processing Systems Women in Machine Learning Workshop, Montreal, Canada (2018)

    Google Scholar 

  6. Helmer, E., Goodwin, N.R., Gond, V., Souza Jr., C.M., Asner, G.P.: Characterizing Tropical Forests with Multispectral Imagery, vol. 2, pp. 367–396. CRC Press, Boca Raton (2015)

    Google Scholar 

  7. Rußwurm, M., Lefèvre, S., Körner, M.: BreizhCrops: a satellite time series dataset for crop type identification. In: Proceedings of the International Conference on Machine Learning Time Series Workshop, Long Beach, CA (2019)

    Google Scholar 

  8. Suraj, P.K., Gupta, A., Sharma, M., Paul, S.B., Banerjee, S.: On monitoring development using high resolution satellite images. arXiv:1712.02282, December 2017

  9. Oshri, B., et al.: Infrastructure quality assessment in Africa using satellite imagery and deep learning. In: Proceedings of the International Conference on Knowledge Discovery & Data Mining, United Kingdom, London (2018)

    Google Scholar 

  10. Guirado, E., Tabik, S., Rivas, M.L., Alcaraz-Segura, D., Herrera, F.: Whale counting in satellite and aerial images with deep learning. Sci. Rep. 9, October 2019

    Google Scholar 

  11. Xia, G., et al.: Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 3974–3983, June 2018

    Google Scholar 

  12. Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, San Jose, CA, pp. 270–279, November 2009

    Google Scholar 

  13. Gupta, R., et al.: xBD: a dataset for assessing building damage from satellite imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 10–17, June 2019

    Google Scholar 

  14. Schmitt, M., Hughes, L.H., Qiu, C., Zhu, X.X.: SEN12MS - a curated dataset of georeferenced multi-spectral sentinel-1/2 imagery for deep learning and data fusion. arXiv:1906.07789, June 2019

  15. Zhou, X., Huang, S., Li, B., Li, Y., Li, J., Zhang, Z.: Text guided person image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA (2019)

    Google Scholar 

  16. Li, B., Qi, X., Lukasiewicz, T., Torr, P.: Controllable text-to-image generation. In: Proceedings of the Neural Information Processing Systems, Vancouver, Canada, pp. 2065–2075, December 2019

    Google Scholar 

  17. Kramer, A.E.: Russian images of Malaysia airlines flight 17 were altered, report finds, The New York Times, 6 November 2018. https://www.nytimes.com/2016/07/16/world/europe/malaysia-airlines-flight-17-russia.html

  18. Byrd, D.: Fake image of diwali still circulating, EarthSky, 15 July 2016. https://earthsky.org/earth/fake-image-of-india-during-diwali-versus-the-real-thing

  19. Edwards, J.: China uses GAN technique to tamper with earth images, ExecutiveGov, 1 April 2019. https://www.executivegov.com/2019/04/ngas-todd-myers-china-uses-gan-technique-to-tamper-with-earth-images/

  20. Rannard, G.: Australia fires: Misleading maps and pictures go viral, BBC, 7 January 2020. https://www.bbc.com/news/blogs-trending-51020564

  21. Barni, M., Phan, Q.-T., Tondi, B.: Copy move source-target disambiguation through multi-branch CNNs. arXiv preprint arXiv:1912.12640 (2019)

  22. Cozzolino, D., Verdoliva, L.: Noiseprint: a CNN-based camera model fingerprint. IEEE Trans. Inf. Forensics Secur. 15, 144–159 (2020)

    Article  Google Scholar 

  23. Ren, C.X., Ziemann, A., Theiler, J., Durieux, A.M.: Deep snow: synthesizing remote sensing imagery with generative adversarial nets. arXiv preprint arXiv:1911.12546 (2020)

  24. Rocha, A., Scheirer, W., Boult, T., Goldenstein, S.: Vision of the unseen: current trends and challenges in digital image and video forensics. ACM Comput. Surv. 43(4), 1–42 (2011)

    Article  Google Scholar 

  25. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, October 2015

    Google Scholar 

  26. Goodfellow, I., et al.: Generative adversarial nets, Montreal, Canada, pp. 2672–2680, December 2014

    Google Scholar 

  27. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA (2020)

    Google Scholar 

  28. Barni, M., Costanzo, A., Sabatini, L.: Identification of cut & paste tampering by means of double-JPEG detection and image segmentation. In: Proceedings of the IEEE International Symposium on Circuits and Systems, Paris, France, pp. 1687–1690, May 2010

    Google Scholar 

  29. McCloskey, S., Albright, M.: Detecting GAN-generated imagery using saturation cues. In: Proceedings of the IEEE International Conference on Image Processing, Taipei, Taiwan, pp. 4584–4588, September 2019

    Google Scholar 

  30. Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., Verdoliva, L.: Forensictransfer: weakly-supervised domain adaptation for forgery detection. arXiv:1812.02510, December 2018

  31. Montserrat, D.M., et al.: Deepfakes detection with automatic face weighting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA (2020)

    Google Scholar 

  32. Cozzolino, D., Poggi, G., Verdoliva, L.: Splicebuster: a new blind image splicing detector. In: Proceedings of the IEEE International Workshop on Information Forensics and Security, Rome, Italy, pp. 1–6, November 2015

    Google Scholar 

  33. Rozsa, A., Zhong, Z., Boult, T.E.: Adversarial attack on deep learning-based splice localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA (2020)

    Google Scholar 

  34. Ho, A.T.S., Woon, W.M.: A semi-fragile pinned sine transform watermarking system for content authentication of satellite images. In: Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Seoul, South Korea, vol. 2, pp. 1–4, July 2005

    Google Scholar 

  35. Kalyan Yarlagadda, S., Güera, D., Bestagini, P., Zhu, S. Tubaro, F., Delp, E.: Satellite image forgery detection and localization using gan and one-class classifier. In: Proceedings of the IS&T International Symposium on Electronic Imaging, Burlingame, CA, vol. 2018, no. 7, pp. 214-1-214-9, February 2018

    Google Scholar 

  36. Horvath, J., et al.: Anomaly-based manipulation detection in satellite images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, pp. 62–71, June 2019

    Google Scholar 

  37. Horváth, J., Montserrat, D.M., Hao, H., Delp, E.J.: Manipulation detection in satellite images using deep belief networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA (2020)

    Google Scholar 

  38. Bartusiak, E.R., et al.: Splicing detection and localization in satellite imagery using conditional GANs. In: Proceedings of the IEEE International Conference on Multimedia Information Processing and Retrieval, San Jose, CA, pp. 91–96, March 2019

    Google Scholar 

  39. Oktay, O., et al.: Attention U-Net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, April 2018

  40. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS 2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1

    Chapter  Google Scholar 

  41. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on U-net (R2U-net) for medical image segmentation," arXiv:1802.06955 (2018)

  42. Ibtehaz, N., Rahman, M.S.: Multiresunet: rethinking the u-net architecture for multimodal biomedical image segmentation. Neural Netw. 121, 74–87 (2020)

    Article  Google Scholar 

  43. Jetley, S., Lord, N.A., Lee, N., Torr, P.: Learn to pay attention. In: Proceedings of the International Conference on Learning Representations, Vancouver, Canada (2018)

    Google Scholar 

  44. Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.J.: The importance of skip connections in biomedical image segmentation. In: Proceedings of the 2nd Deep Learning in Medical Image Analysis Workshop, Greece, Athens (2016)

    Google Scholar 

  45. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, San Diego,California, USA, vol. 38, pp. 562–570, May 2015

    Google Scholar 

  46. Zhang, D., Khoreva, A.: Progressive augmentation of GANs. In: Proceedings of the Neural Information Processing Systems, Vancouver, Canada (2019)

    Google Scholar 

  47. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the International Conference on Computer Vision, Venice, Italy (2017)

    Google Scholar 

  48. Sentinel 2 images, Copernicus programme. https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi/processing-levels

  49. Roerdink, J.B.T.M., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundamenta Informaticae 41(1–2), 187–228 (2000)

    Article  MathSciNet  Google Scholar 

  50. Levandowsky, M., Winter, D.: Measures of the amount of ecologic association between species. Nature 5(234), 34–35 (1971)

    Article  Google Scholar 

Download references

Acknowledgment

This material is based on research sponsored by the Defense Advanced Research Projects Agency (DARPA) and the Air Force Research Laboratory (AFRL) under agreement number FA8750-16-2-0173. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA or AFRL or the U.S. Government.

Address all correspondence to Edward J. Delp, ace@ecn.purdue.edu.

Author information

Authors and Affiliations

  1. Video and Image Processing Laboratory (VIPER), School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA

    János Horváth, Daniel Mas Montserrat, Edward J. Delp & János Horváth

Authors
  1. János Horváth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Mas Montserrat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward J. Delp
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. János Horváth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to János Horváth .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Horváth, J., Montserrat, D.M., Delp, E.J., Horváth, J. (2021). Nested Attention U-Net: A Splicing Detection Method for Satellite Images. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12666. Springer, Cham. https://doi.org/10.1007/978-3-030-68780-9_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68780-9_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68779-3

  • Online ISBN: 978-3-030-68780-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation