Skip to main content
SpringerLink
Log in

Pairwise open-sourced dataSet protection based on adaptive blind watermarking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The cost of collecting and labeling open-sourced datasets which promote the development of deep learning is expensive. Thus, it is important to design an efficient open-sourced data set protection algorithm to detect the open-sourced data sets used in illegal ways. In this paper, a protection algorithm based on adaptive robust blind-watermark is proposed suitable for multiple paired open-sourced datasets, and the evaluation criteria of the algorithm are defined. Specifically, in the embed stage, the highly concealed of the watermark is realized by combining UNet and double GAN to take into account the local and global features of the carrier and the watermark image. A preprocess network is used in the Embedding network to adapt different watermark size. In the extraction part, a modified feature sharing UNet with GAN is used to ensure robustness of the extraction network. Paired datasets are used for training to ensure accurate extraction of watermarks. After the target model is trained using the watermarked dataset, its inference output will contain watermark information. When it is believed that a suspicious model is illegally trained with the dataset, it can be verified by the watermark extracted from inference output of the suspicious model. We evaluate our method on three target models including nine datasets. The results show that our framework successfully verifies the dataset used illegally and without a noticeable impact on the target model task when training with the watermark dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Talo M, Yildirim O, Baloglu UB, Aydin G, Acharya UR (2019) “Convolutional neural networks for multi-class brain disease detection using MRI images,” Comput Med Imag Graph, 78, 12

  2. Zheng X, Chen J, Wang H, Zheng S, Kong Y (2021) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Appl Intell 51(3):1262–1279

    Article  Google Scholar 

  3. Gridach M (2020) A framework based on (probabilistic) soft logic and neural network for NLP. Appl Soft Comput 93:8

    Article  Google Scholar 

  4. Jelodar H, Wang YL, Orji R, Huang SC (2020) Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach. IEEE J Biomed Health Inform 24(10):2733–2742

    Article  Google Scholar 

  5. Wang SL, Takyi-Aninakwa P, Jin SY, Yu CM, Fernandez C, Daniel-Ioan S (2022) An improved feedforward-long short-term memory modeling method for the whole-life-cycle state of charge prediction of lithium-ion batteries considering current-voltage-temperature variation. Energy 254:124224

    Article  Google Scholar 

  6. Wang SL, Jin SY, Bai DE, Fan YC, Shi HT, Fernandez C (2021) A critical review of improved deep learning methods for the remaining useful life prediction of lithium-ion batteries. Energy Rep 7:5562–5574

    Article  Google Scholar 

  7. Liu XL, Lin CC, Yuan SM (2018) Blind dual watermarking for color images' authentication and copyright protection. IEEE Transac Circuits Sys Vid Technol 28(5):1047–1055

    Article  Google Scholar 

  8. Darwish SM, Al-Khafaji LDS (2020) Dual watermarking for color images: a new image copyright protection model based on the fusion of successive and segmented watermarking. Multimed Tools Appl 79(9–10):6503–6530

    Article  Google Scholar 

  9. Yao YZ, Zhang WM, Wang H, Zhou H, Yu NH (2019) Content-adaptive reversible visible watermarking in encrypted images. Sig Proc 164:386–401

    Article  Google Scholar 

  10. Wang X, Shu TZ, Xie M, Pei QQ (2018) “Adaptive robust reversible watermarking scheme,” International Conference on Cloud Computing and Security, pp. 221–230

  11. Liu XL, Wu YF, Shao ZH, Wu JS, Shu HZ (2018) Color image watermarking using a discrete trinion Fourier transform. J Electro ing 27(4):14

    Google Scholar 

  12. Anand A, Singh AK (2020) An improved DWT-SVD domain watermarking for medical information security. Comput Commun 152:72–80

    Article  Google Scholar 

  13. Singh SP, Bhatnagar G (2018) A new robust watermarking system in integer DCT domain. J Vis Commun Image Represent 53:86–101

    Article  Google Scholar 

  14. Ahmadi SBB, Zhang G, Rabbani M, Boukela L, Jelodar H (2020) An intelligent and blind dual color image watermarking for authentication and copyright protection. Appl Intell 51(3):1701–1732

    Article  Google Scholar 

  15. Zhu JR, Kaplan R, Johnson J, Li FF (2018) “HiDDeN: hiding data with deep networks,” Proceedings of the European Conference on Computer Vision, pp. 682–697

  16. Zhong X, Huang PC, Mastorakis S, Shih FY (2021) An automated and robust image watermarking scheme based on deep neural networks. IEEE Transactions on Multimedia 23:1951–1961

    Article  Google Scholar 

  17. Yu C, and I. Assoc Advancement Artificial (2020) “Attention based data hiding with generative adversarial networks,” Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1120–1128

  18. Zhang J, Chen D, Liao J, Zhang W, Feng H, Hua G, Yu N (2021) “Deep model intellectual property protection via deep watermarking,” IEEE Trans Pattern Anal Mach Intell

  19. Li YM, Liu PD, Jiang Y, Xia ST (2021) “Visual privacy protection via mapping distortion.” IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 3740–3744

  20. Chen KJ, Zeng XH, Ying QC, Li S, Qian ZX, Zhang XP (2022) “Invertible image dataset protection,” IEEE International Conference on Multimedia and Expo, pp. 01–06

  21. Xue MF, Wu YH, Zhang YS, Wang J, Liu WQ (2022) “Dataset authorization control: protect the intellectual property of dataset via reversible feature space adversarial examples.” Appl Intell, 1-12

  22. Ma C, Jiang ZY, Rao YM, Lu JW, Zhou J (2020) “Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 5568–5577

  23. Chai XL, Wang YJ, Chen XH, Gan ZH, Zhang YS (2022) “TPE-GAN: thumbnail preserving encryption based on GAN with key,” IEEE Signal Processing Letters, 29, 972–976

  24. Luo YW, Zheng ZD, Zheng L, Guan T, Yu JQ, Yang Y (2018) “Macro-micro adversarial network for human parsing,” Proceedings of the European conference on computer vision pp. 424–440

  25. Yoo S, Bahng H, Chung S, Lee J, Chang J, Choo J, Soc IC (2019) “Coloring with limited data: few-shot colorization via memory augmented networks,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 11275–11284

  26. Isola P, Zhu JY, Zhou TH, Efros AA (2017) “Image-to-image translation with conditional adversarial networks,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 5967–5976

  27. Park T, Liu MY, Wang TC, Zhu JY, Soc IC (2019) “Semantic image synthesis with spatially-adaptive normalization,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 2332–2341

  28. Richardson E, Alaluf Y, Patashnik O, Nitzan Y, Azar Y, Shapiro S, Cohen-Or D (2021) “Encoding in style: a StyleGAN encoder for image-to-image translation,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 2287–2296

  29. Li R, Cao WM, Jiao QF, Wu S, Wong HS (2020) “Simplified unsupervised image translation for semantic segmentation adaptation,” Pattern Recogn, 105, 12

  30. Sultana M, Mahmood A, Jung SK (2021) Unsupervised moving object detection in complex scenes using adversarial regularizations. IEEE Transac Multimed 23:2005–2018

    Article  Google Scholar 

  31. Ronneberger O, Fischer P, Brox T (2015) “U-net: convolutional networks for biomedical image segmentation,” International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241

  32. Isensee F, Jaeger PF, Kohl SAA, Jens P, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods 18:203–211

    Article  Google Scholar 

  33. Wang JD, Sun K, Cheng TH, Jiang BR, Deng CR, Zhao Y, Liu D, Mu YD, Tan MK, Wang XG, Liu WY, Xiao B (2020) Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell 43(10):3349–3364

    Article  Google Scholar 

  34. Qin XB, Zhang ZC, Huang CY, Dehghan M, Zaiane OR, Jagersand M (2020) “U2-Net: going deeper with nested u-structure for salient object detection,” Pattern Recogn, 106, 12

  35. Wang W, Li Q, Xiao CY, Zhang DZ, Miao L, Wang L (2021) An improved boundary-aware U-Net for ore image semantic segmentation. Sensors 21(8):16

    Article  Google Scholar 

  36. Zhu JY, Park T, Isola P, Efros AA (2017) “Unpaired image-to-image translation using cycle-consistent adversarial networks,” Proceedings of the IEEE International Conference on Computer Vision pp. 2242–2251

  37. Johnson J, Alahi A, Li FF (2016) “Perceptual losses for real-time style transfer and super-resolution,” Proceedings of the European Conference on Computer Vision. pp. 694–711

  38. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) “Generative adversarial nets,” Adv Neural Inf Proces Syst pp. 2672–2680

  39. Liu ZW, Luo P, Wang XG, and Tang XO (2015) “Deep learning face attributes in the wild,” Proceedings of the IEEE International Conference on Computer Vision. pp. 3730–3738

  40. V. Le, J. Brandt, Z. Lin, L. Bourdev, and T. S. Huang (2012) “Interactive facial feature localization,” Proceedings of the European Conference on Computer Vision pp. 679–692

  41. M. E. Nilsback, A. Zisserman, and I. C. Society (2008) “Automated flower classification over a large number of classes.” 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing. pp. 722–729

  42. Gong K, Liang XD, Zhang DY, Shen XH, Lin L (2017) “Look into person: self-supervised structure-sensitive learning and a new benchmark for human parsing,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 6757–6765

  43. Chen XJ, Mottaghi R, Liu XB, Fidler S, Urtasun R, Yuille A (2014) “Detect what you can: detecting and representing objects using holistic models and body parts,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 1979–1986

  44. Luo P, Wang XG, Tang XO (2013) “Pedestrian parsing via deep decompositional network,” Proceedings of the IEEE International Conference on Computer Vision. pp. 2648–2655

  45. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang ZH, Shi WZ (2017) “Photo-realistic single image super-resolution using a generative adversarial network,” Proc IEEE Conf Comput Vis Pattern Recognit pp. 105–114

  46. Roop S, Alaknanda A, Mukesh S (2020) Optimised robust watermarking technique using CKGSA in DCT-SVD domain. IET Image Process 14(10):2052–2063

    Article  Google Scholar 

Download references

Acknowledgments

All the authors are deeply grateful to the editors for smooth and fast handling of the manuscript. The authors would also like to thank the anonymous referees for their valuable suggestions to improve the quality of this paper. This work is supported by the National Natural Science Foundation of China (Grant No. 61802111, 61872125), Key Research and Development and Promotion Special Project of Henan Province (Grant No. 202102210380), Postgraduate Education Reform and Quality Improvement Project of Henan Province (Grant No. YJS2022JD26), Postgraduate Education Innovation Training Base (Grant No. SYLJD2022008), Open Foundation of Henan Key Laboratory of Cyberspace Situation Awareness (Grant No. HNTS2022019) and Pre-Research Project of Songshan Laboratory (Grant No. YYJC012022011).

Author information

Authors and Affiliations

  1. School of Software, Intelligent Data Processing Engineering Research Center of Henan Province, Institute of Intelligent Network System, Henan University, Kaifeng, 475004, China

    Zilong Pang & Zhihua Gan

  2. School of Artificial Intelligence, Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Zhengzhou, 450046, China

    Mingxu Wang, Lvchen Cao & Xiuli Chai

  3. Henan Key Laboratory of Cyberspace Situation Awareness, Zhengzhou, 450001, China

    Xiuli Chai

Authors
  1. Zilong Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingxu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lvchen Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuli Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihua Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuli Chai.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, Z., Wang, M., Cao, L. et al. Pairwise open-sourced dataSet protection based on adaptive blind watermarking. Appl Intell 53, 17391–17410 (2023). https://doi.org/10.1007/s10489-022-04416-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04416-0

Keywords

Search

Navigation