Skip to main content
SpringerLink
Log in

Deep neural networks watermark via universal deep hiding and metric learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

With the rising costs of model training, it is urgent to safeguard the intellectual property of deep neural networks. To achieve this, researchers have proposed various model watermarking techniques. Existing methods utilize visible trigger patterns, which are vulnerable to being detected by humans or detectors. Moreover, these approaches fail to establish active protection mechanisms that link the model with the user’s identity. In this study, we present an innovative imperceptible model watermarking approach that utilizes deep hiding to encode the user’s copyright verification information. This process superimposes a trigger pattern onto clean images, resulting in watermark trigger images. These watermark trigger images closely mimic the original images, achieving excellent stealthiness while enabling the retrieval of the user’s copyright verification information, thus definitively asserting ownership rights. Slight alterations made to the images to maintain stealthiness can weaken the triggering of the watermark pattern. We first leverage the triple loss in metric learning to tackle this challenge of training watermark samples. Using watermark trigger images as anchor samples and selecting appropriate positive and negative samples, we enhance the model’s capability to discern the watermark trigger. Experimental results on CIFAR-10, GTSRB, and Tiny-ImageNet confirm the defender’s capability to embed watermark successfully. The average watermark accuracy exceeds 90%, while the average performance loss is less than 0.05% points. It is also robust to existing watermark removal attacks and backdoor detection methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

CIFAR-10 dataset that supports the findings of this study is openly available at https://www.cs.toronto.edu/~kriz/cifar.html, reference number [55]. GTSRB dataset that supports the findings of this study is openly available at https://sid.erda.dk/public/archives/daaeac0d7ce1152aea9b61d9f1e19370/published-archive.html, reference number [56]. Tiny-ImageNet dataset that supports the findings of this study is openly available at http://cs231n.stanford.edu/tiny-imagenet-200.zip, reference number [57].

References

  1. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  2. Ding Y, Hua L, Li S (2022) Research on computer vision enhancement in intelligent robot based on machine learning and deep learning. Neural Comput Appl 2:1–13

    Google Scholar 

  3. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018:12

    Article  Google Scholar 

  4. Abdel-Hamid O, Mohamed A-R, Jiang H, Deng L, Penn G, Yu D (2014) Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio Speech Lang Process 22(10):1533–1545

    Article  Google Scholar 

  5. Zaidi BF, Selouani SA, Boudraa M, Sidi Yakoub M (2021) Deep neural network architectures for dysarthric speech analysis and recognition. Neural Comput Appl 33:9089–9108

    Article  Google Scholar 

  6. Chowdhary K (2020) Natural language processing. Fundam Artif Intell 12:603–649

    Google Scholar 

  7. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  8. Sun C, Yu W (2008) Neural networks for control, robotics and diagnostics. Neural Comput Appl 17:325–326

    Article  Google Scholar 

  9. Bayraktar E, Yigit CB, Boyraz P (2020) Object manipulation with a variable-stiffness robotic mechanism using deep neural networks for visual semantics and load estimation. Neural Comput Appl 32(13):9029–9045

    Article  Google Scholar 

  10. Orekondy T, Schiele B, Fritz M (2019) Knockoff nets: stealing functionality of black-box models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4954–4963

  11. Tramèr F, Zhang F, Juels A, Reiter MK, Ristenpart T (2016) Stealing machine learning models via prediction \(\{\)APIs\(\}\). In: 25th USENIX security symposium (USENIX Security 16), pp 601–618

  12. Chen H, Rouhani BD, Fu C, Zhao J, Koushanfar F (2019) Deepmarks: a secure fingerprinting framework for digital rights management of deep learning models. In: Proceedings of the 2019 on international conference on multimedia retrieval, pp 105–113

  13. Darvish Rouhani B, Chen H, Koushanfar F (2019) Deepsigns: an end-to-end watermarking framework for ownership protection of deep neural networks. In: Proceedings of the twenty-fourth international conference on architectural support for programming languages and operating systems, pp 485–497

  14. Fan L, Ng KW, Chan CS (2019) Rethinking deep neural network ownership verification: embedding passports to defeat ambiguity attacks. Adv Neural Inf Process Syst 32:20

    CAS  Google Scholar 

  15. Uchida Y, Nagai Y, Sakazawa S, Satoh S (2017) Embedding watermarks into deep neural networks. In: Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp 269–277

  16. Wang T, Kerschbaum F (2021) Riga: covert and robust white-box watermarking of deep neural networks. In: Proceedings of the web conference 2021, pp 993–1004

  17. Zhao X, Yao Y, Wu H, Zhang X (2021) Structural watermarking to deep neural networks via network channel pruning. In: 2021 IEEE international workshop on information forensics and security (WIFS). IEEE, pp 1–6

  18. Li Y, Tondi B, Barni M (2021) Spread-transform dither modulation watermarking of deep neural network. J Inf Secur Appl 63:103004

    Google Scholar 

  19. Ribeiro M, Grolinger K, Capretz MA (2015) Mlaas: machine learning as a service. In: 2015 IEEE 14th international conference on machine learning and applications (ICMLA). IEEE, pp 896–902

  20. Adi Y, Baum C, Cisse M, Pinkas B, Keshet J (2018) Turning your weakness into a strength: watermarking deep neural networks by backdooring. In: 27th USENIX security symposium (USENIX Security 18), pp 1615–1631

  21. Guo J, Potkonjak M (2018) Watermarking deep neural networks for embedded systems. In: 2018 IEEE/ACM international conference on computer-aided design (ICCAD). IEEE, pp 1–8

  22. Jia H, Choquette-Choo CA, Chandrasekaran V, Papernot N (2021) Entangled watermarks as a defense against model extraction. In: 30th USENIX security symposium (USENIX Security 21), pp 1937–1954

  23. Le Merrer E, Perez P, Trédan G (2020) Adversarial frontier stitching for remote neural network watermarking. Neural Comput Appl 32(13):9233–9244

    Article  Google Scholar 

  24. Li Z, Hu C, Zhang Y, Guo S (2019) How to prove your model belongs to you: a blind-watermark based framework to protect intellectual property of DNN. In: Proceedings of the 35th annual computer security applications conference, pp 126–137

  25. Xue M, Sun S, Zhang Y, Wang J, Liu W (2022) Active intellectual property protection for deep neural networks through stealthy backdoor and users’ identities authentication. Appl Intell 5:1–15

    Google Scholar 

  26. Zhang J, Gu Z, Jang J, Wu H, Stoecklin MP, Huang H, Molloy I (2018) Protecting intellectual property of deep neural networks with watermarking. In: Proceedings of the 2018 on asia conference on computer and communications security, pp 159–172

  27. Hua G, Teoh ABJ (2023) Deep fidelity in DNN watermarking: a study of backdoor watermarking for classification models. Pattern Recogn 144:109844

    Article  Google Scholar 

  28. Li Y, Zhu L, Jia X, Jiang Y, Xia S-T, Cao X (2022) Defending against model stealing via verifying embedded external features. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 1464–1472

  29. Cao X, Jia J, Gong NZ (2021) IPGuard: protecting intellectual property of deep neural networks via fingerprinting the classification boundary. In: Proceedings of the 2021 ACM asia conference on computer and communications security, pp 14–25

  30. Lukas N, Zhang Y, Kerschbaum F (2019) Deep neural network fingerprinting by conferrable adversarial examples. arXiv preprint arXiv:1912.00888

  31. Peng Z, Li S, Chen G, Zhang C, Zhu H, Xue M (2022) Fingerprinting deep neural networks globally via universal adversarial perturbations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13430–13439

  32. Wang S, Chang C-H (2021) Fingerprinting deep neural networks—a deepfool approach. In: 2021 IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 1–5

  33. Zhao J, Hu Q, Liu G, Ma X, Chen F, Hassan MM (2020) AFA: adversarial fingerprinting authentication for deep neural networks. Comput Commun 150:488–497

    Article  Google Scholar 

  34. Zheng Y, Wang S, Chang C-H (2022) A DNN fingerprint for non-repudiable model ownership identification and piracy detection. IEEE Trans Inf Forensics Secur 17:2977–2989

    Article  ADS  Google Scholar 

  35. Gao Y, Xu C, Wang D, Chen S, Ranasinghe DC, Nepal S (2019) Strip: a defence against trojan attacks on deep neural networks. In: Proceedings of the 35th annual computer security applications conference, pp 113–125

  36. Wang B, Yao Y, Shan S, Li H, Viswanath B, Zheng H, Zhao BY (2019) Neural cleanse: identifying and mitigating backdoor attacks in neural networks. In: 2019 IEEE symposium on security and privacy (SP). IEEE, pp 707–723

  37. Hitaj D, Hitaj B, Mancini LV (2019) Evasion attacks against watermarking techniques found in MLaaS systems. In: 2019 Sixth international conference on software defined systems (SDS). IEEE, pp 55–63

  38. Namba R, Sakuma J (2019) Robust watermarking of neural network with exponential weighting. In: Proceedings of the 2019 ACM asia conference on computer and communications security, pp 228–240

  39. Neeta D, Snehal K, Jacobs D (2006) Implementation of LSB steganography and its evaluation for various bits. In: 2006 1st International conference on digital information management. IEEE, pp 173–178

  40. Zhang L, Lu Y, Li J, Chen F, Lu G, Zhang D (2023) Deep adaptive hiding network for image hiding using attentive frequency extraction and gradual depth extraction. Neural Comput Appl 5:1–19

    Google Scholar 

  41. Zhang C, Lin C, Benz P, Chen K, Zhang W, Kweon IS (2021) A brief survey on deep learning based data hiding, steganography and watermarking. arXiv e-prints 2103

  42. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083

  43. Xue M, Zhang Y, Wang J, Liu W (2021) Intellectual property protection for deep learning models: taxonomy, methods, attacks, and evaluations. IEEE Trans Artif Intell 3(6):908–923

    Article  Google Scholar 

  44. Chen X, Wang W, Bender C, Ding Y, Jia R, Li B, Song D (2021) Refit: a unified watermark removal framework for deep learning systems with limited data. In: Proceedings of the 2021 ACM asia conference on computer and communications security, pp 321–335

  45. Liu X, Li F, Wen B, Li Q (2021) Removing backdoor-based watermarks in neural networks with limited data. In: 2020 25th International conference on pattern recognition (ICPR). IEEE, pp 10149–10156

  46. Shafieinejad M, Lukas N, Wang J, Li X, Kerschbaum F (2021) On the robustness of backdoor-based watermarking in deep neural networks. In: Proceedings of the 2021 ACM workshop on information hiding and multimedia security, pp 177–188

  47. Zhu M, Gupta S (2017) To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878

  48. Hubara I, Courbariaux M, Soudry D, El-Yaniv R, Bengio Y (2017) Quantized neural networks: training neural networks with low precision weights and activations. J Mach Learn Res 18(1):6869–6898

    MathSciNet  Google Scholar 

  49. Jagielski M, Carlini N, Berthelot D, Kurakin A, Papernot N (2020) High accuracy and high fidelity extraction of neural networks. In: 29th USENIX security symposium (USENIX Security 20), pp 1345–1362

  50. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626

  51. Doan BG, Abbasnejad E, Ranasinghe DC (2020) Februus: input purification defense against trojan attacks on deep neural network systems. In: Annual computer security applications conference, pp 897–912

  52. Zhang C, Benz P, Karjauv A, Sun G, Kweon IS (2020) Udh: universal deep hiding for steganography, watermarking, and light field messaging. Adv Neural Inf Process Syst 33:10223–10234

    Google Scholar 

  53. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  ADS  PubMed  Google Scholar 

  54. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  55. Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images

  56. Stallkamp J, Schlipsing M, Salmen J, Igel C (2011) The German traffic sign recognition benchmark: a multi-class classification competition. In: The 2011 international joint conference on neural networks. IEEE, pp 1453–1460

  57. Le Y, Yang X (2015) Tiny imagenet visual recognition challenge. CS231N 7(7):3

    Google Scholar 

  58. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:54

    Google Scholar 

  59. Chou E, Tramèr F, Pellegrino G, Boneh D (2018) Sentinet: detecting physical attacks against deep learning systems. arXiv preprint arXiv:1812.00292

  60. Hampel FR (1974) The influence curve and its role in robust estimation. J Am Stat Assoc 69(346):383–393

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was supported by the Science and Technology Planning Project of Zhejiang Province under Grant 2022C01090 and the National Natural Science Foundation of China under Grants 62072295.

Author information

Authors and Affiliations

  1. School of Communication and Information Engineering, Shanghai University, Shangda Street, Shanghai, 200444, China

    Zhicheng Ye, Xinpeng Zhang & Guorui Feng

Authors
  1. Zhicheng Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinpeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guorui Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guorui Feng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, Z., Zhang, X. & Feng, G. Deep neural networks watermark via universal deep hiding and metric learning. Neural Comput & Applic 36, 7421–7438 (2024). https://doi.org/10.1007/s00521-024-09469-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09469-5

Keywords

Search

Navigation