Abstract
Recent advancement in artificial intelligence (AI) has resulted in the emergence of Machine Learning as a Service (MLaaS) as a lucrative business model which utilizes deep neural networks (DNNs) to generate revenue. With the investment of huge amount of time, resources, and budgets into researching and developing successful DNN models, it is important for us to protect its intellectual property rights (IPRs) as these models can be easily replicated, shared, or redistributed without the consent of the legitimate owners. So far, a robust protection scheme designed for recurrent neural networks (RNNs) does not exist yet. Thus, this chapter proposes a complete protection framework that includes both white-box and black-box protection to enforce IPR on different variants of RNN. Within the framework, a key gate was introduced for the idea of embedding keys to protect IPR. It designates methods to train RNN models in a specific way such that when an invalid or forged key is presented, the performance of the embedded RNN models will be deteriorated. Having said that, the key gate was inspired by the nature of RNN model, to govern the flow of hidden state and designed in such a way that no additional weight parameters were introduced.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adi, Y., Baum, C., Cisse, M., Pinkas, B., Keshet, J.: Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In: 27th USENIX Security Symposium (USENIX Security 18), pp. 1615–1631 (2018). USENIX Association, Baltimore, MD
Boenisch, F.: A survey on model watermarking neural networks. CoRR, abs/2009.12153 (2020)
Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., Soricut, R., Specia, L., Tamchyna, A.: Findings of the 2014 workshop on statistical machine translation. In: Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 12–58, Baltimore, Maryland, USA (2014). Association for Computational Linguistics, New York
Chen, H., Rouhani, B.D., Fu, C., Zhao, J., Koushanfar, F.: Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In: Proceedings of the 2019 on International Conference on Multimedia Retrieval, pp. 105–113 (2019)
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. Association for Computational Linguistics, Doha, Qatar (2014)
Fan, L., Ng, K.W., Chan, C.S.: Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)
Guo, J., Potkonjak, M.: Watermarking deep neural networks for embedded systems. In: Proceedings of the International Conference on Computer-Aided Design (ICCAD ’18). Association for Computing Machinery, New York (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014)
Krause, B., Lu, L., Murray, I., Renals, S.: Multiplicative LSTM for sequence modelling. CoRR, abs/1609.07959 (2016)
Le, Q.V., Jaitly, N., Hinton, G.E.: A simple way to initialize recurrent networks of rectified linear units. CoRR, abs/1504.00941 (2015)
Le Merrer, E., Perez, P., Trédan, G.: Adversarial frontier stitching for remote neural network watermarking. CoRR, abs/1711.01894 (2017)
Li, X., Roth, D.: Learning question classifiers. In: Proceedings of the 19th International Conference on Computational Linguistics—Volume 1 (COLING ’02), pp. 1–7. Association for Computational Linguistics, USA (2002)
Ong, D.S., Chan, C.S., Ng, K.W., Fan, L., Yang, Q.: Protecting intellectual property of generative adversarial networks from ambiguity attack. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: A method for automatic evaluation of machine translation (ACL ’02), pp. 311–318. Association for Computational Linguistics, USA (2002)
Rouhani, B.D., Chen, H., Koushanfar, F.: Deepsigns: A generic watermarking framework for IP protection of deep learning models. CoRR, abs/1804.00750 (2018)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Uchida, Y., Nagai, Y., Sakazawa, S., Satoh, S.: Embedding watermarks into deep neural networks. In: Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval (2017)
Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M.P., Huang, H., Molloy, I.: Protecting intellectual property of deep neural networks with watermarking. In: Proceedings of the 2018 on Asia Conference on Computer and Communications Security (ASIACCS ’18), pp. 159–172. Association for Computing Machinery, New York (2018)
Zhang, J., Chen, D., Liao, J., Zhang, W., Hua, G., Yu, N.: Passport-aware normalization for deep model protection. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)
Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., Xu, B.: Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. CoRR, abs/1611.06639 (2016)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Tan, Z.Q., Wong, H.S., Chan, C.S. (2023). Protecting Recurrent Neural Network by Embedding Keys. In: Fan, L., Chan, C.S., Yang, Q. (eds) Digital Watermarking for Machine Learning Model. Springer, Singapore. https://doi.org/10.1007/978-981-19-7554-7_9
Download citation
DOI: https://doi.org/10.1007/978-981-19-7554-7_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-7553-0
Online ISBN: 978-981-19-7554-7
eBook Packages: Computer ScienceComputer Science (R0)