Abstract
A secure machine learning technology that performs prediction while encrypting data using homomorphic encryption is being developed. However, Convolutional Neural Networks (CNN) on homomorphic encryption cannot use general non-linear activation functions, Thus, the classification accuracy is low. We proposed a novel method to create an activation function that improves the classification accuracy by performing a pre-training optimization on the coefficients of the polynomial approximation of the Mish function. We confirmed the improvement of classification accuracy for MNIST, Fashion-MNIST, and CIFAR-10 by optimizing the Mish function through pre-training. The classification accuracy can be improved by 4.27% for CIFAR-10. Furthermore, we showed that classification accuracy improves for Fashion-MNIST and CIFAR-10 even when different networks and datasets are optimized by pre-training the activation function. These results show that the activation function of CNNs on a homomorphic encryption can be optimized to improve classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Nair, V., Hinton, G.: Cifar-10 and cifar-100 datasets (canadian institutefor advanced research). http://www.cs.toronto.edu/~kriz/cifar.html. Accessed 23 Jan 2022
Abadi, M., et al.: Tensorflow: A system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 2016), pp. 265–283 (2016)
Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019)
Boemer, F., Costache, A., Cammarota, R., Wierzynski, C.: ngraph-he2: a high-throughput framework for neural network inference on encrypted data. In: Proceedings of the 7th ACM Workshop on Encrypted Computing & Applied Homomorphic Cryptography, pp. 45–56 (2019)
Boemer, F., Lao, Y., Cammarota, R., Wierzynski, C.: ngraph-he: a graph compiler for deep learning on homomorphically encrypted data. In: Proceedings of the 16th ACM International Conference on Computing Frontiers, pp. 3–13 (2019)
Chabanne, H., de Wargny, A., Milgram, J., Morel, C., Prouff, E.: Privacy-preserving classification on deep neural network. IACR Cryptol. ePrint Arch. 2017, 35 (2017)
Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_15
Erickson, B.J., Korfiatis, P., Akkus, Z., Kline, T.L.: Machine learning for medical imaging. Radiographics 37(2), 505–515 (2017)
Gentry, C.: A fully homomorphic encryption scheme. Stanford university (2009)
Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., Wernsing, J.: Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. In: International Conference on Machine Learning, pp. 201–210. PMLR (2016)
Hesamifard, E., Takabi, H., Ghasemi, M.: Deep neural networks classification over encrypted data, pp. 97–108. Association for Computing Machinery (2019). https://doi.org/10.1145/3292006.3300044
Ishiyama, T., Suzuki, T., Yamana, H.: Highly accurate CNN inference using approximate activation functions over homomorphic encryption. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 3989–3995. IEEE (2020)
Laine, K.: Microsoft seal. https://github.com/microsoft/SEAL. Accessed 30 Jan 2022
Misra, D.: Mish: a self regularized non-monotonic neural activation function. arXiv preprint arXiv:1908.08681 4, 2 (2019)
Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Workshop Track Proceedings. OpenReview.net (2018). https://openreview.net/forum?id=Hkuq2EkPf
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1409.1556
Smart, N.P., Vercauteren, F.: Fully homomorphic simd operations. Des. Codes Cryptogr. 71(1), 57–81 (2014)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)
LeCun, C.Y.: Cortes: mnist handwritten digit database. http://yann.lecun.com/exdb/mnist/. Accessed 23 Jan 2022
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yagyu, K., Takeuchi, R., Nishigaki, M., Ohki, T. (2023). Improving Classification Accuracy by Optimizing Activation Function for Convolutional Neural Network on Homomorphic Encryption. In: Barolli, L. (eds) Advances on Broad-Band Wireless Computing, Communication and Applications. BWCCA 2022. Lecture Notes in Networks and Systems, vol 570. Springer, Cham. https://doi.org/10.1007/978-3-031-20029-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-20029-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20028-1
Online ISBN: 978-3-031-20029-8
eBook Packages: EngineeringEngineering (R0)