Abstract
Open-set recognition and adversarial defense study two key aspects of deep learning that are vital for real-world deployment. The objective of open-set recognition is to identify samples from open-set classes during testing, while adversarial defense aims to robustify the network against images perturbed by imperceptible adversarial noise. This paper demonstrates that open-set recognition systems are vulnerable to adversarial samples. Furthermore, this paper shows that adversarial defense mechanisms trained on known classes are unable to generalize well to open-set samples. Motivated by these observations, we emphasize the necessity of an Open-Set Adversarial Defense (OSAD) mechanism. This paper proposes an Open-Set Defense Network with Clean-Adversarial Mutual Learning (OSDN-CAML) as a solution to the OSAD problem. The proposed network designs an encoder with dual-attentive feature-denoising layers coupled with a classifier to learn a noise-free latent feature representation, which adaptively removes adversarial noise guided by channel and spatial-wise attentive filters. Several techniques are exploited to learn a noise-free and informative latent feature space with the aim of improving the performance of adversarial defense and open-set recognition. First, we incorporate a decoder to ensure that clean images can be well reconstructed from the obtained latent features. Then, self-supervision is used to ensure that the latent features are informative enough to carry out an auxiliary task. Finally, to exploit more complementary knowledge from clean image classification to facilitate feature denoising and search for a more generalized local minimum for open-set recognition, we further propose clean-adversarial mutual learning, where a peer network (classifying clean images) is further introduced to mutually learn with the classifier (classifying adversarial images). We propose a testing protocol to evaluate OSAD performance and show the effectiveness of the proposed method on white-box attacks, black-box attacks, as well as the rectangular occlusion attack in multiple object classification datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alex Krizhevsky, V. N., & Hinton, G. (2010). Cifar-10(canadian institute for advanced research).
Baweja, Y., Oza, P., Perera, P., & Patel, V. M. (2020). Anomaly detection-based unknown face pre- sentation attack detection. In IJCB.
Bendale, A., & Boult, T. E. (2016). Towards open set deep networks. In CVPR.
Buades, A., Coll, B., & Morel, J. M. (2005). A non-local algorithm for image denoising. In CVPR.
Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In SP.
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In CVPR.
Doersch, C., Gupta, A., & Efros, A. A. (2015). Unsupervised visual representation learning by context prediction. In ICCV.
Doersch, C., & Zisserman, A. (2017). Multi-task self-supervised visual learning. In ICCV.
Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., & Song, D. (2018). Robust physical-world attacks on deep learning models. In CVPR.
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018). Robust physical-world attacks on deep learning visual classification. In CVPR.
Ge, Z., Demyanov, S., Chen, Z., & Garnavi, R. (2017). Generative openmax for multi-class open set classification. In BMVC.
Gidaris, S., Singh, P., & Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. In ICLR.
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. In ICLR.
Gupta, P., & Rahtu, E. (2019). Ciidefence: Defeating adversarial attacks by fusing class-specific image inpainting and image denoising. In CVPR.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In CVPR.
Hendrycks, D., & Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR.
Hendrycks, D., Mazeika, M., Kadavath, S., & Song, D. (2019). Using self-supervised learning can improve model robustness and uncertainty. In NIPS.
Jang, Y., Zhao, T., Hong, S., & Lee, H. (2019). Adversarial defense via learning to generate diverse attacks. In ICCV.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Kurakin, A., Goodfellow, I., & Bengio, S. (2017). Adversarial machine learning at scale. In ICLR.
Lan, X., Ye, M., Shao, R., Zhong, B., Yuen, P. C., & Zhou, H. (2019). Learningmodality-consistency feature templates: Arobust rgb-infrared tracking system. IEEE Transactions Industrial Electronics, 66(12), 9887–9897.
Liang, S., Li, Y., & Srikant, R. (2018). Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR.
Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., & Zhu, J. (2018). Defense against adversarial attacks using high-level representation guided denoiser. In CVPR.
Liu, Y., Chen, X., Liu, C., & Song, D. (2017). Delving into transferable adversarial examples and black-box attacks. In ICLR.
Maaten, Lvd, & Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(Nov), 2579–2605.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In ICLR.
Neal, L., Olson, M., Fern, X., Wong, W. K., & Li, F. (2018). Open set learning with counterfactual images. In ECCV.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., & Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning.
Oza, P., Nguyen, H. V., & Patel, V. M. (2020). Multiple class novelty detection under data distribution shift. In ECCV.
Oza, P., Patel, V.M.: One-class convolutional neural network. IEEE Signal Processing Letters 26(2), 277–281 (2018).
Oza, P., & Patel, V. M. (2019). C2ae: Class conditioned auto-encoder for open-set recognition. In CVPR.
Oza, P., & Patel, V. M. (2020). Utilizing patch-level activity patterns for multiple class novelty detection. In ECCV.
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017). Practical black-box attacks against machine learning. In ASI-ACCS.
Perera, P., Morariu, V. I., Jain, R., Manjunatha, V., Wigington, C., Ordonez, V., & Patel, V. M. (2020). Generative-discriminative feature representations for open-set recognition. In CVPR.
Perera, P., Nallapati, R., & Xiang, B. (2019). OCGAN: One-class novelty detection using gans with constrained latent representations. In CVPR.
Perera, P., & Patel, V. M. (2019). Deep transfer learning for multiple class novelty detection. In CVPR.
Perera, P., Patel, V.M.: Learning deep features for one-class classification. IEEE Transactions on Image Processing 28(11), 5450–5463 (2019).
Scheirer, W. J., Rocha, A., Sapkota, A., & Boult, T. E. (2013). Towards open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 35:1.
Shao, R., Lan, X., Li, J., & Yuen, P. C. (2019). Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In CVPR.
shao, R., Lan, X., & Yuen, P. C. (2019). Joint discriminative learning of deep dynamic textures for 3D mask face anti-spoofing. IEEE Transactions on Information Forensics and Security, 14(4), 923– 938
Shao, R., Lan, X., & Yuen, P. C. (2020). Regularized fine-grained meta face anti-spoofing. In AAAI.
Shao, R., Perera, P., Yuen, P. C., & Patel, V. M. (2020). Open-set adversarial defense. In ECCV.
Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016). Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In ASI-ACCS.
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In ICLR.
Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In ICCV.
Woo, S., Park, J., Lee, J. Y., & So Kweon, I. (2018). Cbam: Convolutional block attention module. In ECCV.
Wu, T., Tong, L., & Vorobeychik, Y. (2020). Defending against physically realizable attacks on image classification. In ICLR.
Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A. L., & Le, Q. V. (2020). Adversarial examples improve image recognition. In CVPR.
Xie, C., Wu, Y., Maaten, L. v. d., Yuille, A. L., & He, K. (2019). Feature denoising for improving adversarial robustness. In CVPR.
Xie, C., & Yuille, A. (2020). Intriguing properties of adversarial training at scale. In ICLR.
Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., & Hoi, S. C. H. (2020). Deep learning for person re-identification: A survey and outlook. arXiv preprint arXiv:2001.04193.
Ye, M., Shen, J., Zhang, X., Yuen, P. C., & Chang, S. F. (2020). Augmentation invariant and instance spreading feature for softmax embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Ye, M., Zhang, X., Yuen, P. C., & Chang, S. F. (2019). Unsupervised embedding learning via invariant and spreading instance feature. In CVPR.
Yoshihashi, R., Shao, W., Kawakami, R., You, S., Iida, M., & Naemura, T. (2019). Classification-reconstruction learning for open-set recognition. In CVPR.
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., & Xiao, J. (2015). Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365
Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In ECCV.
Zhang, H., Patel, V.M.: Sparse representation-based open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(8), 1690–1696 (2016).
Zhang, Y., Xiang, T., Hospedales, T. M., & Lu, H. (2018). Deep mutual learning. In CVPR.
Acknowledgements
This work is partially supported by Research Grants Council (RGC/HKBU12200820), Hong Kong. Vishal M. Patel was supported by an ARO Grant W911NF-21-1-0135.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Wenjun Kevin Zeng.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shao, R., Perera, P., Yuen, P.C. et al. Open-Set Adversarial Defense with Clean-Adversarial Mutual Learning. Int J Comput Vis 130, 1070–1087 (2022). https://doi.org/10.1007/s11263-022-01581-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-022-01581-0