Abstract
Non-I.I.D. image classification is an important research topic for both academic and industrial communities. However, it is a very challenging task, as it violates the famous hypothesis of independent and identically distributed (I.I.D.) in conventional machine learning, and the classifier minimizing empirical errors on training images does not perform well on testing images. In this work, we propose a novel model called Contrastive Learning in Frequency Domain (CLFD) to learn invariant representations for Non-I.I.D. image classification. In CLFD, model learning includes two steps: contrastive learning in the frequency domain for pre-training, and image classification with fine-tuning. In the first pre-training step, anchor, positive and negative images are transformed by Discrete Cosine Transform (DCT) and then projected into vector space. This step is to obtain stable invariant features by minimizing the contrastive loss. In the step of image classification with fine-tuning, the features from ResNet are mapped into the label space by a simple fully connected layer, and the classification loss is utilized to fine-tune the parameters in the ResNet. Extensive experiments conducted on public NICO dataset demonstrate the effectiveness of the proposed CLFD, which outperforms the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, N., Natarajan, T., Rao, K.R.: Discrete cosine transform. IEEE Trans. Comput. C-23(1), 90–93 (1974)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709 (2020)
Ehrlich, M., Davis, L.S.: Deep residual learning in the jpeg transform domain. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 1180–1189. PMLR, Lille, France, 07–09 Jul 2015
Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D.: Domain generalization for object recognition with multi-task autoencoders. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015
Gueguen, L., Sergeev, A., Kadlec, B., Liu, R., Yosinski, J.: Faster neural networks straight from JPEG. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 3933–3944. Curran Associates, Inc. (2018)
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1735–1742 (2006)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
He, Y., Shen, Z., Cui, P.: Towards non-I.I.D. image classification: a dataset and baselines. Pattern Recogn. 110, 07383 (2020)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)
Li, D., Yang, Y., Song, Y.Z., Hospedales, T.M.: Deeper, broader and artier domain generalization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), October 2017
Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: International conference on machine learning, pp. 2208–2217 (2017)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv, arXiv-1807 (2018)
Shen, Z., Cui, P., Kuang, K., Li, B., Chen, P.: Causally regularized learning with agnostic data selection bias. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 411–419, MM 2018. Association for Computing Machinery, New York (2018)
Shen, Z., Cui, P., Zhang, T., Kuang, K.: Stable learning via sample reweighting. In: AAAI, pp. 5692–5699 (2020)
Torfason, R., Mentzer, F., Ágústsson, E., Tschannen, M., Timofte, R., Gool, L.V.: Towards image understanding from deep compression without decoding. In: International Conference on Learning Representations (2018)
Watson, A.B.: Image compression using the discrete cosine transform. Math. J. 4(1), 81 (1994)
Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Xu, K., Qin, M., Sun, F., Wang, Y., Chen, Y.K., Ren, F.: Learning in the frequency domain. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China under Grants 61802053 and 61772436, the Sichuan Science and Technology Program under Grant 2020YJ0037 and 2020YJ0207, the Foundation for Department of Transportation of Henan Province under Grant 2019J-2-2, and the Fundamental Research Funds for the Central Universities under Grant 2682019CX62.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Shao, H., Yuan, Z., Peng, X., Wu, X. (2021). Contrastive Learning in Frequency Domain for Non-I.I.D. Image Classification. In: Lokoč, J., et al. MultiMedia Modeling. MMM 2021. Lecture Notes in Computer Science(), vol 12572. Springer, Cham. https://doi.org/10.1007/978-3-030-67832-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-67832-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67831-9
Online ISBN: 978-3-030-67832-6
eBook Packages: Computer ScienceComputer Science (R0)