Abstract
In this paper, we introduce an image dataset for fine-grained classification of dog breeds: the Tsinghua Dogs Dataset. It is currently the largest dataset for fine-grained classification of dogs, including 130 dog breeds and 70,428 real-world images. It has only one dog in each image and provides annotated bounding boxes for the whole body and head. In comparison to previous similar datasets, it contains more breeds and more carefully chosen images for each breed. The diversity within each breed is greater, with between 200 and 7000+ images for each breed. Annotation of the whole body and head makes the dataset not only suitable for the improvement of finegrained image classification models based on overall features, but also for those locating local informative parts. We show that dataset provides a tough challenge by benchmarking several state-of-the-art deep neural models. The dataset is available for academic purposes at https://cg.cs.tsinghua.edu.cn/ThuDogs/.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Cai, S.; Zuo, W.; Zhang, L. Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization. In: Proceedings of the IEEE International Conference on Computer Vision, 511–520, 2017.
Cui, Y.; Song, Y.; Sun, C.; Howard, A.; Belongie, S. J. Large scale fine-grained categorization and domain-specific transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4109–4118, 2018.
Wang, Y.; Morariu, V. I.; Davis, L. S. Learning a discriminative filter bank within a CNN for fine-grained recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4148–4157, 2018.
Yang, Z.; Luo, T. G.; Wang, D.; Hu, Z. Q.; Gao, J.; Wang, L. W. Learning to navigate for fine-grained classification. In: Computer Vision — ECCV 2018. Lecture Notes in Computer Science Vol. 11218. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 438–454, 2018.
Khosla, A.; Jayadevaprakash, N.; Yao, B.; Li, F.-F. Novel dataset for fine-grained image categorization. In: Proceedings of the 1st Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, 2011.
Krizhevsky, A.; Sutskever, I.; Hinton, G. E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, Vol. 1, 1097–1105, 2012.
Chen, L.; Yang, M. Semi-supervised dictionary learning with label propagation for image classification. Computational Visual Media Vol. 3, No. 1, 83–94, 2017.
Chen, K. X.; Wu, X. J. Component SPD matrices: A low-dimensional discriminative data descriptor for image set classification. Computational Visual Media Vol. 4, No. 3, 245–252, 2018.
Ren, J. Y.; Wu, X. J. Vectorial approximations of infinite-dimensional covariance descriptors for image classification. Computational Visual Media Vol. 3, No. 4, 379–385, 2017.
Wah, C.; Branson, S.; Welinder, P.; Perona, P.; Belongie, S. The Caltech-UCSD Birds-200–2011 Dataset. Computation & Neural Systems Technical Report, CNS-TR-2011-001. California Institute of Technology, 2011.
Liu, J.; Kanazawa, A.; Jacobs, D.; Belhumeur, P. Dog breed classification using part localization. In: Proceedings of the 12th European Conference on Computer Vision, Vol. Part I, 172–185, 2012.
Berg, T.; Belhumeur, P. N. POOF: Part-based one-vs.-one features for fine-grained categorization, face verification, and attribute estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 955–962, 2013.
Branson, S.; Horn, G. V.; Belongie, S.; Perona, P. Bird species categorization using pose normalized deep convolutional nets. arXiv preprint arXiv:1406.2952, 2014.
Zhang, N.; Donahue, J.; Girshick, R.; Darrell, T. Part-based R-CNNs for fine-grained category detection. In: Computer Vision-ECCV 2014. Lecture Notes in Computer Science Vol. 8689. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 834–849, 2014.
Lin, D.; Shen, X.; Lu, C.; Jia, J. Deep LAC: Deep localization, alignment and classification for fine-grained recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1666–1674, 2015.
Lam, M.; Mahasseni, B.; Todorovic, S. Fine-grained recognition as HSnet search for informative image parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6497–6506, 2017.
Chen, Y.; Bai, Y.; Zhang, W.; Mei, T. Destruction and construction learning for finegrained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5157–5166, 2019.
Ge, W. F.; Lin, X. R.; Yu, Y. Z. Weakly supervised complementary parts models for fine-grained image classification from the bottom up. arXiv preprint arXiv:1903.02827, 2019.
Du, R. Y.; Chang, D. L.; Bhunia, A. K.; Xie, J. Y.; Ma, Z. Y.; Song, Y. Z.; Guo, J. Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. arXiv preprint arXiv:2003.03836, 2020.
Zheng, H.; Fu, J.; Mei, T.; Luo, J. Learning multi-attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 5219–5227, 2017.
Fu, J.; Zheng, H.; Mei, T. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4476–4484, 2017.
Zheng, H.; Fu, J.; Zha, Z.; Luo, J.; Looking for the devil in the details: Learning trilinear attention sampling network for fine-grained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5012–5021, 2019.
Zhang, F.; Li, M.; Zhai, G.; Liu, Y. Three-branch and multi-scale learning for fine-grained image recognition (TBMSL-Net). arXiv preprint arXiv:2003.09150, 2020.
Sun, G. L.; Cholakkal, H.; Khan, S.; Khan, F. S.; Shao, L. Fine-grained recognition: Accounting for subtle differences between similar classes. arXiv preprint arXiv:1912.06842, 2019.
Lin, T.-Y.; RoyChowdhury, A.; Maji, S. Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision, 1449–1457, 2015.
Gao, Y.; Beijbom, O.; Zhang, N.; Darrell, T. Compact bilinear pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 317–326, 2016.
Yu, C.; Zhao, X.; Zheng, Q.; Zhang, P.; You, X. Hierarchical bilinear pooling for fine-grained visual recognition. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11220. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 595–610, 2018.
Wang, Y.; Choi, J.; Morariu, V. I.; Davis, L. S. Mining discriminative triplets of patches for fine-grained classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1163–1172, 2016.
Zhang, X.; Zhou, F.; Lin, Y.; Zhang, S. Embedding label structures for finegrained feature representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1114–1123, 2016.
Dubey, A.; Gupta, O.; Raskar, R.; Naik, N. Maximum-entropy fine grained classification. arXiv preprint arXiv:1809.05934, 2018.
Qian, Q.; Jin, R.; Zhu, S.; Lin, Y. Fine-grained visual categorization via multi-stage metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3716–3724, 2015.
Sun, M.; Yuan, Y.; Zhou, F.; Ding, E. Multi-attention multi-class constraint for fine-grained image recognition. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11220. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 834–850, 2018.
Dubey, A.; Gupta, O.; Guo, P.; Raskar, R.; Farrell, R.; Naik, N. Pairwise confusion for fine-grained visual classification. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11216. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 71–88, 2018.
Zhuang, P.; Wang, Y.; Qiao, Y. Learning attentive pairwise interaction for fine-grained classification. arXiv preprint arXiv:2002.10191, 2020.
Xu, Z.; Huang, S.; Zhang, Y.; Tao, D. Augmenting strong supervision using web data for finegrained categorization. In: Proceedings of the IEEE International Conference on Computer Vision, 2524–2532, 2015.
Niu, L.; Veeraraghavan, A.; Sabharwal, A. Finegrained classification using heterogeneous web data and auxiliary categories. arXiv preprint arXiv:1811.07567, 2018.
Torralba, A.; Efros, A. A. Unbiased look at dataset bias. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1521–1528, 2011.
Hu, T.; Qi, H. G.; Huang, Q. M.; Lu, Y. See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification. arXiv preprint arXiv:1901.09891, 2019.
Krause, J.; Stark, M.; Deng, J.; L. Fei-Fei. 3D object representations for fine-grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 554–561, 2013.
Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; Vedaldi, A. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.
Nilsback, M.; Zisserman, A. Automated flower classification over a large number of classes. In: Proceedings of the 6th Indian Conference on Computer Vision, Graphics & Image Processing, 722–729, 2008.
Deng, J.; Dong, W.; Socher, R.; Li, L.; Li, K.; Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 248–255, 2009.
Everingham, M.; van Gool, L.; Williams, C. K. I.; Winn, J.; Zisserman, A. The pascal visual object classes (VOC) challenge. International Journal of Computer-Vision Vol. 88, No. 2, 303–338, 2010.
Lin, T.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C. L.; Dollár, P. Microsoft COCO: Common objects in context. arXiv preprint arXiv:1405.0312, 2014.
Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004.
Russell, B. C.; Torralba, A.; Murphy, K. P.; Freeman, W. T. LabelMe: A database and web-based tool for image annotation. International Journal of Computer-Vision Vol. 77, Nos. 1–3, 157–173, 2008.
Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K. Q. Densely connected convolutional networks. arXiv preprint arXiv:1608.06993, 2016.
Acknowledgements
The authors would like to thank Wei-Yu Xie for his assistance on paper writing, and also thank Qiu Xin and Zhi-Ping Zhang for much help on image processing and labeling. This work was supported by the National Natural Science Foundation of China (Project Nos. 61521002 and 61772298), a Research Grant of Beijing Higher Institution Engineering Research Center, and Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.
Author information
Authors and Affiliations
Corresponding author
Additional information
Ding-Nan Zou is a master candidate in the Department of Computer Science and Technology at Tsinghua University, Beijing. His research interests include computer graphics and computer vision, especially dog face and iris recognition.
Song-Hai Zhang received his Ph.D. degree in computer science and technology from Tsinghua University, Beijing, in 2007. He is currently an associate professor in the Department of Computer Science and Technology at Tsinghua University. His research interests include image and video analysis and processing as well as geometric computing.
Tai-Jiang Mu is currently an assistant researcher in the Department of Computer Science and Technology, Tsinghua University, Beijing, where he received his bachelor and doctor degrees in computer science and technology in 2011 and 2016, respectively. His research interests include visual media learning, SLAM, and human robot interaction.
Min Zhang is a researcher in Harvard Medical School, Brigham and Women’s Hospital. She received her Ph.D. degree in computer science from Stony Brook University and the other Ph.D. degree in mathematics from Zhejiang University. She is an expert in the fields of geometric modeling, medical imaging, graphics, visualization, machine learning, 3D technologies, etc.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Zou, DN., Zhang, SH., Mu, TJ. et al. A new dataset of dog breed images and a benchmark for finegrained classification. Comp. Visual Media 6, 477–487 (2020). https://doi.org/10.1007/s41095-020-0184-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-020-0184-6