Museum Exhibit Identification Challenge for the Supervised Domain Adaptation and Beyond

  • Piotr KoniuszEmail author
  • Yusuf TasEmail author
  • Hongguang Zhang
  • Mehrtash Harandi
  • Fatih Porikli
  • Rui Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11220)


We study an open problem of artwork identification and propose a new dataset dubbed Open Museum Identification Challenge (Open MIC). It contains photos of exhibits captured in 10 distinct exhibition spaces of several museums which showcase paintings, timepieces, sculptures, glassware, relics, science exhibits , natural history pieces, ceramics, pottery, tools and indigenosus crafts. The goal of Open MIC is to stimulate research in domain adaptation, egocentric recognition and few-shot learning by providing a testbed complementary to the famous Office dataset which reaches \(\sim \)90% accuracy. To form our dataset, we captured a number of images per art piece with a mobile phone and wearable cameras to form the source and target data splits, respectively. To achieve robust baselines, we build on a recent approach that aligns per-class scatter matrices of the source and target CNN streams. Moreover, we exploit the positive definite nature of such representations by using end-to-end Bregman divergences and the Riemannian metric. We present baselines such as training/evaluation per exhibition and training/evaluation on the combined set covering 866 exhibit identities. As each exhibition poses distinct challenges e.g., quality of lighting, motion blur, occlusions, clutter, viewpoint and scale variations, rotations, glares, transparency, non-planarity, clipping, we break down results w.r.t. these factors.



Big thanks go to Ondrej Hlinka and (Tim) Ka Ho from the Scientific Computing Services at CSIRO for their can-do attitude and help with Bracewell.

Supplementary material

474218_1_En_48_MOESM1_ESM.pdf (1.3 mb)
Supplementary material 1 (pdf 1349 KB)


  1. 1.
    Baxter, J., Caruana, R., Mitchell, T., Pratt, L.Y., Silver, D.L., Thrun, S.: Learning to learn: knowledge consolidation and transfer in inductive systems. In: NIPS Workshop. (1995). Accessed 30 Oct 2016
  2. 2.
    Li, W., et al.: Task-CV: transferring and adapting source knowledge in computer vision. ECCV Workshop. (2016). Accessed 22 Nov 2016
  3. 3.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Tommasi, T., Orabona, F., Caputo, B.: Safety in numbers: learning categories from few examples with multi model knowledge transfer. In: CVPR, pp. 3081–3088 (2010)Google Scholar
  5. 5.
    Koniusz, P., Tas, Y., Porikli, F.: Domain adaptation by mixture of alignments of second- or higher-order scatter tensors. In: CVPR, p. 2 (2017)Google Scholar
  6. 6.
    Chopra, S., Balakrishnan, S., Gopalan, R.: Dlid: Deep learning for domain adaptation by interpolating between domains. In: ICML Workshop (2013)Google Scholar
  7. 7.
    Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: ICCV, pp. 4068–4076 (2015)Google Scholar
  8. 8.
    Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. In: CoRR arXiv:abs/1511.05547 (2015)
  9. 9.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. JMLR 17(1), 2030–2096 (2016)Google Scholar
  10. 10.
    Daumé, III, H., Kumar, A., Saha, A.: Frustratingly easy semi-supervised domain adaptation. In: Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, pp. 53–59 (2010)Google Scholar
  11. 11.
    Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. TPAMI 28, 594–611 (2006)Google Scholar
  12. 12.
    Herath, S., Harandi, M., Porikli, F.: Learning an invariant hilbert space for domain adaptation. In: CVPR (2017)Google Scholar
  13. 13.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)Google Scholar
  14. 14.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR arXiv:abs/1409.1556 (2015)
  15. 15.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: ECCV, pp. 213–226 (2010)Google Scholar
  16. 16.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR, pp. 580–587 (2014)Google Scholar
  17. 17.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., Lecun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. In: ICLR (2014)Google Scholar
  18. 18.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: CVPR pp. 2066–2073 (2012)Google Scholar
  19. 19.
    Cherian, A., Sra, S., Banerjee, A., Papanikolopoulos, N.: Jensen-Bregman LogDet divergence with application to efficient similarity search for covariance matrices. TPAMI 35(9), 2161–2174 (2013)CrossRefGoogle Scholar
  20. 20.
    Pennec, X., Fillard, P., Ayache, N.: A riemannian framework for tensor computing. IJCV 66(1), 41–66 (2006)CrossRefGoogle Scholar
  21. 21.
    Bhatia, R.: Positive Definite Matrices. Princeton University Press, Princeton (2007)Google Scholar
  22. 22.
    Wang, Y.X., Hebert, M.: Learning to learn: model regression networks for easy small sample learning. In: ECCV (2016)Google Scholar
  23. 23.
    Kuzborskij, I., Carlucci, F.M., Caputo, B.: When naïve bayes nearest neighbors meet convolutional neural networks. In: CVPR (2016)Google Scholar
  24. 24.
    Tommasi, T., Lanzi, M., Russo, P., Caputo, B.: Learning the roots of visual domain shift. In: ECCV Workshop (2016)Google Scholar
  25. 25.
    Tommasi, T., Tuytelaars, T., Caputo, B.: A testbed for cross-dataset analysis. Technical Report (2014)Google Scholar
  26. 26.
    Rebuffi, S.A., Bilen, H., Vedaldi, A.: Learning multiple visual domains with residual adapters. Part of the PASCAL in Detail Workshop Challenge. (2017). Accessed 30 Oct 2017
  27. 27.
    Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: CVPR (2017)Google Scholar
  28. 28.
    Gebru, T., Krause, J., Wang, Y., Chen, D., Deng, J., Fei-Fei, L.: Fine-grained car detection for visual census estimation. In: AAAI (2017)Google Scholar
  29. 29.
    Gebru, T., Hoffman, J., Fei-Fei, L.: Fine-grained recognition in the wild: a multi-task domain adaptation approach. In: ICCV (2017)Google Scholar
  30. 30.
    Rajapakse, J.C., Wang, L.: Neural Information Processing: Research and Development. Springer-Verlag, Berlin and GmbH & Co. KG, Heidelberg (2004)Google Scholar
  31. 31.
    Koniusz, P., Yan, F., Gosselin, P.H., Mikolajczyk, K.: Higher-order occurrence pooling for bags-of-words: visual concept detection. TPAMI 39(2), 313–326 (2017)CrossRefGoogle Scholar
  32. 32.
    Koniusz, P., Zhang, H., Porikli, F.: A deeper look at power normalizations. In: CVPR, pp. 5774–5783 (2018)Google Scholar
  33. 33.
    Yeh, Y.R., Huang, C.H., Wang, Y.C.F.: Heterogeneous domain adaptation and classification by exploiting the correlation subspace. Trans. Image Process. 23(5), 2009–2018 (2014)Google Scholar
  34. 34.
    Kumar Roy, S., Mhammedi, Z., Harandi, M.: Geometry aware constrained optimization techniques for deep learning. In: CVPR (2018)Google Scholar
  35. 35.
    Harandi, M., Salzmann, M., Hartley, R.: Joint dimensionality reduction and metric learning: a geometric take. ICML 70, 1404–1413 (2017)Google Scholar
  36. 36.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  37. 37.
    Bo, L., Sminchisescu, C.: Efficient match kernels between sets of features for visual recognition. In: NIPS (2009)Google Scholar
  38. 38.
    Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition. In: ICML (2014)Google Scholar
  39. 39.
    Ghifary, M., Kleijn, W.B., Zhang., M.: Domain adaptive neural networks for object recognition. In: CoRR arXiv:abs/1409.6041 (2014)Google Scholar
  40. 40.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  41. 41.
    Zhang, R., Tas, Y., Koniusz, P.: Artwork identification from wearable camera images for enhancing experience of museum audiences. In: Museums and the Web (2017)Google Scholar
  42. 42.
    Long, M., Wang, J., Jordan, M.I.: Unsupervised domain adaptation with residual transfer networks. In: CoRR arXiv:abs/1602.04433 (2016)
  43. 43.
    Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: ICML (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Data61/CSIROCanberraAustralia
  2. 2.Australian National UniversityCanberraAustralia
  3. 3.Monash UniversityMelbourneAustralia
  4. 4.Hubei University of Arts and ScienceXiangyangChina

Personalised recommendations