Low-Shot Learning of Plankton Categories
Abstract
The size of current plankton image datasets renders manual classification virtually infeasible. The training of models for machine classification is complicated by the fact that a large number of classes consist of only a few examples. We employ the recently introduced weight imprinting technique in order to use the available training data to train accurate classifiers in absence of enough examples for some classes.
The model architecture used in this work succeeds in the identification of plankton using machine learning with its unique challenges, i.e. a limited number of training examples and a severely skewed class size distribution. Weight imprinting enables a neural network to recognize small classes immediately without re-training. This permits the mining of examples for novel classes.
Notes
Acknowledgements
Rainer Kiko was supported by the Deutsche Forschungsgemeinschaft (DFG) as part of the Collaborative Research Centre (SFB) 754 “Climate-Biogeochemistry Interactions in the Tropical Ocean.” Rainer Kiko, Reinhard Koch and Simon-Martin Schröder were furthermore supported by grants CP1650 and CP1733 of the Cluster of Excellence 80 “The Future Ocean.” “The Future Ocean” is funded within the framework of the Excellence Initiative by the Deutsche Forschungsgemeinschaft (DFG) on behalf of the German federal and state governments. Jean-Olivier Irisson was supported by CNRS LEFE-MANU through project DL-PIC.
References
- 1.Blaschko, M.B., et al.: Automatic in situ identification of plankton. In: 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION 2005), vol. 1, pp. 79–86. IEEE (2005)Google Scholar
- 2.Canziani, A., Paszke, A., Culurciello, E.: An Analysis of Deep Neural Network Models for Practical Applications (2016). http://arxiv.org/abs/1605.07678
- 3.Choe, J., Park, S., Kim, K., Park, J.H., Kim, D., Shim, H.: Face generation for low-shot learning using generative adversarial networks. In: ICCVW 2017, pp. 1940–1948 (2017)Google Scholar
- 4.Christiansen, S., et al.: Particulate matter flux interception in oceanic mesoscale eddies by the polychaete Poeobius sp. Limnol. Oceanogr. 63, 2093–2109 (2018)CrossRefGoogle Scholar
- 5.Chu, B., Madhavan, V., Beijbom, O., Hoffman, J., Darrell, T.: Best practices for fine-tuning visual classifiers to new domains. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 435–442. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_34CrossRefGoogle Scholar
- 6.Cowen, R.K., Guigand, C.M.: In situ ichthyoplankton imaging system (ISIIS): system design and preliminary results. Limnol. Oceanogr.: Methods 6, 126–132 (2008)CrossRefGoogle Scholar
- 7.Culverhouse, P.F., Macleod, N., Williams, R., Benfield, M.C., Lopes, R.M., Picheral, M.: An empirical assessment of the consistency of taxonomic identifications. Marine Biol. Res. 10, 73–84 (2014)CrossRefGoogle Scholar
- 8.Culverhouse, P.F., et al.: Automatic classification of field-collected dinoflagellates by artificial neural network. Marine Ecol. Progress Ser. 139(1/3), 281–287 (1996)CrossRefGoogle Scholar
- 9.Douze, M., Szlam, A., Hariharan, B., Jégou, H.: Low-shot learning with large-scale diffusion (2017). https://arxiv.org/pdf/1706.02332.pdf
- 10.Elineau, A., et al.: ZooScanNet: plankton images captured with the ZooScan (2018). http://doi.org/10.17882/55741
- 11.Ellen, J., Li, H., Ohman, M.D.: Quantifying California current plankton samples with efficient machine learning techniques. In: OCEANS 2015 - MTS/IEEE Washington, pp. 1–9. IEEE (2015)Google Scholar
- 12.Faillettaz, R., Picheral, M., Luo, J.Y., Guigand, C., Cowen, R.K., Irisson, J.O.: Imperfect automatic image classification successfully describes plankton distribution patterns. Methods Oceanogr. 15–16, 60–77 (2016)CrossRefGoogle Scholar
- 13.Gorsky, G.: Digital zooplankton image analysis using the ZooScan integrated system. J. Plankton Res. 32(3), 285–303 (2010)CrossRefGoogle Scholar
- 14.Graham, B., van der Maaten, L.: Submanifold Sparse Convolutional Networks (2017). http://arxiv.org/abs/1706.01307
- 15.Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3037–3046. IEEE (2017)Google Scholar
- 16.Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)Google Scholar
- 17.Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2014). http://arxiv.org/abs/1412.6980
- 18.Lee, H., Park, M., Kim, J.: Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3713–3717. IEEE (2016)Google Scholar
- 19.Luo, C., Zhan, J., Wang, L., Yang, Q.: Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks (2017). http://arxiv.org/abs/1702.05870
- 20.Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: generalizing to new classes at near-zero cost. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2624–2637 (2013)CrossRefGoogle Scholar
- 21.Olson, R.J., Sosik, H.M.: A submersible imaging-in-flow instrument to analyze nano-and microplankton: Imaging FlowCytobot. Limnol. Oceanogr.: Methods 5(6), 195–203 (2007)CrossRefGoogle Scholar
- 22.Orenstein, E.C., Beijbom, O.: Transfer learning and deep feature extraction for planktonic image data sets. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1082–1088. IEEE (2017)Google Scholar
- 23.Paszke, A., et al.: Automatic differentiation in PyTorch. In: Advances in Neural Information Processing Systems (NIPS), vol. 30, pp. 1–4 (2017)Google Scholar
- 24.Picheral, M., Colin, S., Irisson, J.O.: EcoTaxa (2017). http://ecotaxa.obs-vlfr.fr/
- 25.Picheral, M., Guidi, L., Stemmann, L., Karl, D.M., Iddaoud, G., Gorsky, G.: The Underwater Vision Profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton. Limnol. Oceanogr.: Methods 8(1), 462–473 (2010)CrossRefGoogle Scholar
- 26.Py, O., Hong, H., Zhongzhi, S.: Plankton classification with deep convolutional neural networks. In: 2016 IEEE Information Technology, Networking, Electronic and Automation Control Conference, pp. 132–136 (2016)Google Scholar
- 27.Qi, H., Brown, M., Lowe, D.G.: Low-shot learning with imprinted weights. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5822–5830 (2018)Google Scholar
- 28.Sosik, H.M., Olson, R.J.: Automated taxonomic classification of phytoplankton sampled with imaging-in-flow cytometry. Limnol. Oceanogr.: Methods 5(6), 204–216 (2007)CrossRefGoogle Scholar
- 29.Wang, F., Xiang, X., Cheng, J., Yuille, A.L.: NormFace: \(L_2\) hypersphere embedding for face verification. In: Proceedings of the 2017 ACM on Multimedia Conference, MM 2017, pp. 1041–1049. ACM Press, New York (2017)Google Scholar