Automated Identification of Herbarium Specimens at Different Taxonomic Levels

  • Jose Carranza-Rojas
  • Alexis Joly
  • Hervé Goëau
  • Erick Mata-Montero
  • Pierre Bonnet
Part of the Multimedia Systems and Applications book series (MMSA)


The estimated number of flowering plant species on Earth is around 400,000. In order to classify all known species via automated image-based approaches, current datasets of plant images will have to become considerably larger. To achieve this, some authors have explored the possibility of using herbarium sheet images. As the plant datasets grow and start reaching the tens of thousands of classes, unbalanced datasets become a hard problem. This causes models to be inaccurate for certain species due to intra- and inter-specific similarities. Additionally, automatic plant identification is intrinsically hierarchical. In order to tackle this problem of unbalanced datasets, we need ways to classify and calculate the loss of the model by taking into account the taxonomy, for example, by grouping species at higher taxon levels. In this research we compare several architectures for automatic plant identification, taking into account the plant taxonomy to classify not only at the species level, but also at higher levels, such as genus and family.



Special thanks to the Colaboratorio Nacional de Computación Avanzada (CCNA) in Costa Rica for sharing their Tesla K40-based cluster and providing technical support for this research. We also thank the Costa Rica Institute of Technology for the financial support for this research.


  1. 1.
    Matsunaga, A., Thompson, A., Figueiredo, R. J., Germain-Aubrey, C. C., Collins, M., Beaman, R. S., …& Fortes, J. A. (2013, October). A computational-and storage-cloud for integration of biodiversity collections. In eScience (eScience), 2013 IEEE 9th International Conference on (pp. 78–87). IEEE.Google Scholar
  2. 2.
    Page, L. M., MacFadden, B. J., Fortes, J. A., Soltis, P. S., & Riccardi, G. (2015). Digitization of biodiversity collections reveals biggest data on biodiversity. BioScience, 65(9), 841–842.CrossRefGoogle Scholar
  3. 3.
    E. Mata-Montero and J. Carranza-Rojas, Automated Plant Species Identification: Challenges and Opportunities. Springer International Publishing, 2016, pp. 26–36.CrossRefGoogle Scholar
  4. 4.
    J. Carranza-Rojas, H. Goeau, P. Bonnet, E. Mata-Montero, and A. Joly, “Going deeper in the automated identification of herbarium specimens,” BMC Evolutionary Biology, vol. 17, no. 1, p. 181, Aug 2017.Google Scholar
  5. 5.
    D. P. Bebber, M. A. Carine, J. R. Wood, A. H. Wortley, D. J. Harris, G. T. Prance, G. Davidse, J. Paige, T. D. Pennington, N. K. Robson et al., “Herbaria are a major frontier for species discovery,” Proceedings of the National Academy of Sciences, vol. 107, no. 51, pp. 22 169–22 171, 2010.CrossRefGoogle Scholar
  6. 6.
    A. Joly, H. Goëau, H. Glotin, C. Spampinato, P. Bonnet, W.-P. Vellinga, J. Champ, R. Planqué, S. Palazzo, and H. Müller, LifeCLEF 2016: Multimedia Life Species Identification Challenges. Cham: Springer International Publishing, 2016, pp. 286–310.Google Scholar
  7. 7.
    N. Kumar, P. N. Belhumeur, A. Biswas, D. W. Jacobs, W. J. Kress, I. C. Lopez, and J. V. Soares, “Leafsnap: A computer vision system for automatic plant species identification,” in Computer Vision–ECCV 2012. Springer, 2012, pp. 502–516.CrossRefGoogle Scholar
  8. 8.
    A. Joly, P. Bonnet, H. Goëau, J. Barbe, S. Selmi, J. Champ, S. Dufour-Kowalski, A. Affouard, J. Carré, J.-F. Molino et al., “A look inside the pl@ntnet experience,” Multimedia Systems, vol. 22, no. 6, pp. 751–766, 2016.CrossRefGoogle Scholar
  9. 9.
    C. N. Silla and A. A. Freitas, “A survey of hierarchical classification across different application domains,” Data Min Knowl Disc, vol. 22, pp. 31–72, 2011.MathSciNetCrossRefGoogle Scholar
  10. 10.
    F. Wu, J. Zhang, and V. Honavar, “Learning classifiers using hierarchically structured class taxonomies,” in Proceedings of the 6th International Conference on Abstraction, Reformulation and Approximation, ser. SARA’05. Berlin, Heidelberg: Springer-Verlag, 2005, pp. 313–320. [Online]. Available: Google Scholar
  11. 11.
    B. Shahbaba and R. M. Neal, “Improving classification when a class hierarchy is available using a hierarchy-based prior,” Bayesian Anal., vol. 2, no. 1, pp. 221–237, 03 2007. [Online]. Available:
  12. 12.
    Z. Yan, H. Zhang, R. Piramuthu, V. Jagadeesh, D. DeCoste, W. Di, and Y. Yu, “Hd-cnn: Hierarchical deep convolutional neural network for large scale visual recognition,” in ICCV’15: Proc. IEEE 15th International Conf. on Computer Vision, 2015.Google Scholar
  13. 13.
    J. Carranza-Rojas, A. A.J.Joly, P. Bonnet, H.H.G. Goëau, and E. Mata-Montero, “Automated herbarium specimen identification using deep learning,” Biodiversity Information Science and Standards, vol. 1, p. e20302, 2017.CrossRefGoogle Scholar
  14. 14.
    H. Goëau, P. Bonnet, and A. Joly, “LifeCLEF Plant Identification Task 2015,” in CLEF: Conference and Labs of the Evaluation forum, ser. CLEF2015 Working notes, CEUR-WS, Ed., vol. 1391, Toulouse, France, Sep. 2015. [Online]. Available:
  15. 15.
    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 211–252, 2015.MathSciNetCrossRefGoogle Scholar
  16. 16.
    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” CoRR, vol. abs/1502.03167, 2015. [Online]. Available:
  17. 17.
    S. Dieleman, J. Schlüter, C. Raffel, E. Olson, S. K. Sønderby, D. Nouri et al., “Lasagne: First release.” Aug. 2015. [Online]. Available:
  18. 18.
    Theano Development Team, “Theano: A Python framework for fast computation of mathematical expressions,” arXiv e-prints, vol. abs/1605.02688, May 2016. [Online]. Available:
  19. 19.
    C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07-12-June, pp. 1–9, 2015.Google Scholar
  20. 20.
    I. J. Goodfellow, Y. Bulatov, J. Ibarz, S. Arnoud, and V. Shet, “Multi-digit number recognition from street view imagery using deep convolutional neural networks,” 2014. [Online]. Available:

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Jose Carranza-Rojas
    • 1
  • Alexis Joly
    • 2
  • Hervé Goëau
    • 3
    • 4
  • Erick Mata-Montero
    • 1
  • Pierre Bonnet
    • 3
    • 4
  1. 1.School of ComputingCosta Rica Institute of TechnologyCartagoCosta Rica
  2. 2.Inria ZENITH TeamMontpellierFrance
  3. 3.CIRAD, UMR AMAPMontpellierFrance
  4. 4.AMAP, Univ Montpellier, CIRAD, CNRS, INRA, IRDMontpellierFrance

Personalised recommendations