Leafsnap: A Computer Vision System for Automatic Plant Species Identification

  • Neeraj Kumar
  • Peter N. Belhumeur
  • Arijit Biswas
  • David W. Jacobs
  • W. John Kress
  • Ida C. Lopez
  • João V. B. Soares
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


We describe the first mobile app for identifying plant species using automatic visual recognition. The system – called Leafsnap – identifies tree species from photographs of their leaves. Key to this system are computer vision components for discarding non-leaf images, segmenting the leaf from an untextured background, extracting features representing the curvature of the leaf’s contour over multiple scales, and identifying the species from a dataset of the 184 trees in the Northeastern United States. Our system obtains state-of-the-art performance on the real-world images from the new Leafsnap Dataset – the largest of its kind. Throughout the paper, we document many of the practical steps needed to produce a computer vision system such as ours, which currently has nearly a million users.


Input Image Contour Point Leaf Image Computer Vision System Histogram Intersection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 145–175 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Branson, S., Wah, C., Schroff, F., Babenko, B., Welinder, P., Perona, P., Belongie, S.: Visual Recognition with Humans in the Loop. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 438–451. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Nilsback, M., Zisserman, A.: Automated flower classification over a large number of classes. In: Indian Conference on Computer Vision, Graphics and Image Processing (2008)Google Scholar
  4. 4.
    Goëau, H., Bonnet, P., Joly, A., Boujemaa, N., Barthelemy, D., Molino, J.F., Birnbaum, P., Mouysset, E., Picard, M.: The clef 2011 plant images classification task. In: CLEF (Notebook Papers/Labs/Workshop) (2011)Google Scholar
  5. 5.
    Belhumeur, P.N., Chen, D., Feiner, S.K., Jacobs, D.W., Kress, W.J., Ling, H., Lopez, I., Ramamoorthi, R., Sheorey, S., White, S., Zhang, L.: Searching the World’s Herbaria: A System for Visual Identification of Plant Species. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 116–129. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Agarwal, G., Belhumeur, P., Feiner, S., Jacobs, D., Kress, W.J., Ramamoorthi, R., Bourg, N.A., Dixit, N., Ling, H., Mahajan, D., Russell, R., Shirdhonkar, S., Sunkavalli, K., White, S.: First steps toward an electronic field guide for plants. Taxon 55, 597–610 (2006)CrossRefGoogle Scholar
  7. 7.
    Ling, H., Jacobs, D.: Shape classification using the inner-distance. TPAMI 29, 286–299 (2007)CrossRefGoogle Scholar
  8. 8.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20 (1995)Google Scholar
  9. 9.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011)Google Scholar
  10. 10.
    Douze, M., Jégou, H., Sandhawalia, H., Amsaleg, L., Schmid, C.: Evaluation of gist descriptors for web-scale image search. In: International Conference on Image and Video Retrieval (2009)Google Scholar
  11. 11.
    Efron, B.: The efficiency of logistic regression compared to normal discriminant analysis. Journal of the American Statistical Association 70, 892–898 (1975)zbMATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    Gonzalez, R., Woods, R.: Digital image processing. Pearson/Prentice Hall (2008)Google Scholar
  13. 13.
    Connolly, M.L.: Measurement of protein surface shape by solid angles. Journal of Molecular Graphics 4, 3–6 (1986)CrossRefGoogle Scholar
  14. 14.
    Manay, S., Cremers, D., Hong, B.W., Yezzi, A.J., Soatto, S.: Integral invariants for shape matching. TPAMI 28, 1602–1618 (2006)CrossRefGoogle Scholar
  15. 15.
    Pottmann, H., Wallner, J., Huang, Q.X., Yang, Y.L.: Integral invariants for robust geometry processing. Computer Aided Geometric Design 26, 37–60 (2009)zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Neeraj Kumar
    • 1
  • Peter N. Belhumeur
    • 2
  • Arijit Biswas
    • 3
  • David W. Jacobs
    • 3
  • W. John Kress
    • 4
  • Ida C. Lopez
    • 4
  • João V. B. Soares
    • 3
  1. 1.University of WashingtonSeattleUSA
  2. 2.Columbia UniversityNew YorkUSA
  3. 3.University of MarylandCollege ParkUSA
  4. 4.National Museum of Natural History, Smithsonian InstitutionWashingtonUSA

Personalised recommendations