Searching the World’s Herbaria: A System for Visual Identification of Plant Species

  • Peter N. Belhumeur
  • Daozheng Chen
  • Steven Feiner
  • David W. Jacobs
  • W. John Kress
  • Haibin Ling
  • Ida Lopez
  • Ravi Ramamoorthi
  • Sameer Sheorey
  • Sean White
  • Ling Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)

Abstract

We describe a working computer vision system that aids in the identification of plant species. A user photographs an isolated leaf on a blank background, and the system extracts the leaf shape and matches it to the shape of leaves of known species. In a few seconds, the system displays the top matching species, along with textual descriptions and additional images. This system is currently in use by botanists at the Smithsonian Institution National Museum of Natural History. The primary contributions of this paper are: a description of a working computer vision system and its user interface for an important new application area; the introduction of three new datasets containing thousands of single leaf images, each labeled by species and verified by botanists at the US National Herbarium; recognition results for two of the three leaf datasets; and descriptions throughout of practical lessons learned in constructing this system.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abbasi, S., Mokhtarian, F., Kittler, J.: Reliable classification of chrysanthemum leaves through curvature scale space. In: ter Haar Romeny, B.M., Florack, L.M.J., Viergever, M.A. (eds.) Scale-Space 1997. LNCS, vol. 1252, pp. 284–295. Springer, Heidelberg (1997)Google Scholar
  2. 2.
    Agarwal, G., Belhumeur, P., Feiner, S., Jacobs, D., Kress, W.J., Ramamoorthi, R., Bourg, N., Dixit, N., Ling, H., Mahajan, D., Russell, R., Shirdhonkar, S., Sunkavalli, K., White, S.: First steps towards an electronic field guide for plants. Taxon 55, 597–610 (2006)Google Scholar
  3. 3.
    Bederson, B.: PhotoMesa: A zoomable image browser using quantum treemaps and bubblemaps. In: Proc. ACM UIST 2001, pp. 71–80 (2001)Google Scholar
  4. 4.
    Bederson, B., Grosjean, J., Meyer, J.: Toolkit design for interactive structured graphics. IEEE Trans. on Soft. Eng. 30(8), 535–546 (2004)CrossRefGoogle Scholar
  5. 5.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Context. IEEE Trans. on Patt. Anal. and Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  6. 6.
    Edwards, M., Morse, D.R.: The potential for computer-aided identification in biodiversity research. Trends in Ecology and Evolution 10, 153–158 (1995)CrossRefGoogle Scholar
  7. 7.
    Fiala, M.: ARTag, a fiducial marker system using digital techniques. In: Proc. CVPR 2005, pp. 590–596 (2005)Google Scholar
  8. 8.
    Felzenszwalb, P., Schwartz, J.: Hierarchical matching of deformable shapes. In: Proc. CVPR 2007, pp. 1–8 (2007)Google Scholar
  9. 9.
    Forsyth, D., Ponce, J.: Computer vision: A modern approach. Prentice Hall, Upper Saddle River (2003)Google Scholar
  10. 10.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: Proc. CVPR, pp. 716–723 (2003)Google Scholar
  11. 11.
    Heidorn, P.B.: A tool for multipurpose use of online flora and fauna: The Biological Information Browsing Environment (BIBE). First Monday 6(2) (2001), http://firstmonday.org/issues/issue6_2/heidorn/index.html
  12. 12.
    Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., Tachibana, K.: Virtual object manipulation of a table-top AR environment. In: Proc. IEEE and ACM ISAR, pp. 111–119 (2000)Google Scholar
  13. 13.
    Ling, H., Jacobs, D.: Shape Classification Using the Inner-Distance. IEEE Trans. on Patt. Anal. and Mach. Intell. 29(2), 286–299 (2007)CrossRefGoogle Scholar
  14. 14.
    Mokhtarian, F., Abbasi, S.: Matching shapes with self-intersections: Application to leaf classification. Proc. IEEE Trans. on Image 13(5), 653–661 (2004)CrossRefGoogle Scholar
  15. 15.
    Nilsback, M., Zisserman, A.: A visual vocabulary for flower classification. In: Proc. CVPR, pp. 1447–1454 (2006)Google Scholar
  16. 16.
    Pankhurst, R.J.: Practical taxonomic computing. Cambridge University Press, Cambridge (1991)Google Scholar
  17. 17.
    Ruiz, E.: An algorithm for finding nearest neighbours in (approximately) constant average time. Patt. Rec. Lett. 4(3), 145–157 (1986)CrossRefGoogle Scholar
  18. 18.
    Saitoh, T., Kaneko, T.: Automatic recognition of wild flowers. Proc. ICPR 2, 2507–2510 (2000)Google Scholar
  19. 19.
    Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Trans. on Patt. Anal. and Mach. Intell. 22(8), 888–905 (2000)CrossRefGoogle Scholar
  20. 20.
    Söderkvist, O.: Computer vision classification of leaves from Swedish trees. Master Thesis, Linköping Univ. (2001)Google Scholar
  21. 21.
    Stevenson, R.D., Haber, W.A., Morris, R.A.: Electronic field guides and user communities in the eco-informatics revolution. Conservation Ecology 7(3) (2003), http://www.consecol.org/vol7/iss1/art3
  22. 22.
    Vidal, E.: New formulation and improvements of the nearest-neighbour approximating and eliminating search algorithm (AESA). Patt. Rec. Lett. 15(1), 1–7 (1994)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Wang, Z., Chi, W., Feng, D.: Shape based leaf image retrieval. IEE Proc. Vision, Image and Signal Processing 150(1), 34–43 (2003)CrossRefGoogle Scholar
  24. 24.
    White, S., Feiner, S., Kopylec, J.: Virtual vouchers: Prototyping a mobile augmented reality user interface for botanical species identification. In: Proc. IEEE Symp. on 3DUI, pp. 119–126 (2006)Google Scholar
  25. 25.
    White, S., Marino, D., Feiner, S.: Designing a mobile user interface for automated species identification. In: Proc. CHI 2007, pp. 291–294 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Peter N. Belhumeur
    • 1
  • Daozheng Chen
    • 2
  • Steven Feiner
    • 1
  • David W. Jacobs
    • 2
  • W. John Kress
    • 3
  • Haibin Ling
    • 4
  • Ida Lopez
    • 3
  • Ravi Ramamoorthi
    • 1
  • Sameer Sheorey
    • 2
  • Sean White
    • 1
  • Ling Zhang
    • 3
  1. 1.Dept. Of Computer ScienceColumbia UniversityNew York NY 
  2. 2.Dept. Of Computer ScienceUniversity of MarylandCollege Park MD 
  3. 3.Department Of Botany, National Museum of Natural History, Smithsonian InstitutionWashington, DC 
  4. 4.Information Science and Technology CenterTemple UniversityPhiladelphia PA 

Personalised recommendations