Skip to main content
SpringerLink
Log in

Efficient segmentation of leaves in semi-controlled conditions

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We present a study on segmentation of leaf images restricted to semi-controlled conditions, in which leaves are photographed against a solid light-colored background. Such images can be used in practice for plant species identification, by analyzing the distinctive shapes of the leaves. We restrict our attention to segmentation in this semi-controlled condition, providing us with a more well-defined problem, which at the same time presents several challenges. The most important of these are: the variety of leaf shapes, inevitable presence of shadows and specularities, and the time constraints required by interactive species identification applications. We evaluate several popular segmentation algorithms on this task. Different datasets of leaf images are used, with manually segmented images serving as ground truth for quantitative comparisons. We observe that many of the methods are not immediately applicable: they are either too slow or would require that important modifications be introduced. We thus present extensions to our previously published segmentation method, which are able to improve its performance. The previous approach was based on pixel clustering in color space. Our extensions introduce a graph cut step and the use of a training set of manual segmentations in order to adjust important parameters of the method. The new method is fast enough for an interactive application, while producing state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. GBIS is an exception, since even at the smallest observation scale (\(k = 1\)), it still does not produce a fine enough segmentation.

References

  1. Agarwal, G., Belhumeur, P., Feiner, S., Jacobs, D., Kress, W.J., Ramamoorthi, R., Bourg, N.A., Dixit, N., Ling, H., Mahajan, D., Russell, R., Shirdhonkar, S., Sunkavalli, K., White, S.: First steps toward an electronic field guide for plants. Taxon 55(3), 597–610 (2006)

    Article  Google Scholar 

  2. Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  3. Arbelaez, P., Fowlkes, C., Martin, D., Malik, J.: Berkeley segmentation and boundary detection benchmark and dataset (Accessed 2011). http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html

  4. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  5. Arora, A., Gupta, A., Bagma, N., Mishra, S., Bhattacharya, A.: A plant identification system using shape and morphological features on segmented leaflets. In: Working Notes of CLEF 2012 Conference (2012)

  6. Bakic, V., Yahiaoui, I., Mouine, S., Litayem, S., Ouertani, W., Verroust-Blondet, A., Goëau, H., Joly, A.: Inria IMEDIA2’s participation at ImageCLEF 2012 plant identification task. In: Working Notes of CLEF 2012 Conference (2012)

  7. Belhumeur, P., Chen, D., Feiner, S., Jacobs, D., Kress, W., Ling, H., Lopez, I., Ramamoorthi, R., Sheorey, S., White, S., Zhang, L.: Searching the world’s herbaria: A system for visual identification of plant species. In: European Conference on Computer Vision, pp. 116–129 (2008)

  8. Bishop, C.M.: In: Pattern recognition and machine learning. Information Science and Statistics. Springer, New York (2006)

  9. Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: International Conference on Computer Vision, vol. 1, pp. 105–112 (2001)

  10. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Google Scholar 

  11. Camargo Neto, J., Meyer, G.E., Jones, D.D.: Individual leaf extractions from young canopy images using Gustafson-Kessel clustering and a genetic algorithm. Comput. Electron. Agric. 51(1), 66–85 (2006)

    Article  Google Scholar 

  12. Casanova, D., Florindo, J.B., Gonçalves, W.N., Bruno, O.M.: IFSC/USP at ImageCLEF 2012: Plant identification task. In: Working notes of CLEF 2012 Conference (2012)

  13. Cerutti, G., Antoine, V., Tougne, L., Mille, J., Coquin, D., Vacavant, A.: ReVeS Participation—Tree species classification using random forests and botanical features. In: Working Notes of CLEF 2012 Conference (2012)

  14. Chai, Y., Lempitsky, V., Zisserman, A.: BiCoS: A bi-level co-segmentation method for image classification. In: International Conference on Computer Vision (2011)

  15. Christoudias, C., Georgescu, B., Meer, P.: Synergism in low-level vision. In: International Conference on Pattern Recognition, vol. 4, pp. 150–155 (2002)

  16. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  17. Cour, T., Benezit, F., Shi, J.: Spectral segmentation with multiscale graph decomposition. In: Computer Vision and Pattern Recognition, pp. 1124–1131 (2005)

  18. Dixon, W.J., Mood, A.M.: The statistical sign test. J. Am. Stat. Assoc. 41(236), 557–566 (1946)

    Article  Google Scholar 

  19. Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 167–181 (2004)

    Article  Google Scholar 

  20. Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic press, San Diego (1990)

    MATH  Google Scholar 

  21. Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: Computer Vision and Pattern Recognition (CVPR), pp. 716–723 (2003)

  22. Goëau, H., Bonnet, P., Joly, A., Yahiaoui, I., Barthélémy, D., Boujemaa, N., Molino, J.: The ImageCLEF 2012 plant identification task. In: Working Notes of CLEF 2012 Conference (2012)

  23. Heckemann, R.A., Hajnal, J.V., Aljabar, P., Rueckert, D., Hammers, A.: Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage 33(1), 115–126 (2006)

    Article  Google Scholar 

  24. Kohli, P., Ladick, L., Torr, P.H.S.: Robust higher order potentials for enforcing label consistency. Int. J. Comput. Vis. 82(3), 302–324 (2009)

    Article  Google Scholar 

  25. Kumar, M., Torr, P., Zisserman, A.: Obj Cut. In: Computer Vision and Pattern Recognition, vol. 1, pp. 18–25 (2005)

  26. Kumar, N., Belhumeur, P.N., Biswas, A., Jacobs, D.W., Kress, W.J., Lopez, I.C., Soares, J.V.B.: Leafsnap: a computer vision system for automatic plant species identification. In: European Conference on Computer Vision, pp. 502–516 (2012)

  27. Kumar, S., Hebert, M.: Discriminative random fields. Int. J. Comput. Vis. 68(2), 179–201 (2006)

    Article  Google Scholar 

  28. Leung, T., Malik, J.: Contour continuity in region based image segmentation. In: European Conference on Computer Vision, pp. 544–559. Springer, Berlin (1998)

  29. Manh, A.G., Rabatel, G., Assemat, L., Aldon, M.J.: Weed leaf image segmentation by deformable templates. J. Agric. Eng. Res. 80(2), 139–146 (2001)

    Article  Google Scholar 

  30. Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)

    Article  Google Scholar 

  31. Nilsback, M.E., Zisserman, A.: Delving into the whorl of flower segmentation. In: BMVC (2007)

  32. Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Indian Conference on Computer Vision, Graphics and Image Processing (2008)

  33. Open source computer vision library (OpenCV). http://opencv.org/

  34. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Google Scholar 

  35. Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: Interactive foreground extraction using iterated graph cuts. In: SIGGRAPH, pp. 309–314 (2004)

  36. Sharon, E., Galun, M., Sharon, D., Basri, R., Brandt, A.: Hierarchy and adaptivity in segmenting visual scenes. Nature 442(7104), 719–846 (2006)

    Article  Google Scholar 

  37. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Google Scholar 

  38. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P.: Caltech-UCSD birds 200. Tech. Rep. CNS-TR-2010-001, California Institute of Technology (2010)

  39. Yanikoglu, B., Aptoula, E., Tirkaz, C.: Sabanci-Okan System at ImageClef 2012: Combining features and classifiers for plant identification. In: Working Notes of CLEF 2012 Conference (2012)

Download references

Acknowledgments

The authors would like to gratefully acknowledge Peter N. Belhumeur, Neeraj Kumar, and Arijit Biswas for helping organize the collections of images used in this work. W. John Kress, Ida C. Lopez, and collaborators at the Smithsonian Institution’s Department of Botany collected and curated the Lab and Field datasets. The authors are grateful to Aditya Malik for manually segmenting several of the leaf images and for helpful discussions. We would also like to acknowledge the authors of the several segmentation methods whose publically available implementations we have used. This work was supported by National Science Foundation grants #0968546, #0325867, and #1116631.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Maryland, College Park, USA

    João V. B. Soares & David W. Jacobs

Authors
  1. João V. B. Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David W. Jacobs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João V. B. Soares.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 892 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Soares, J.V.B., Jacobs, D.W. Efficient segmentation of leaves in semi-controlled conditions. Machine Vision and Applications 24, 1623–1643 (2013). https://doi.org/10.1007/s00138-013-0530-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-013-0530-0

Keywords

Search

Navigation