Machine Vision and Applications

, Volume 27, Issue 4, pp 585–606 | Cite as

Leaf segmentation in plant phenotyping: a collation study

  • Hanno Scharr
  • Massimo Minervini
  • Andrew P. French
  • Christian Klukas
  • David M. Kramer
  • Xiaoming Liu
  • Imanol Luengo
  • Jean-Michel Pape
  • Gerrit Polder
  • Danijela Vukadinovic
  • Xi Yin
  • Sotirios A. Tsaftaris
Survey

Abstract

Image-based plant phenotyping is a growing application area of computer vision in agriculture. A key task is the segmentation of all individual leaves in images. Here we focus on the most common rosette model plants, Arabidopsis and young tobacco. Although leaves do share appearance and shape characteristics, the presence of occlusions and variability in leaf shape and pose, as well as imaging conditions, render this problem challenging. The aim of this paper is to compare several leaf segmentation solutions on a unique and first-of-its-kind dataset containing images from typical phenotyping experiments. In particular, we report and discuss methods and findings of a collection of submissions for the first Leaf Segmentation Challenge of the Computer Vision Problems in Plant Phenotyping workshop in 2014. Four methods are presented: three segment leaves by processing the distance transform in an unsupervised fashion, and the other via optimal template selection and Chamfer matching. Overall, we find that although separating plant from background can be accomplished with satisfactory accuracy (\(>\)90 % Dice score), individual leaf segmentation and counting remain challenging when leaves overlap. Additionally, accuracy is lower for younger leaves. We find also that variability in datasets does affect outcomes. Our findings motivate further investigations and development of specialized algorithms for this particular application, and that challenges of this form are ideally suited for advancing the state of the art. Data are publicly available (online at http://www.plant-phenotyping.org/datasets) to support future challenges beyond segmentation within this application domain.

Keywords

Plant phenotyping Leaf Multi-instance segmentation Machine learning 

References

  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)CrossRefGoogle Scholar
  3. 3.
    Aksoy, E., Abramov, A., Wörgötter, F., Scharr, H., Fischbach, A., Dellen, B.: Modeling leaf growth of rosette plants using infrared stereo image sequences. Comput. Electron. Agric. 110, 78–90 (2015)CrossRefGoogle Scholar
  4. 4.
    Alenyà, G., Dellen, B., Torras, C.: 3D modelling of leaves from color and ToF data for robotized plant measuring. In: IEEE International Conference on Robotics and Automation, pp. 3408–3414 (2011)Google Scholar
  5. 5.
    Arvidsson, S., Pérez-Rodríguez, P., Mueller-Roeber, B.: A growth phenotyping pipeline for Arabidopsis thaliana integrating image analysis and rosette area modeling for robust quantification of genotype effects. New Phytol 191(3), 895–907 (2011)CrossRefGoogle Scholar
  6. 6.
    Augustin, M., Haxhimusa, Y., Busch, W., Kropatsch, W.G.: Image-based phenotyping of the mature Arabidopsis shoot system. In: Computer Vision—ECCV 2014 Workshops, vol. 8928, pp. 231–246. Springer (2015)Google Scholar
  7. 7.
    Bansal, S., Aggarwal, D.: Color image segmentation using CIELab color space using ant colony optimization. Int. J. Comput. Appl. 29(9), 28–34 (2011)Google Scholar
  8. 8.
    Barrow, H., Tenenbaum, J., Bolles, R., Wolf, H.: Parametric correspondence and chamfer matching: two new techniques for image matching. Tech. rep, DTIC (1977)Google Scholar
  9. 9.
    Beucher, S.: The watershed transformation applied to image segmentation. Scanning Microsc. Int. 6, 299–314 (1992)Google Scholar
  10. 10.
    Biskup, B., Scharr, H., Schurr, U., Rascher, U.: A stereo imaging system for measuring structural parameters of plant canopies. Plant Cell Environ. 30, 1299–1308 (2007)CrossRefGoogle Scholar
  11. 11.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  12. 12.
    Casanova, D., Florindo, J.B., Gonçalves, W.N., Bruno, O.M.: IFSC/USP at ImageCLEF 2012: plant identification task. In: CLEF (Online Working Notes/Labs/Workshop) (2012)Google Scholar
  13. 13.
    Cerutti, G., Antoine, V., Tougne, L., Mille, J., Valet, L., Coquin, D., Vacavant, A.: ReVeS participation: tree species classification using random forests and botanical features. In: Conference and Labs of the Evaluation Forum (2012)Google Scholar
  14. 14.
    Cerutti, G., Tougne, L., Mille, J., Vacavant, A., Coquin, D.: Understanding leaves in natural images: a model-based approach for tree species identification. Comput. Vis. Image Underst. 10(117), 1482–1501 (2013)CrossRefGoogle Scholar
  15. 15.
    CORESTA, C.: A scale for coding growth stages in tobacco crops (2009). http://www.coresta.org/Guides/Guide-No07-Growth-Stages_Feb09.pdf
  16. 16.
    De Vylder, J., Ochoa, D., Philips, W., Chaerle, L., Van Der Straeten, D.: Leaf segmentation and tracking using probabilistic parametric active contours. In: International Conference on Computer Vision/Computer Graphics Collaboration Techniques, pp. 75–85 (2011)Google Scholar
  17. 17.
    De Vylder, J., Vandenbussche, F.J., Hu, Y., Philips, W., Van Der Straeten, D.: Rosette Tracker: an open source image analysis tool for automatic quantification of genotype effects. Plant Physiol. 160(3), 1149–1159 (2012)CrossRefGoogle Scholar
  18. 18.
    Dellen, B., Scharr, H., Torras, C.: Growth signatures of rosette plants from time-lapse video. IEEE/ACM Trans. Comput. Biol. Bioinform. PP(99), 1–11 (2015)Google Scholar
  19. 19.
    Dubuisson, M.P., Jain, A.K.: A modified Hausdorff distance for object matching. In: Proceedings of the 12th IAPR International Conferenced on Pattern Recognition, vol. 1, pp. 566–568 (1994)Google Scholar
  20. 20.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  21. 21.
    Giuffrida, M.V., Minervini, M., Tsaftaris, S.A.: Learning to count leaves in rosette plants. In: Proceedings of the Computer Vision Problems in Plant Phenotyping (CVPPP) Workshop, pp. 1.1–1.13. BMVA Press (2015)Google Scholar
  22. 22.
    Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)CrossRefGoogle Scholar
  23. 23.
    Granier, C., Aguirrezabal, L., Chenu, K., Cookson, S.J., Dauzat, M., Hamard, P., Thioux, J.J., Rolland, G., Bouchier-Combaud, S., Lebaudy, A., Muller, B., Simonneau, T., Tardieu, F.: PHENOPSIS, an automated platform for reproducible phenotyping of plant responses to soil water deficit in Arabidopsis thaliana permitted the identification of an accession with low sensitivity to soil water deficit. New Phytol. 169(3), 623–635 (2006)CrossRefGoogle Scholar
  24. 24.
    Hartmann, A., Czauderna, T., Hoffmann, R., Stein, N., Schreiber, F.: HTPheno: an image analysis pipeline for high-throughput plant phenotyping. BMC Bioinform. 12(1), 148 (2011)CrossRefGoogle Scholar
  25. 25.
    He, X., Gould, S.: An exemplar-based CRF for multi-instance object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 296–303 (2014)Google Scholar
  26. 26.
    Jansen, M., Gilmer, F., Biskup, B., Nagel, K., Rascher, U., Fischbach, A., Briem, S., Dreissen, G., Tittmann, S., Braun, S., Jaeger, I.D., Metzlaff, M., Schurr, U., Scharr, H., Walter, A.: Simultaneous phenotyping of leaf growth and chlorophyll fluorescence via GROWSCREEN FLUORO allows detection of stress tolerance in Arabidopsis thaliana and other rosette plants. Funct. Plant Biol. 36(10/11), 902–914 (2009)CrossRefGoogle Scholar
  27. 27.
    Jin, J., Tang, L.: Corn plant sensing using real-time stereo vision. J. Field Robot. 26(6–7), 591–608 (2009)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Kalyoncu, C., Toygar, Ö.: Geometric leaf classification. Comput. Vis. Image Underst. 133, 102–109 (2015)CrossRefGoogle Scholar
  29. 29.
    Klukas, C., Chen, D., Pape, J.M.: Integrated analysis platform: an open-source information system for high-throughput plant phenotyping. Plant Physiol. 165(2), 506–518 (2014)CrossRefGoogle Scholar
  30. 30.
    Kurugollu, F., Sankur, B., Harmanci, A.E.: Color image segmentation using histogram multithresholding and fusion. Image Vis. Comput. 19(13), 915–928 (2001)CrossRefGoogle Scholar
  31. 31.
    Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)CrossRefGoogle Scholar
  32. 32.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Int. Conf. Comput. Vis. (ICCV) 2, 416–423 (2001)Google Scholar
  33. 33.
    Mezaris, V., Kompatsiaris, I., Strintzis, M.: Still image objective segmentation evaluation using ground truth. In: 5th COST 276 Workshop, pp. 9–14 (2003)Google Scholar
  34. 34.
    Minervini, M., Abdelsamea, M.M., Tsaftaris, S.A.: Image-based plant phenotyping with incremental learning and active contours. Ecol. Inform. 23, 35–48 (2014). (Special Issue on Multimedia in Ecology and Environment)CrossRefGoogle Scholar
  35. 35.
    Minervini, M., Fschbach, A., Scharr, H., Tsaftaris, S.: Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recogn. Lett. (2015) (In press)Google Scholar
  36. 36.
    Minervini, M., Giuffrida, M.V., Tsaftaris, S.A.: An interactive tool for semi-automated leaf annotation. In: Proceedings of the Computer Vision Problems in Plant Phenotyping (CVPPP) Workshop, pp. 6.1-6.13. BMVA Press (2015)Google Scholar
  37. 37.
    Minervini, M., Scharr, H., Tsaftaris, S.A.: Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Process. Mag. 32(4), 126–131 (2015)CrossRefGoogle Scholar
  38. 38.
    Müller-Linow, M., Pinto-Espinosa, F., Scharr, H., Rascher, U.: The leaf angle distribution of natural plant populations: assessing the canopy with a novel software tool. Plant Methods 11(1), 11 (2015)CrossRefGoogle Scholar
  39. 39.
    Nagel, K., Putz, A., Gilmer, F., Heinz, K., Fischbach, A., Pfeifer, J., Faget, M., Blossfeld, S., Ernst, M., Dimaki, C., Kastenholz, B., Kleinert, A.K., Galinski, A., Scharr, H., Fiorani, F., Schurr, U.: GROWSCREEN-Rhizo is a novel phenotyping robot enabling simultaneous measurements of root and shoot growth for plants grown in soil-filled rhizotrons. Funct. Plant Biol. 39, 891–904 (2012)CrossRefGoogle Scholar
  40. 40.
    Nieuwenhuis, C., Cremers, D.: Spatially varying color distributions for interactive multilabel segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1234–1247 (2013)CrossRefGoogle Scholar
  41. 41.
    Pape, J.M., Klukas, C.: 3-D histogram-based segmentation and leaf detection for rosette plants. In: Computer Vision—ECCV 2014 Workshops, vol. 8928, pp. 61–74. Springer (2015)Google Scholar
  42. 42.
    Polak, M., Zhang, H., Pi, M.: An evaluation metric for image segmentation of multiple objects. Image Vis. Comput. 27(8), 1223–1227 (2009)CrossRefGoogle Scholar
  43. 43.
    Pratt, W.K.: Digital Image Processing. Wiley-Interscience, New York, NY (1978)MATHGoogle Scholar
  44. 44.
    Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.: Image-based plant modeling. ACM Trans. Graph. 25(3), 599–604 (2006)CrossRefGoogle Scholar
  45. 45.
    Riemenschneider, H., Sternig, S., Donoser, M., Roth, P.M., Bischof, H.: Hough regions for joining instance localization and segmentation. In: Computer Vision—ECCV 2012, vol. 7574, pp. 258–271. Springer (2012)Google Scholar
  46. 46.
    Scharr, H., Minervini, M., Fischbach, A., Tsaftaris, S.A.: Annotated image datasets of rosette plants. Tech. Rep. FZJ-2014-03837, Forschungszentrum Jülich GmbH, (2014). http://hdl.handle.net/2128/5848
  47. 47.
    Silva, L., Koga, M., Cugnasca, C., Costa, A.: Comparative assessment of feature selection and classification techniques for visual inspection of pot plant seedlings. Comput. Electron. Agric. 97, 47–55 (2013)CrossRefGoogle Scholar
  48. 48.
    Soares, J.V.B., Jacobs, D.W.: Efficient segmentation of leaves in semi-controlled conditions. Mach. Vis. Appl. 24(8), 1623–1643 (2013)CrossRefGoogle Scholar
  49. 49.
    Song, Y., Wilson, R., Edmondson, R., Parsons, N.: Surface modelling of plants from stereo images. In: Proceedings of the 6th International Conference on 3-D Digital Imaging and Modeling (3DIM ’07), pp. 312–319 (2007)Google Scholar
  50. 50.
    Teng, C.H., Kuo, Y.T., Chen, Y.S.: Leaf segmentation, classification, and three-dimensional recovery from a few images with close viewpoints. Opt. Eng. 50(3), 1–13 (2011)Google Scholar
  51. 51.
    Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991)CrossRefGoogle Scholar
  52. 52.
    van der Heijden, G., Song, Y., Horgan, G., Polder, G., Dieleman, A., Bink, M., Palloix, A., van Eeuwijk, F., Glasbey, C.: SPICY: towards automated phenotyping of large pepper plants in the greenhouse. Funct. Plant Biol. 39(11), 870–877 (2012)CrossRefGoogle Scholar
  53. 53.
    W3C: Portable network graphics (PNG) specification (2003)Google Scholar
  54. 54.
    Walter, A., Scharr, H., Gilmer, F., Zierer, R., Nagel, K.A., Ernst, M., Wiese, A., Virnich, O., Christ, M.M., Uhlig, B., Jünger, S., Schurr, U.: Dynamics of seedling growth acclimation towards altered light conditions can be quantified via GROWSCREEN: a setup and procedure designed for rapid optical phenotyping of different plant species. New Phytol. 174(2), 447–455 (2007)CrossRefGoogle Scholar
  55. 55.
    Walter, A., Schurr, U.: The modular character of growth in Nicotiana tabacum plants under steady-state nutrition. J. Exp. Bot. 50(336), 1169–1177 (1999)CrossRefGoogle Scholar
  56. 56.
    Wang, J., He, J., Han, Y., Ouyang, C., Li, D.: An adaptive thresholding algorithm of field leaf image. Comput. Electron. Agric. 96, 23–39 (2013)CrossRefGoogle Scholar
  57. 57.
    Wu, B., Nevatia, R.: Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Comput. Vis. 82(2), 185–204 (2009)CrossRefGoogle Scholar
  58. 58.
    Yanikoglu, B., Aptoula, E., Tirkaz, C.: Automatic plant identification from photographs. Mach. Vis. Appl. 6(25), 1369–1383 (2014)CrossRefGoogle Scholar
  59. 59.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf alignment from fluorescence plant images. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 437–444 (2014)Google Scholar
  60. 60.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf tracking from fluorescence plant videos. In: IEEE International Conference on Image Processing (ICIP), pp. 408–412 (2014)Google Scholar
  61. 61.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-Leaf Segmentation, Alignment and Tracking from Fluorescence Plant Videos. arXiv:1505.00353 (2015)
  62. 62.
    Ziou, D., Tabbone, S.: Edge detection techniques: an overview. Int. J. Pattern Recogn. Image Anal. 8(4), 537–559 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Hanno Scharr
    • 1
  • Massimo Minervini
    • 2
  • Andrew P. French
    • 3
  • Christian Klukas
    • 9
  • David M. Kramer
    • 5
  • Xiaoming Liu
    • 6
  • Imanol Luengo
    • 3
  • Jean-Michel Pape
    • 4
  • Gerrit Polder
    • 7
  • Danijela Vukadinovic
    • 7
  • Xi Yin
    • 6
  • Sotirios A. Tsaftaris
    • 2
    • 8
  1. 1.Institute of Bio- and Geosciences: Plant Sciences (IBG-2) Forschungszentrum Jülich GmbHJülichGermany
  2. 2.IMT Institute for Advanced StudiesLuccaItaly
  3. 3.Schools of Biosciences and Computer Science, Centre for Plant Integrative BiologyUniversity of NottinghamNottinghamUK
  4. 4.Department of Molecular GeneticsLeibniz Institute of Plant Genetics and Crop Plant Research (IPK)GaterslebenGermany
  5. 5.Departments of Energy Plant Research Lab, and Biochemistry and Molecular BiologyMichigan State UniversityEast LansingUSA
  6. 6.Department of Computer Science and EngineeringMichigan State UniversityEast LansingUSA
  7. 7.Greenhouse HorticultureWageningen University and Research CentreWageningenNetherlands
  8. 8.School of EngineeringUniversity of EdinburghEdinburghUK
  9. 9.LemnaTec GmbHAachenGermany

Personalised recommendations