Plant phenotyping is the identification of effects on the phenotype (i.e., the plant appearance and behavior) as a result of genotype differences (i.e., differences in the genetic code) and the environment. Previously, the process of taking phenotypic measurements has been laborious, costly, and time-consuming. In recent years, noninvasive, imaging-based methods have become more common. These images are recorded by a range of capture devices from small embedded camera systems to multi-million Euro smart greenhouses, at scales ranging from microscopic images of cells, to entire fields captured by UAV imaging.

These images need to be analyzed in a high-throughput, robust, and accurate manner. UN-FAO statistics show that according to current population predictions we will need to achieve a 70 % increase in food productivity by 2050, simply to maintain current global agricultural demands. Phenomics—large-scale measurement of plant traits—is the bottleneck in knowledge-based bioeconomy, and machine vision is ideally placed to help [16]. However, the occurring challenges differ from usual tasks addressed by the computer vision community due to the requirements posed by this application scenario. Dealing with these new problems has spawned new specialized workshops such as Computer Vision Problems in Plant Phenotyping (CVPPP) which was held for the first time in conjunction with the European Conference on Computer Vision (ECCV) 2014 and the second time with the British Machine Vision Conference (BMVC) 2015, and the stand-alone workshop IAMPS (Image Analysis Methods for the Plant Sciences) now in its fourth year.

The overriding goal of this special issue is not only to present interesting computer vision solutions, but also to introduce challenging computer vision problems in the increasingly important plant phenotyping domain, accompanied with benchmark datasets and suitable performance evaluation methods.

The 12 papers presented in this special issue [1, 3, 58, 10, 11, 13, 17, 18, 21] were selected and revised from submissions received in response to an open call for papers. Six of them [1, 3, 11, 13, 18, 21] were developed from conference contributions at CVPPP 2014 [2, 4, 12, 14, 19, 20]. Two other papers received by the open call already appeared in a preceding or an adjacent issue of this journal and are also part of this special issue [9, 22].

Overall, we have received contributions that image the plant either above the ground or below the ground. Some rely on 2D or multiple 2D images [1, 6], 3D surfaces from 2D [8, 21], 3D reconstruction [18], and hyperspectral imaging [3, 5], while others rely on 3D volumetric imaging [13] and proper analysis algorithms.

Two papers [11, 17] deal with recognizing and classifying plants or whole trees, which can be extremely useful when one is interested in categorizing new cultivars and hybrids against a bank of known phenotypes.

A survey on results of the Leaf Segmentation Challenge at CVPPP 2014 presents state-of-the-art algorithms competing on the challenging problem of segmenting each single leaf from images showing rosette plants [22] on the basis of a recently published benchmark dataset [15]. This was the first serious computer vision dataset within the plant domain. The availability of datasets is further enriched by the paper of Cruz et al. [7] in which multimodal images of model plants are presented. A benchmark in leaf segmentation is achieved relying on a template matching method. The presence of public and shared datasets has been a major driver of progress in mainstream computer vision, and it is very promising to see an emerging openness about data in this new subdomain.

All the papers discussed as of now demonstrate the complexity of extracting phenotyping information when plants are imaged in controlled environmental settings. When we move to the field the ultimate and most useful frontier, things become exceedingly more complex. To motivate further research into this area, Kelly et al. [9] discuss an interesting set of challenges when phenotyping field crops.

The application of plant phenotyping opens up a wealth of novel and important opportunities for the machine vision community. The wide variety of image scales (from cell to field), modalities (visible, X-ray, infrared, hyperspectral), and dimensions (2D, 3D, time series), in addition to complex environmental factors, make the research challenging. The fruits of the advances, though, can have a direct impact on our ability to understand plant growth and improve the efficiency of how we grow our food.

To conclude, we are truly excited to have made the first steps in introducing this fascinating application domain for computer vision. With this special issue, the introduction and release of open benchmark datasets [7, 15] and workshops at prestigious venues, we hope to attract more colleagues from the computer vision community in our quest toward advancing the state of the art in this societally and environmentally important area. We hope also that through highlighting some recent computer vision systems and applications within this domain we can also convince the reader that there are some deep, interesting, and challenging problems in the world of plant imaging.