Machine Vision and Applications

, Volume 27, Issue 5, pp 735–749 | Cite as

Multi-modality imagery database for plant phenotyping

  • Jeffrey A. Cruz
  • Xi Yin
  • Xiaoming Liu
  • Saif M. Imran
  • Daniel D. Morris
  • David M. Kramer
  • Jin Chen
Special Issue Paper


Among many applications of machine vision, plant image analysis has recently began to gain more attention due to its potential impact on plant visual phenotyping, particularly in understanding plant growth, assessing the quality/performance of crop plants, and improving crop yield. Despite its importance, the lack of publicly available research databases containing plant imagery has substantially hindered the advancement of plant image analysis. To alleviate this issue, this paper presents a new multi-modality plant imagery database named “MSU-PID,” with two distinct properties. First, MSU-PID is captured using four types of imaging sensors, fluorescence, infrared, RGB color, and depth. Second, the imaging setup and the variety of manual labels allow MSU-PID to be suitable for a diverse set of plant image analysis applications, such as leaf segmentation, leaf counting, leaf alignment, and leaf tracking. We provide detailed information on the plants, imaging sensors, calibration, labeling, and baseline performances of this new database.


Plant phenotyping Computer vision Plant image Leaf segmentation Leaf tracking Multiple sensors Arabidopsis Bean 


  1. 1.
    Baker, N.R.: Chlorophyll fluorescence: a probe of photosynthesis in vivo. Annu. Rev. Plant Biol. 59, 89–113 (2008)CrossRefGoogle Scholar
  2. 2.
    Barbosa, I.B., Cristani, M., Del Bue, A., Bazzani, L., Murino, V.: Re-identification with RGB-D sensors. In: First International Workshop on Re-Identification, pp. 433–442 (2012)Google Scholar
  3. 3.
    Barrow, H.G., Tenenbaum, J.M., Bolles, R.C., Wolf, H.C.: Parametric correspondence and Chamfer matching: two new techniques for image matching. In: Technical report, DTIC Document (1977)Google Scholar
  4. 4.
    Butler, W.L., Hendricks, S.B., Siegelman, H.W.: Action spectra of phytochrome in vitro. Photochem. Photobiol. 3(4), 521–528 (1964)CrossRefGoogle Scholar
  5. 5.
    Chelle, M.: Could plant leaves be treated as Lambertian surfaces in dense crop canopies to estimate light absorption? Ecol. Model. 198(1), 219–228 (2006)CrossRefGoogle Scholar
  6. 6.
    Chen, D., Neumann, K., Friedel, S., Kilian, Benjamin, Chen, M., Altmann, T., Klukas, C.: Dissecting the phenotypic components of crop plant growth and drought responses based on high-throughput image analysis. Plant Cell 26(12), 4636–4655 (2014)CrossRefGoogle Scholar
  7. 7.
    Chen, J., Liu, X.: Transfer learning with one-class data. Pattern Recognit. Lett. 37, 32–40 (2013)CrossRefGoogle Scholar
  8. 8.
    Chen, J., Liu, X., Lyu, S.: Boosting with side information. In: Proceedings of the Asian Conference on Computer Vision (ACCV), pp. 563–577. Springer, New York (2013)Google Scholar
  9. 9.
    Ding, Z., Ming, S., Fu, Y.: Latent low-rank transfer subspace learning for missing modality recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2014)Google Scholar
  10. 10.
    Döös, B.R.: Population growth and loss of arable land. Glob. Environ. Change 12(4), 303–311 (2002)CrossRefGoogle Scholar
  11. 11.
    Eskins, K.: Light-quality effects on Arabidopsis development. Red, blue and far-red regulation of flowering and morphology. Physiol. Plant. 86(3), 439–444 (1992)Google Scholar
  12. 12.
    Fahlgren, N., Gehan, M.A., Baxter, I.: Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. Curr. Opin. Plant Biol. 24, 93–99 (2015)CrossRefGoogle Scholar
  13. 13.
    Furbank, R.T., Tester, M.: Phenomics-technologies to relieve the phenotyping bottleneck. Trends Plant Sci. 16(12), 635–644 (2011)CrossRefGoogle Scholar
  14. 14.
    Hansard, M., Lee, S., Choi, O., Horaud, R.: Time-of-Flight Cameras: Principles. Methods and Applications. Springer, New York (2013)CrossRefGoogle Scholar
  15. 15.
    Haug, S., Ostermann, J.: A crop/weed field image dataset for the evaluation of computer vision based precision agriculture tasks. In: Proceedings of the European Conference on Computer Vision Workshops (ECCVW), pp. 105–116. Springer, New York ( 2014)Google Scholar
  16. 16.
    Houle, D., Govindaraju, D.R., Omholt, S.: Phenomics: the next challenge. Nat. Rev. Genet. 11(12), 855–866 (2010)CrossRefGoogle Scholar
  17. 17.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In:Technical Report 07-49, University of Massachusetts, Amherst (2007)Google Scholar
  18. 18.
    Johannsen, W.L.: Erblichkeit in Populationen und in reinen Linien. Gustav Fischer, Germany (1903)Google Scholar
  19. 19.
    Kramer, D., Cruz, J., Hall, C., Kovac, W.K., Zegarac, R.: Plant Phenometrics Systems and Methods and Devices Related Thereto. In: US Patent 20, 150, 204, 787 (2015)Google Scholar
  20. 20.
    Kumar, N., Belhumeur, P.N., Biswas, A., Jacobs, D.W., John Kress, W., Lopez, I.C., Soares, J.V.B.: Leafsnap: a computer vision system for automatic plant species identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 502–516. Springer, New York (2012)Google Scholar
  21. 21.
    Kevin, L., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1817–1824 (2011)Google Scholar
  22. 22.
    McClung, C.R.: Plant circadian rhythms. Plant Cell 18(4), 792–803 (2006)CrossRefGoogle Scholar
  23. 23.
    Minervini, M., Fischbach, A., Scharr, H., Tsaftaris, S.A.: Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recogn. Lett. (2015). doi: 10.1016/j.patrec.2015.10.013
  24. 24.
    Minervini, M., Scharr, H., Tsaftaris, S.A.: Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Process. Mag. 32(4), 126–131 (2015)CrossRefGoogle Scholar
  25. 25.
    Nguyen, V.D., Chew, M.T., Demidenko, S.: Vietnamese sign language reader using Intel Creative Senz3D. In: IEEE International Conference on Automation, Robotics and Applications (ICARA), pp. 77–82 (2015)Google Scholar
  26. 26.
    Oxborough, Kevin: Imaging of chlorophyll a fluorescence: theoretical and practical aspects of an emerging technique for the monitoring of photosynthetic performance. J. Exp. Bot. 55(400), 1195–1205 (2004)CrossRefGoogle Scholar
  27. 27.
    Phillips, P.J., Moon, H., Rauss, P.J., Rizvi, S.: The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)CrossRefGoogle Scholar
  28. 28.
    Scharr, H., Minervini, M., Fischbach, A., Tsaftaris, S.A.: Annotated image datasets of rosette plants. In:Technical Report FZJ-2014-03837 (2014)Google Scholar
  29. 29.
    Schöttler, M.A., Tóth, S.Z., Boulouis, A., Kahlau, S.: Photosynthetic complex stoichiometry dynamics in higher plants: biogenesis, function, and turnover of ATP synthase and the cytochrome b 6 f complex. J. Exp. Bot. 66(9), 2373–2400 (2015)CrossRefGoogle Scholar
  30. 30.
    Silberman, N., Fergus, R.: Indoor scene segmentation using a structured light sensor. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 601–608 (2011)Google Scholar
  31. 31.
    Söderkvist, O.: Computer vision classification of leaves from Swedish trees. In: Master thesis, Linköping University (2001)Google Scholar
  32. 32.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of the International Conference on Intelligent Robot Systems (IROS), pp. 573–580 (2012)Google Scholar
  33. 33.
    Sung, J., Ponce, C., Selman, B., Saxena, A.: Human activity detection from RGBD images. CoRR 64 (2011)Google Scholar
  34. 34.
    Walter, A., Liebisch, F., Hund, A.: Plant phenotyping: from bean weighing to image analysis. Plant Methods 11(1), 14 (2015)CrossRefGoogle Scholar
  35. 35.
    Woolley, J.T.: Reflectance and transmittance of light by leaves. Plant Physiol. 47(5), 656–662 (1971)CrossRefGoogle Scholar
  36. 36.
    Wu, S.G., Bao, F.S., Xu, E.Y., Wang, Y.-X., Chang, Y.-F., Xiang, Q.-L.: A leaf recognition algorithm for plant classification using probabilistic neural network. In: IEEE International Symposium on Signal Processing and Information Technology, pp. 11–16 (2007)Google Scholar
  37. 37.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf alignment from fluorescence plant images. In: IEEE Winter Conference on Applications of Computer Vision (WACV), Steamboat Springs (2014)Google Scholar
  38. 38.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf tracking from fluorescence plant videos. In: Proceedings of the International Conference on Image Processing (ICIP), Paris (2014)Google Scholar
  39. 39.
    Yin, X., Liu, X., Chen, J., Kramer, D.M.: Joint Multi-Leaf Segmentation, Alignment, and Tracking from Fluorescence Plant Videos (2015). arXiv:1505.00353
  40. 40.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Jeffrey A. Cruz
    • 1
  • Xi Yin
    • 2
  • Xiaoming Liu
    • 2
  • Saif M. Imran
    • 3
  • Daniel D. Morris
    • 3
  • David M. Kramer
    • 1
  • Jin Chen
    • 1
  1. 1.Department of Energy Plant Research LaboratoryMichigan State UniversityEast LansingUSA
  2. 2.Department of Computer Science and EngineeringMichigan State UniversityEast LansingUSA
  3. 3.Department of Electrical and Computer EngineeringMichigan State UniversityEast LansingUSA

Personalised recommendations