Abstract
Hyperspectral imaging (HSI) is a prevalent method in crop phenotyping. Nevertheless, current HSI remote sensing techniques are compromised by changing ambient lighting conditions, long imaging distances, and comparatively low resolutions. Proximal HSI sensors such as LeafSpec were developed to improve the imaging quality. However, the application of proximal sensors remains contrained by their low throughput and intensive labor costs. Moreover, few automation solutions were available to use LeafSpec in phenotyping dicot plants. In this paper, a novel robotic system is presented as a sensor platform to operate LeafSpec to collect leaf-level hyperspectral images for in vivo phenotyping of soybean. A machine vision algorithm was developed to detect the top mature trifoliate and estimate the poses of the leaflets. A control and motion planning algorithm was developed for an articulated robotic manipulator to grasp the target leaflets. An experiment was conducted in March 2021 in a greenhouse with 64 soybean plants of 2 genotypes and 2 nitrogen treatments. The machine vision detected the target leaflets with a first trial success rate of 84.13% and an overall success rate of 90.66%. The robotic manipulator operated LeafSpec to image the target leaflets with a first trial success rate of 87.30% and an overall success rate of 93.65%. The average cycle time for one soybean plant was 63.20 s. The PLS predictions from the robot-collected data had an R2 of 0.84 with the measured nitrogen content and an R2 of 0.82 with the predictions from human-collected data. The results demonstrated the potential of applying the system for automated in vivo leaf-level HSI for soybean phenotyping in the field.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atefi, A., Ge, Y., Pitla, S., & Schnable, J. (2019). In vivo human-like robotic phenotyping of leaf traits in maize and sorghum in greenhouse. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2019.104854
Bao, Y., Shah, D., & Tang, L. (2018). 3D Perception-based collision-free robotic leaf probing for automated indoor plant phenotyping. Transactions of the ASABE, 61(3), 859–872. https://doi.org/10.13031/trans.12653
Bradley, C. A., Allen, T. W., Sisson, A. J., Bergstrom, G. C., Bissonnette, K. M., Bond, J., Byamukama, E., Chilvers, M. I., Collins, A. A., Damicone, J. P., Dorrance, A. E., Dufault, N. S., Esker, P. D., Faske, T. R., Fiorellino, N. M., Giesler, L. J., Hartman, G. L., Hollier, C. A., Isakeit, T., & Wise, K. A. (2021). Soybean yield loss estimates due to diseases in the United States and Ontario, Canada, from 2015 to 2019. Plant Health Progress, 22(4), 483–495. https://doi.org/10.1094/PHP-01-21-0013-RS
Campbell, T. (2021). TSC07921.
Chen, Z., Wang, J., Wang, T., Song, Z., Li, Y., Huang, Y., Wang, L., & Jin, J. (2021). Automated in-field leaf-level hyperspectral imaging of corn plants using a Cartesian robotic platform. Computers and Electronics in Agriculture, 183, 105996. https://doi.org/10.1016/j.compag.2021.105996
Cui, D., Zhang, Q., Li, M., Zhao, Y., & Hartman, G. L. (2009). Detection of soybean rust using a multispectral image sensor. Sensing and Instrumentation for Food Quality and Safety, 3(1), 49–56. https://doi.org/10.1007/s11694-009-9070-8
da Silva Junior, C. A., Nanni, M. R., Shakir, M., Teodoro, P. E., de Oliveira-Júnior, J. F., Cezar, E., de Gois, G., Lima, M., Wojciechowski, J. C., & Shiratsuchi, L. S. (2018). Soybean varieties discrimination using non-imaging hyperspectral sensor. Infrared Physics and Technology, 89, 338–350. https://doi.org/10.1016/j.infrared.2018.01.027
FAO. (2020). World Food and Agriculture-Statistical Yearbook. Cham: FAO. https://doi.org/10.4060/cb1329en
Fehr, W. R., & Caviness, C. E. (1977). Stages of soybean development. Special Report, 80(March), 11.
Fletcher, R. S., & Reddy, K. N. (2016). Random forest and leaf multispectral reflectance data to differentiate three soybean varieties from two pigweeds. Computers and Electronics in Agriculture, 128, 199–206. https://doi.org/10.1016/j.compag.2016.09.004
Gowen, A. A., O’Donnell, C. P., Cullen, P. J., Downey, G., & Frias, J. M. (2007). Hyperspectral imaging-An emerging process analytical tool for food quality and safety control. Trends in Food Science and Technology, 18(12), 590–598. https://doi.org/10.1016/j.tifs.2007.06.001
Gui, J., Fei, J., Wu, Z., Fu, X., & Diakite, A. (2021). Grading method of soybean mosaic disease based on hyperspectral imaging technology. Information Processing in Agriculture, 8(3), 380–385. https://doi.org/10.1016/j.inpa.2020.10.006
Guilherme Teixeira Crusiol, L., Braga, P., Rafael Nanni, M., Furlanetto, R. H., Sibaldelli, R. N. R., Cezar, E., Sun, L., Foloni, J. S. S., Mertz-Henning, L. M., Lima Nepomuceno, A., Neumaier, N., & Bouças Farias, J. R. (2021). Using leaf-based hyperspectral reflectance for genotype classification within a soybean germplasm collection assessed under different levels of water availability. International Journal of Remote Sensing, 42(21), 8165–8184. https://doi.org/10.1080/01431161.2021.1975841
Hellal, F. A., & Abdelhamid, M. T. (2013). Revisión nutrient management practices for enhancing soybean (Glycine max L.) PRODUCTION Prácticas de gestión de nutrientes para mejoramiento en la producción de soja (Glycine max L.). Acta Biológica Colombiana, 18(2), 239–250.
Iriel, A., Cordon, G., Fernández Cirelli, A., & Lagorio, M. G. (2019). Non-destructive methodologies applied to track the occurrence of natural micropollutants in watering: glycine max as a biomonitor. Ecotoxicology and Environmental Safety, 182(June), 109368. https://doi.org/10.1016/j.ecoenv.2019.109368
Kao, W.-Y., & Forseth, I. N. (1992). Dirunal leaf movement, chlorophyll fluorescence and carbon assimilation in soybean grown under different nitrogen and water availabilities. Plant, Cell & Environment, 15(6), 703–710. https://doi.org/10.1111/j.1365-3040.1992.tb01012.x
Kovar, M., Brestic, M., Sytar, O., Barek, V., Hauptvogel, P., & Zivcak, M. (2019). Evaluation of hyperspectral reflectance parameters to assess the leafwater content in soybean. Water (switzerland), 11(3), 1–12. https://doi.org/10.3390/w11030443
Kruse, F. A., Lefkoff, A. B., Boardman, J. W., Heidebrecht, K. B., Shapiro, A. T., Barloon, P. J., & Goetz, A. F. H. (1993). The spectral image processing system (SIPS)—Interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 44(2–3), 145–163. https://doi.org/10.1016/0034-4257(93)90013-N
Lee, C. S. G., & Ziegler, M. (1984). Geometric approach in solving inverse kinematics of puma robots. IEEE Transactions on Aerospace and Electronic Systems, AES, 20(6), 695–706. https://doi.org/10.1109/TAES.1984.310452
Li, X., Chen, Z., Wang, J., & Jin, J. (2023). LeafSpec-dicot: An accurate and portable hyperspectral imaging device for dicot leaves. Sensors (basel, Switzerland). https://doi.org/10.3390/s23073687
Ma, D., Rehman, T. U., Zhang, L., Maki, H., Tuinstra, M. R., & Jin, J. (2021a). Modeling of diurnal changing patterns in airborne crop remote sensing images. Remote Sensing, 13(9), 1–19. https://doi.org/10.3390/rs13091719
Ma, D., Rehman, T. U., Zhang, L., Maki, H., Tuinstra, M. R., & Jin, J. (2021b). Modeling of environmental impacts on aerial hyperspectral images for corn plant phenotyping. Remote Sensing. https://doi.org/10.3390/rs13132520
Mehmood, T., Sæbø, S., & Liland, K. H. (2020). Comparison of variable selection methods in partial least squares regression. Journal of Chemometrics, 34(6), 1–14. https://doi.org/10.1002/cem.3226
Oerke, E. C. (2006). Crop losses to pests. Journal of Agricultural Science, 144(1), 31–43. https://doi.org/10.1017/S0021859605005708
Pandey, P., Ge, Y., Stoerger, V., & Schnable, J. C. (2017). High throughput in vivo analysis of plant leaf chemical properties using hyperspectral imaging. Frontiers in Plant Science, 8, 1348. https://doi.org/10.3389/fpls.2017.01348
Rehman, T. U., Zhang, L., Ma, D., Wang, L., & Jin, J. (2020). Calibration transfer across multiple hyperspectral imaging-based plant phenotyping systems: I-Spectral space adjustment. Computers and Electronics in Agriculture, 176(June), 105685. https://doi.org/10.1016/j.compag.2020.105685
ROBOTIS INC. (n.d.). OpenMANIPULATOR-P (RM-P60-RNH). ROBOTIS INC. Retrieved April 2, 2022, from https://www.robotis.us/openmanipulator-p-rm-p60-rnh/
Wang, L., Jin, J., Song, Z., Wang, J., Zhang, L., Rehman, T. U., Ma, D., Carpenter, N. R., & Tuinstra, M. R. (2020). LeafSpec: An accurate and portable hyperspectral corn leaf imager. Computers and Electronics in Agriculture, 169(January), 105209. https://doi.org/10.1016/j.compag.2019.105209
Wu, C., Zeng, R., Pan, J., Wang, C. C. L., & Liu, Y. J. (2019). Plant phenotyping by deep-learning-based planner for multi-robots. IEEE Robotics and Automation Letters, 4(4), 3113–3120. https://doi.org/10.1109/LRA.2019.2924125
Yuan, H., Yang, G., Li, C., Wang, Y., Liu, J., Yu, H., Feng, H., Xu, B., Zhao, X., & Yang, X. (2017). Retrieving soybean leaf area index from unmanned aerial vehicle hyperspectral remote sensing: Analysis of RF, ANN, and SVM regression models. Remote Sensing. https://doi.org/10.3390/rs9040309
Zhang, J., Huang, Y., Reddy, K. N., & Wang, B. (2019a). Assessing crop damage from dicamba on non-dicamba-tolerant soybean by hyperspectral imaging through machine learning. Pest Management Science, 75(12), 3260–3272. https://doi.org/10.1002/ps.5448
Zhang, L., Maki, H., Ma, D., Sánchez-Gallego, J. A., Mickelbart, M. V., Wang, L., Rehman, T. U., & Jin, J. (2019b). Optimized angles of the swing hyperspectral imaging system for single corn plant. Computers and Electronics in Agriculture, 156, 349–359. https://doi.org/10.1016/j.compag.2018.11.030
Zhang, X., Zhao, J., Yang, G., Liu, J., Cao, J., Li, C., Zhao, X., & Gai, J. (2019c). Establishment of plot-yield prediction models in soybean breeding programs using UAV-based hyperspectral remote sensing. Remote Sensing. https://doi.org/10.3390/rs11232752
Acknowledgements
The authors would like to thank Haokun Dong (Graduate Student, Department of Statistics, Columbia University) for his expertise in statistics and Yikai Li (Graduate Student, Department of Agricultural and Biological Engineering, Purdue University) for his hardware support.
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, Z., Wang, J. & Jin, J. Fully automated proximal hyperspectral imaging system for high-resolution and high-quality in vivo soybean phenotyping. Precision Agric 24, 2395–2415 (2023). https://doi.org/10.1007/s11119-023-10045-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11119-023-10045-5