Skip to main content
SpringerLink
Log in

Wheat-Net: An Automatic Dense Wheat Spike Segmentation Method Based on an Optimized Hybrid Task Cascade Model

  • Chapter
  • First Online:
Unmanned Aerial Systems in Precision Agriculture

Part of the book series: Smart Agriculture ((SA,volume 2))

Abstract

Precise segmentation of wheat spikes from a complex background is necessary for obtaining image-based phenotypic information of wheat traits such as yield estimation and disease evaluation. A new instance segmentation method based on a Hybrid Task Cascade model was trained and validated to improve previous attempts of wheat spike detection. In this study, wheat images were collected from fields where the environment varied both spatially and temporally. Res2Net50 was adopted as a backbone network, combined with multi-scale training, deformable convolutional networks, and Generic ROI Extractor for rich feature learning. The proposed methods were trained and validated, and the average precision (AP) obtained for the bounding box and mask was 0.904 and 0.907, respectively, and the accuracy for wheat spike counting was 99.29%. Comprehensive empirical analyses revealed that our method (Wheat-Net) performed well on challenging field-based datasets with mixed qualities, particularly those with various backgrounds and wheat spike adjacence/occlusion. These results provide evidence for dense wheat spike detection capabilities using the latest date deep learning algorithms, which can be useful for improving wheat breeding and disease screening efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ni C, Wang D, Vinson R, Holmes M, Tao Y (2019) Automatic inspection machine for maize kernels based on deep convo-lutional neural networks. Biosys Eng 178:131–144. https://doi.org/10.1016/j.biosystemseng.2018.11.010

    Article  Google Scholar 

  2. Da Silva LA, Bressan PO, Gonçalves DN, Freitas DM, Machado BB, Gonçalves Wesley N (2019) Estimating soybean leaf defoliation using convolutional neural networks and synthetic images. Comput Electron Agric 156:360–368. https://doi.org/10.1016/j.compag.2018.11.040

  3. Desai SV, Balasubramanian VN, Fukatsu T, Ninomiya S, Guo W (2019) Automatic estimation of heading date of paddy rice using deep learning. Plant Methods 15(1):76. https://doi.org/10.1186/s13007-019-0457-1

    Article  PubMed  PubMed Central  Google Scholar 

  4. Senthilkumar T, Jayas DS, White NDG, Fields PG, Gräfenhan T (2017) Detection of ochratoxin A contamination in stored wheat using nearinfrared hyperspectral imaging. Infrared Phys Techn 81:228–235. https://doi.org/10.1016/j.infred.2017.01.015

  5. Aich S, Josuttes A, Ovsyannikov I, Strueby K, Ahmed I, Duddu HS, Pozniak C, Shirtliffe S, Stavness I (2018) DeepWheat: estimating phenotypic traits from crop images with deep learning. In: 2018 18th IEEE winter conference on applications of computer vision (WACV), pp 323–332. https://doi.org/10.1109/wacv.2018.00042

  6. Momeny M, Jahanbakhshi A, Jafarnezhad K, Zhang YD (2020) Accurate classification of cherry fruit using deep cnn based on hybrid pooling approach. Postharvest Biol Technol 166:111204. https://doi.org/10.1016/j.postharvbio.2020.111204

  7. Suresh G, Gnanaprakash V, Santhiya R (2019) Performance analysis of different CNN architecture with different optimisers for plant disease classification. In: 2019 5th international conference on advanced computing and communication systems (ICACCS), Coimbatore, India. 916–921. https://doi.org/10.1109/ICACCS.2019.8728282

  8. Khaki S, Wang L, Archontoulis SV (2020) A cnn-rnn framework for crop yield prediction. Front Plant Sci. https://doi.org/10.3389/fpls.2019.01750

  9. Sun J, Di L, Sun Z, Shen Y, Lai Z (2019) County-level soybean yield prediction using deep cnn-lstm model. Sensors 19(20):4363. https://doi.org/10.3390/s19204363

    Article  PubMed Central  Google Scholar 

  10. Yang Q, Shi L, Lin L (2019) Plot-scale rice grain yield estimation using UAV-based remotely sensed images via CNN with time-invariant deep features decomposition. In: IGARSS 2019–2019 IEEE international geoscience and remote sensing symposium, Yokohama, Japan, 7180–7183. https://doi.org/10.1109/IGARSS.2019.8898061

  11. Luna RG de, Dadios EP, Bandala AA, Vicerra RRP (2020) J Agric Sci 42:24–36. https://doi.org/10.17503/agrivita.v42i1.2499

  12. Zhuang S, Wang P, Jiang B, Li M (2020) Learned features of leaf phenotype to monitor maize water status in the fields. Comput Electron Agric 172. https://doi.org/10.1016/j.compag.2020.105347

  13. Kovalchuk N, Laga H, Cai J, Kumar P, Parent B, Lu Z et al (2017) Phenotyping of plants in competitive but controlled environments: a study of drought response in transgenic wheat. Funct Plant Biol 44(3):290. https://doi.org/10.1071/FP16202

    Article  PubMed  Google Scholar 

  14. Misra T, Arora A, Marwaha S, Chinnusamy V, Goel S (2020) Spikesegnet-a deep learning approach utilizing encoder-decoder network with hourglass for spike segmentation and counting in wheat plant from visual imaging. Plant Methods. https://doi.org/10.1186/s13007-020-00582-9

  15. Qiongyan L, Cai J, Berger B, Okamoto M, Miklavcic SJ (2017) Detecting spikes of wheat plants using neural networks with laws texture energy. Plant Methods 13(1):83. https://doi.org/10.1186/s13007-017-0231-1

    Article  PubMed  PubMed Central  Google Scholar 

  16. Pound MP, Atkinson JA, Wells DM, Pridmore TP, French AP (2017) Deep learning for multi-task plant phenotyping. In: 2017 IEEE international conference on computer vision workshops (ICCVW). IEEE

    Google Scholar 

  17. Chandra AL, Desai SV, Balasubramanian VN, Ninomiya S, Guo W (2020) Active learning with point supervision for cost-effective panicle detection in cereal crops. Plant Methods 16(1). https://doi.org/10.1186/s13007-020-00575-8

  18. David E, Madec S, Sadeghi-Tehran P, Aasen H, Zheng B, Liu S et al (2020) Global wheat head detection (gwhd) dataset: a large and diverse dataset of high resolution rgb labelled images to develop and benchmark wheat head detection methods. Plant Phenomics. https://doi.org/10.34133/2020/3521852

  19. Hasan MM, Chopin JP, Laga H, Miklavcic SJ (2018) Detection and analysis of wheat spikes using convolutional neural networks. Plant Methods 14(1):100. https://doi.org/10.1186/s13007-018-0366-8

    Article  PubMed  PubMed Central  Google Scholar 

  20. Sadeghi-Tehran P, Virlet N, Ampe EM, Reyns P, Hawkesford MJ (2019) Deepcount: in-feld automatic quantifcation of wheat spikes using simple linear iterative clustering and deep convolutional neural networks. Front Plant Sci 10:1176. https://doi.org/10.3389/fpls.2019.01176

    Article  PubMed  PubMed Central  Google Scholar 

  21. Xu X, Li H, Yin F, Xi L, Ma X (2020) Wheat ear counting using k-means clustering segmentation and convolutional neural network. Plant Methods 16(1). https://doi.org/10.1186/s13007-020-00648-8

  22. Zhang D, Wang D, Gu C, Jin N, Liang D (2019) Using neural network to identify the severity of wheat fusarium head blight in the field environment. Remote Sens 11(20):2375. https://doi.org/10.1016/10.3390/rs11202375

    Article  Google Scholar 

  23. Alkhudaydi T, Reynolds D, Griffiths S, Zhou J, Iglesia BDL (2019) An exploration of deep-learning based phenotypic analysis to detect spike regions in field conditions for uk bread wheat. Plant Phenomics (1, article no. 45):1–17. https://doi.org/10.34133/2019/7368761

  24. Tan C, Zhang P, Zhang Y, Zhou X, Wang Z, Du Y, Mao W, Li W, Wang D, Guo W (2020) Rapid recognition of field-grown wheat spikes based on a Superpixel segmentation algorithm using digital images. Front Plant Sci 11:259. https://doi.org/10.3389/fpls.2020.00259

    Article  PubMed  PubMed Central  Google Scholar 

  25. Ma J, Li Y, Liu H, Du K, Zhang L (2020) Improving segmentation accuracy for ears of winter wheat at flowering stage by semantic segmentation. Comput Electron Agric 176:105662.https://doi.org/10.1016/j.compag.2020.105662

  26. Chen K, Pang J, Wang J, Xiong Y, Li X, Sun S et al (2019) Hybrid task cascade for instance segmentation. https://doi.org/10.1109/CVPR.2019.00511

  27. Li X, Liu Z, Luo P, Loy CC, Tang X (2017) Not all pixels are equal: difficulty-aware semantic segmentation via deep layer cascade.https://doi.org/10.1109/CVPR.2017.684

  28. Qiu R, Yang C, Moghimi A, Zhang M, Steffenson BJ, Hirsch CD (2019) Detection of fusarium head blight in wheat using a deep neural network and color imaging. Remote Sens 11:2658. https://doi.org/10.20944/preprints201910.0056.v1

  29. Cai Z, Vasconcelos N (2019) Cascade r-cnn: high quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2956516

    Article  PubMed  Google Scholar 

  30. Kaiming H, Georgia G, Piotr D, Ross G (2017) Mask r-cnn. In: IEEE international conference on computer vision. https://doi.org/10.1109/TPAMI.2018.2844175

  31. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision–ECCV 2014. Springer International Publishing, Cham, Switzerland, pp 740–755

    Google Scholar 

  32. Russell BC, Torralba A, Murphy KP, Freeman WT (2008) Labelme: a database and web-based tool for image annotation. Int J Comput Vis 77(1–3). https://doi.org/10.1007/s11263-007-0090-8

  33. Gao S, Cheng MM, Zhao K, Zhang XY, Torr PHS (2019) Res2net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell PP(99):1–1. https://doi.org/10.1109/TPAMI.2019.2938758

  34. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://arxiv.org/abs/1512.03385

  35. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated Residual Transformations for Deep Neural Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. arXiv:1611.05431

  36. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. https://doi.org/10.1109/ICCV.2017.89

  37. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6). https://doi.org/10.1109/TPAMI.2016.2577031

  38. Rossi L, Karimi A, Prati A (2020) A novel region of interest extraction layer for instance segmentation. arXiv:2004.13665

  39. Pont-Tuset J, Pablo A, Barron JT, Marques F, Malik J (2016) Multiscale combinatorial grouping for image segmentation and object proposal generation. IEEE Trans Pattern Anal Mach Intell 39(1). https://doi.org/10.1186/10.1109/TPAMI.2016.2537320

  40. Caruana R (1998) Multitask learnin. https://doi.org/10.1007/978-1-4615-5529-2_5

  41. Rosenfield A, Thurston M, Lee YH (1971) Edge and curve detection for visual scene analysis. IEEE Trans Comput C 20(5):562–569. https://doi.org/10.1109/T-C.1971.223290

  42. Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-nms—improving object detection with one line of code. arXiv:1704.04503

  43. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. https://doi.org/10.1007/978-3-319-10578-9_23

  44. Deng J, Dong W, Socher R, Li LJ, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision & pattern recognition. https://doi.org/10.1109/CVPR.2009.5206848

  45. Su WH, Zhang J, Yang C, Page R, Steffenson BJ (2021) Automatic evaluation of wheat resistance to fusarium head blight using dual mask-rcnn deep learning frameworks in computer vision. Remote Sens 13(1):1–21. https://doi.org/10.3390/rs13010026

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the USDA-ARS United States Wheat and Barley Scab Initiative (grant numbers 58-5062-8-018), the Lieberman-Okinow Endowment at the University of Minnesota, and the State of Minnesota Small Grains Initiative.

Author information

Authors and Affiliations

  1. College of Information and Electronics Engineering, Institute of Automation at Chinese Academy of Science, China Agricultural University, Beijing, 100083, China

    JiaJing Zhang

  2. Department of Bioproducts and Biosystems Engineering, University of Minnesota, Saint Paul, MN, 55108, USA

    An Min & Ce Yang

  3. Department of Plant Pathology, University of Minnesota, Saint Paul, MN, 55108, USA

    Brian J. Steffenson & Cory D. Hirsch

  4. College of Engineering, Department of Bioproducts and Biosystems Engineering, China Agricultural University, University of Minnesota, Saint Paul, MN, 55108, USA

    Wenhao Su

  5. Department of Agronomy and Plant Genetics, University of Minnesota, Saint Paul, MN, 55108, USA

    James Anderson

  6. College of Information and Electronics Engineering, China Agricultural University, Beijing, 100083, China

    Jian Wei

Authors
  1. JiaJing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. An Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian J. Steffenson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenhao Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cory D. Hirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. James Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ce Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ce Yang .

Editor information

Editors and Affiliations

  1. Key Lab of Smart Agriculture Systems, Ministry of Education, China Agricultural University, Beijing, China

    Zhao Zhang

  2. Chinese Academy of Sciences, Northwest Institute of Eco-Environment and Resources, Lanzhou, Gansu, China

    Hu Liu

  3. Department of Bioproducts and Biosystems Engineering, University of Minnesota, St. Paul, MN, USA

    Ce Yang

  4. UF/IFAS Southwest Florida Research and Education Center, University of Florida, Immokalee, FL, USA

    Yiannis Ampatzidis

  5. University of Missouri, Columbia, MO, USA

    Jianfeng Zhou

  6. School of Integrative Plant Science, Cornell University, Geneva, NY, USA

    Yu Jiang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, J. et al. (2022). Wheat-Net: An Automatic Dense Wheat Spike Segmentation Method Based on an Optimized Hybrid Task Cascade Model. In: Zhang, Z., Liu, H., Yang, C., Ampatzidis, Y., Zhou, J., Jiang, Y. (eds) Unmanned Aerial Systems in Precision Agriculture. Smart Agriculture, vol 2. Springer, Singapore. https://doi.org/10.1007/978-981-19-2027-1_6

Download citation

Publish with us

Policies and ethics

Search

Navigation