Skip to main content
SpringerLink
Log in

Counting spikelets from infield wheat crop images using fully convolutional networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Wheat is one of the world’s three main crops, with global consumption projected to reach more than 850 million tons by 2050. Stabilising yield and quality of wheat cultivation is a major issue. With the use of remote sensing and non-invasive imaging technology, the Internet of things (IoT) has allowed us to constantly monitor crop development in agriculture. The output of such technologies may be analysed using machine-learning algorithms and image processing methods to extract useful information for crop management assistance. Counting wheat spikelets from infield images is considered one of the challenges related to estimating yield traits of wheat crops. For this challenging problem, we propose a density estimation approach related to crowd counting, SpikeCount. Our proposed counting methods are based on deep learning architectures as those have the advantage of being able to identify features automatically. Annotation of images with the ground truth is required for machine learning approaches, but those are expensive in terms of time and resources. We use transfer Learning in both tasks, segmentation and counting. Our results indicate the segmentation is beneficial as focusing only on the regions of interest improves counting accuracy in most scenarios. In particular, transfer learning from similar images produced excellent results for the counting task for most of the stages of wheat development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

https://github.com/tanh86/SpikeProject2.

References

  1. Alkhudaydi T, Reynolds D, Griffiths S, Zhou J, De La Iglesia B (2019) An exploration of deep-learning based phenotypic analysis to detect spike regions in field conditions for UK bread wheat. Plant Phenomics 2019:7368761

    Article  Google Scholar 

  2. Alkhudaydi T, Zhou J, de la Iglesia B (2019) SpikeletFCN: counting spikelets from infield wheat crop images using fully convolutional networks. In: international conference on artificial intelligence and soft computing, pp 3–13. Springer

  3. Arteta C, Lempitsky V, Noble JA, Zisserman A (2014) Interactive object counting. In: European conference on computer vision, pp 504–518. Springer

  4. Arteta C, Lempitsky V, Zisserman A (2016) Counting in the wild. In: European conference on computer vision, pp 483–498. Springer

  5. Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. Neural networks: tricks of the trade. Springer, Berlin, pp 437–478

    Chapter  Google Scholar 

  6. Chan AB, John Liang ZS, Vasconcelos N (2008) Privacy preserving crowd monitoring: Counting people without people models or tracking. In: computer vision and pattern recognition, 2008. CVPR 2008. IEEE Conference on, pp 1–7. IEEE

  7. Change Loy C, Gong S, Xiang T (2008) From semi-supervised to transfer counting of crowds. In: proceedings of the IEEE international conference on computer vision (ICCV), pp 2256–2263, December x

  8. Chattopadhyay P, Vedantam R, Ramprasaath RS, Batra D, Parikh D (2016) Counting everyday objects in everyday scenes. arXiv preprint arXiv:1604.03505

  9. Cho SY, Chow TES, Leung CT (1999) A neural-based crowd estimation by hybrid global learning algorithm. IEEE Trans Syst, Man, Cybernet, Part B 29(4):535–541

    Article  Google Scholar 

  10. Cohen JP, Boucher G, Glastonbury A, Lo HZ, Bengio Y (2017) Count-ception: counting by fully convolutional redundant counting. In: computer vision workshop (ICCVW), 2017 IEEE international conference on, pp 18–26. IEEE

  11. Fernandez-Gallego JA, Kefauver SC, Gutiérrez NA, Nieto-Taladriz MT, Araus JL (2018) Wheat ear counting in-field conditions: high throughput and low-cost approach using rgb images. Plant Methods 14(1):22

    Article  Google Scholar 

  12. Fiaschi L, Koethe U, Nair R, Hamprecht FA (2012) Learning to count with regression forest and structured labels. In: pattern recognition (ICPR), 2012 21st international conference on, pp 2685–2688. IEEE

  13. French G, Fisher M, Mackiewicz M, Needle C (2015). Convolutional neural networks for counting fish in fisheries surveillance video. BMVA Press

  14. French G, Fisher M, Mackiewicz M, Needle C (2015) Uea computer vision - image labelling tool. https://bitbucket.org/ueacomputervision/image-labelling-tool

  15. Giuffrida MV, Minervini M, Tsaftaris SA (2015) Learning to count leaves in rosette plants. In: proceedings of the computer vision problems in plant phenotyping (CVPPP), pp 7–10 x

  16. Habibzadeh Mehdi, Krzyżak Adam, Fevens Thomas (2013) White blood cell differential counts using convolutional neural networks for low resolution images. Springer, Berlin, pp 263–274

    Google Scholar 

  17. Hasan MM, Chopin JP, Laga H, Miklavcic SJ (2018) Detection and analysis of wheat spikes using convolutional neural networks. Plant Methods 14(1):100

    Article  Google Scholar 

  18. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: proceedings of the IEEE international conference on computer vision, pp 1026–1034

  19. Hinton G, Srivastava N, Swersky K (2012) Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on, 14(8)

  20. Kong D, Gray D, Tao H (2006) A viewpoint invariant approach for crowd counting. In: pattern recognition, 2006. ICPR 2006. 18th international conference on, volume 3, pp 1187–1190. IEEE

  21. Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res, 10(1)

  22. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  23. Lempitsky V, Zisserman A (2010) Learning to count objects in images. In: advances in neural information processing systems, pp 1324–1332

  24. Li Y, Cui Z, Ni Y, Zheng M, Yang D, Jin M, Chen J, Wang Z, Yin Y (2016) Plant density effect on grain number and weight of two winter wheat cultivars at different spikelet and grain positions. PloS one 11(5):e0155351

    Article  Google Scholar 

  25. Lin Z, Davis LS (2010) Shape-based human detection and segmentation via hierarchical part-template matching. IEEE Trans Pattern Anal Mach Intell 32(4):604–618

    Article  Google Scholar 

  26. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  27. Hao Lu, Cao Zhiguo, Xiao Yang, Zhuang Bohan, Shen Chunhua (2017) Tasselnet: counting maize tassels in the wild via local counts regression network. Plant Methods 13(1):79

    Article  Google Scholar 

  28. Hao L, Liu L, Li Y-N, Zhao X-M, Wang X-Q, Cao Z-G (2021) Tasselnetv3: explainable plant counting with guided upsampling and background suppression. IEEE Trans Geosci Remote Sens 60:1–15

    Google Scholar 

  29. Madec S, Jin X, Hao L, De Solan B, Liu S, Duyme F, Heritier E, Baret F (2019) Ear density estimation from high resolution rgb imagery using deep learning technique. Agric For Meteorol 264:225–234

    Article  Google Scholar 

  30. University of Bristol. Wheat: The big picture, 2011. Bristol Wheat Genomics

  31. Pask AJD , Pietragalla J, Mullan DM, Reynolds MP (2012) Physiological breeding II: a field guide to wheat phenotyping. CIMMYT

  32. Pham VQ, Kozakaya T, Yamaguchi O, Okada R (2015) Count forest: co-voting uncertain number of targets using random forest for crowd density estimation. In: proceedings of the IEEE international conference on computer vision, pp 3253–3261

  33. Dave J P (2015) Picamera package. https://picamera.readthedocs.io/en/release-1.13/, x

  34. Pound MP, Atkinson JA, Wells DM, Pridmore TP, French AP (2017) Deep learning for multi-task plant phenotyping. In computer vision workshop (ICCVW), 2017 IEEE international conference on, pp 2055–2063. IEEE

  35. Rabaud V, Belongie S (2006) Counting crowded moving objects. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), volume 1, pp 705–711, June

  36. Ren M, Zemel RS (2017) End-to-end instance segmentation with recurrent attention. In CVPR

  37. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  38. Seguí S, Pujol O, Vitrià J (2015) Learning to count with deep object features. CoRR, abs/1505.08082

  39. Shaw LM, Turner AS, Herry L, Griffiths S, Laurie DA (2013) Mutant alleles of photoperiod-1 in wheat (triticum aestivum l.) that confer a late flowering phenotype in long days. PLoS One 8(11):e79459

    Article  Google Scholar 

  40. Shaw LM, Turner AS, Laurie DA (2012) The impact of photoperiod insensitive ppd-1a mutations on the photoperiod pathway across the three genomes of hexaploid wheat (triticum aestivum). Plant J 71(1):71–84

    Article  Google Scholar 

  41. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  42. Tardieu François, Cabrera-Bosquet Llorenc, Pridmore Tony, Bennett Malcolm (2017) Plant phenomics, from sensors to knowledge. Curr Biol 27(15):R770–R783

    Article  Google Scholar 

  43. Van der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, Gouillart E, Yu T (2014) scikit-image: image processing in python. Peer J 2:e453

    Article  Google Scholar 

  44. Wang M, Wang X (2011) Automatic adaptation of a generic pedestrian detector to a specific traffic scene. In CVPR 2011:3401–3408

    Google Scholar 

  45. Wang Y, Zou Y (2016) Fast visual object counting via example-based density estimation. In: 2016 IEEE international conference on image processing (ICIP), pp 3653–3657. IEEE

  46. Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. In: 10th IEEE international conference on computer vision (ICCV’05) Vol 1, pp 90–97, Oct

  47. Xu B, Qiu G (2016) Crowd density estimation based on rich features and random projection forest. In: 2016 IEEE winter conference on applications of computer vision (WACV), pp 1–8. IEEE

  48. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. In Adv Neural Inf Process Syst, pp 3320–3328

  49. Zadoks JC, Chang TT, Konzak CF et al (1974) A decimal code for the growth stages of cereals. Weed Res 14(6):415–421

    Article  Google Scholar 

  50. Zhang C, Li H, Wang X, Yang X (2015) Cross-scene crowd counting via deep convolutional neural networks. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 833–841

  51. Zhou J, Reynolds D, Corn TL, et al (2017) Cropquant: the next-generation automated field phenotyping platform for breeding and digital agriculture. BioRxiv, Sep

  52. Zhou J, Reynolds D, Griffiths S (2017) Cropquant: an automated and scalable field phenotyping platform for crop monitoring and trait measurements to facilitate breeding and digital agriculture. bioRxiv

Download references

Funding

Beatriz de la Iglesia received no funds, grants, or other support in relation to this work. Tahani Alkhudaydi was funded for her PhD studies by University of Tabuk.

Author information

Authors and Affiliations

  1. University of Tabuk, Tabuk, 71491, SA, Saudi Arabia

    Tahani Alkhudaydi

  2. University of East Anglia, Norwich Research Park, Norwich, NR4 7TJ, UK

    Beatriz De La lglesia

Authors
  1. Tahani Alkhudaydi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beatriz De La lglesia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tahani Alkhudaydi.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Code availability

Not applicable.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alkhudaydi, T., De La lglesia, B. Counting spikelets from infield wheat crop images using fully convolutional networks. Neural Comput & Applic 34, 17539–17560 (2022). https://doi.org/10.1007/s00521-022-07392-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07392-1

Keywords

Search

Navigation