Skip to main content
SpringerLink
Log in

Crop classification using aerial images by analyzing an ensemble of DCNNs under multi-filter & multi-scale framework

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The rapid collection and development of multimedia data and devices allows agricultural monitoring to be automated. Crop classification using aerial images is the challenge of identifying the appropriate crop type planted on a certain area of land. These approaches, however, have several drawbacks that may inhibit the performance owing to a lack of appropriate data and the lack sufficient investigation of automatic feature extraction techniques. To deal with this, an ensemble approach has been investigated using two different deep convolutional neural networks (DCNN), namely Multi-Filter Multi-Scale deep convolutional neural network (MFMS-DCNN) and a pre-trained Inception V3 architecture, for crop classification using aerial images. At the onset, the MFMS-DCNN is constructed using Convolution- batchnormalization-activation units along with max and average pooling layers. Here, input images have been down-sampled using the max- pooling operation instead of traditional image processing approaches which facilitates to extract the important features in multiple levels, and the feature maps are concatenated by embedding a step-by-step fusion approach. The pre-trained Inception V3 model is initially fine-tuned using the three datasets under consideration (publicly available Plant seedling, and two new aerial image datasets obtained from two regions across India.). Finally, two combiners (Mean rule and Product rule) are enforced to ensemble the outputs of MFMS-DCNN and pre-trained Inception V3, to give the final prediction. To validate the feasibility of the proposed method, experimentations have been performed on the considered datasets and with state-of-the-art techniques. The findings derived from these image datasets produce superior performance for the proposed schemes as compared to the state-of-the-art techniques. Apart from that, with average accuracy ≈9% to ≈99% for all datasets, the ensemble methods with both combiners are proven to be more efficient than the two individual proposed schemes (MFMS-DCNN and fine-tuned Inception V3).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Code Availability

The developed code will be made available from the corresponding author.

References

  1. Alimboyong C R, Hernandez A A (2019) An improved deep neural network for classification of plant seedling images. In: International colloquium on signal processing & its applications (CSPA). IEEE, pp 217–222

  2. Amara J, Bouaziz B, Algergawy A, et al. (2017) A deep learning-based approach for banana leaf diseases classification. In: BTW (Workshops), pp 79–88

  3. Bansal M, Kumar M, Kumar M (2021) 2d object recognition: a comparative analysis of sift, surf and orb feature descriptors. Multimed Tools Applic 80(12):18839–18857

    Article  Google Scholar 

  4. Bansal M, Kumar M, Kumar M (2021) 2d object recognition techniques: state-of-the-art work. Arch Comput Methods Eng 28(3):1147–1161

    Article  MathSciNet  Google Scholar 

  5. Bansal M, Kumar M, Kumar M, Kumar K (2021) An efficient technique for object recognition using shi-tomasi corner detection algorithm. Soft Comput 25(6):4423–4432

    Article  Google Scholar 

  6. Bansal M, Kumar M, Sachdeva M, Mittal A (2021) Transfer learning for image classification using vgg19: Caltech-101 image data set. J Ambient Intell Humaniz Comput, 1–12

  7. Bargoti S, Underwood J (2017) Deep fruit detection in orchards. In: 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3626–3633

  8. Biradar C M, Xiao X (2011) Quantifying the area and spatial distribution of double-and triple-cropping croplands in india with multi-temporal modis imagery in 2005. Int J Remote Sens 32(2):367–386

    Article  Google Scholar 

  9. Chen S W, Shivakumar S S, Dcunha S, Das J, Okon E, Qu C, Taylor C J, Kumar V (2017) Counting apples and oranges with deep learning: a data-driven approach. IEEE Robot Autom Lett 2(2):781–788

    Article  Google Scholar 

  10. Cheng G, Ma C, Zhou P, Yao X, Han J (2016) Scene classification of high resolution remote sensing images using convolutional neural networks. In: International geoscience and remote sensing symposium (IGARSS), vol 2016-Novem. IEEE, pp 767–770

  11. Chew R, Rineer J, Beach R, ONeil M, Ujeneza N, Lapidus D, Miano T, Hegarty-Craver M, Polly J, Temple D S (2020) Deep neural networks and transfer learning for food crop identification in uav images. Drones 4(1):7

    Article  Google Scholar 

  12. Du C, Gao S (2017) Image segmentation-based multi-focus image fusion through multi-scale convolutional neural network. IEEE access 5:15750–15761

    Article  Google Scholar 

  13. Dyrmann M, Jørgensen R N, Midtiby H S (2017) Roboweedsupport-detection of weed locations in leaf occluded cereal crops using a fully convolutional neural network. Adv Anim Biosci 8(2):842–847

    Article  Google Scholar 

  14. Dyrmann M, Karstoft H, Midtiby H S (2016) Plant species classification using deep convolutional neural network. Biosys Eng 151:72–80

    Article  Google Scholar 

  15. Ferentinos K P (2018) Deep learning models for plant disease detection and diagnosis. Comput Electron Agric 145:311–318

    Article  Google Scholar 

  16. Giselsson T M, Jørgensen R N, Jensen P K, Dyrmann M, Midtiby H S (2017) A public image database for benchmark of plant seedling classification algorithms. arXiv:1711.05458

  17. Grinblat G L, Uzal L C, Larese M G, Granitto P M (2016) Deep learning for plant identification using vein morphological patterns. Comput Electron Agric 127:418–424

    Article  Google Scholar 

  18. Hall D, McCool C, Dayoub F, Sunderhauf N, Upcroft B (2015) Evaluation of features for leaf classification in challenging conditions. In: 2015 IEEE Winter conference on applications of computer vision, pp 797–804

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition (CVPR), pp 770–778

  20. Hu J, Chen Z, Yang M, Zhang R, Cui Y (2018) A multiscale fusion convolutional neural network for plant leaf recognition. IEEE Signal Process Lett 25(6):853–857

    Article  Google Scholar 

  21. Huang Z, Pan Z, Lei B (2017) Transfer learning with deep convolutional neural network for sar target classification with limited labeled data. Remote Sens, 9(9)

  22. Imani M, Ghassemian H (2015) Feature extraction using weighted training samples. IEEE Geosci Remote Sens Lett 12(7):1387–1386

    Article  Google Scholar 

  23. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  24. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: International conference on multimedia, pp 675–678

  25. Kalita I, Chakraborty S, Roy M (2019) Deep ensemble network for handling class-imbalance problem in land-cover classification. In: 2019 International conference on information technology (ICIT). IEEE, pp 505–509

  26. Kalita I, Roy M (2020) Deep neural network-based heterogeneous domain adaptation using ensemble decision making in land cover classification. IEEE Trans Artif Intell 1(2):167–180

    Article  Google Scholar 

  27. Kamilaris A, Prenafeta-Boldú F X (2018) Deep learning in agriculture: a survey. Comput Electron Agri 147:70–90

    Article  Google Scholar 

  28. Kingma D P, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  29. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, vol 1. ACM, pp 1097–1105

  30. Kumar K (2019) Evs-dk: Event video skimming using deep keyframe. J Vis Commun Image Represent 58:345–352

    Article  Google Scholar 

  31. Kumar K, Shrimankar D D (2018) Deep event learning boost-up approach: delta. Multimed Tools Applic 77(20):26635–26655

    Article  Google Scholar 

  32. Kumar M, Kumar M, et al. (2021) Performance comparison of various feature extraction methods for object recognition on caltech-101 image dataset. In: Applications of artificial intelligence and machine learning. Springer, pp 289–303

  33. Kumar M, Kumar M, et al. (2021) Xgboost: 2d-object recognition using shape descriptors and extreme gradient boosting classifier. In: Computational methods and data engineering. Springer, pp 207–222

  34. Kung H-Y, Kuo T-H, Chen C-H, Tsai P-Y (2016) Accuracy analysis mechanism for agriculture data using the ensemble neural network method. Sustainability 8(8):735

    Article  Google Scholar 

  35. Kussul N, Lavreniuk M, Skakun S, Shelestov A (2017) Deep learning classification of land cover and crop types using remote sensing data. IEEE Geosci Remote Sens Lett 14(5):778–782

    Article  Google Scholar 

  36. Kuwata K, Shibasaki R (2015) Estimating crop yields with deep learning and remotely sensed data. In: 2015 IEEE International geoscience and remote sensing symposium (IGARSS). IEEE, pp 858–861

  37. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  38. Lee S H, Chan C S, Wilkin P, Remagnino P (2015) Deep-plant: plant identification with convolutional neural networks. In: 2015 IEEE International conference on image processing (ICIP), pp 452–456

  39. Liang P, Shi W, Zhang X (2018) Remote sensing image classification based on stacked denoising autoencoder. Rem Sens 10(1):16

    Google Scholar 

  40. Lv Q, Dou Y, Niu X, Xu J, Xu J, Xia F (2015) Urban land use and land cover classification using remotely sensed sar data through deep belief networks. Journal of Sensors, 2015

  41. Mahdianpari M, Rezaee M, Zhang Y, Salehi B (2018) Wetland classification using deep convolutional neural network. In: International geoscience and remote sensing symposium (IGARSS). IEEE, pp 9249–9252

  42. McCool C, Perez T, Upcroft B (2017) Mixtures of lightweight deep convolutional neural networks: applied to agricultural robotics. IEEE Robot Autom Lett 2(3):1344–1351

    Article  Google Scholar 

  43. Milioto A, Lottes P, Stachniss C (2017) Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks. ISPRS Annals of the photogrammetry, remote sensing and spatial information sciences 4:41

    Article  Google Scholar 

  44. Mohanty S P, Hughes D P, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419

    Article  Google Scholar 

  45. Mortensen A K, Dyrmann M, Karstoft H, Jørgensen R N, Gislum R, et al. (2016) Semantic segmentation of mixed crops using deep convolutional neural network.. In: CIGR-AgEng Conference, 26-29 June 2016, Aarhus, Denmark. Abstracts and Full papers. Organising Committee, CIGR 2016, pp 1–6

  46. Negi A, Kumar K (2021) Classification and detection of citrus diseases using deep learning. In: Data science and its applications. Chapman and Hall/CRC, pp 63–85

  47. Negi A, Kumar K, Chauhan P (2021) Deep neural network-based multi-class image classification for plant diseases. Agricultural Informatics: automation using the IoT and machine learning, 117–129

  48. Niculescu S, Ienco D, Hanganu J (2018) Application of deep learning of multi-temporal Sentinel-1 images for the classification of coastal vegetation zone of the danube delta. In: International archives of the photogrammetry, remote sensing and spatial information sciences (ISPRS), vol 42. ISPRS, pp 1311–1318

  49. Pantazi X E, Moshou D, Alexandridis T, Whetton R L, Mouazen A M (2016) Wheat yield prediction using machine learning and advanced sensing techniques. Comput Electron Agric 121:57–65

    Article  Google Scholar 

  50. Penatti Otavio AB, Nogueira K, Dos Santos J A (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?. In: International conference on computer vision and pattern recognition workshops, pp 44–51

  51. Postadjian T, Le-Bris A, Sahbi H, Malle C (2018) Domain adaptation for large scale classification of very high resolution satellite images with deep convolutional neural networks. In: International geoscience and remote sensing symposium (IGARSS). IEEE, pp 3623–3626

  52. Potena C, Nardi D, Pretto A (2016) Fast and accurate crop and weed identification with summarized train sets for precision agriculture. In: International conference on intelligent autonomous systems, pp 105–121

  53. Rahnemoonfar M, Sheppard C (2017) Deep count: fruit counting based on deep simulated learning. Sensors 17(4):905

    Article  Google Scholar 

  54. Ramos PJ, Prieto F A, Montoya EC, Oliveros C E (2017) Automatic fruit count on coffee branches using computer vision. Comput Electron Agric 137:9–22

    Article  Google Scholar 

  55. Rebetez J, Satizábal H F, Mota M, Noll D, Büchi L, Wendling M, Cannelle B, Pérez-Uribe A, Burgos S (2016) Augmenting a convolutional neural network with local histograms-a case study in crop classification from high-resolution uav imagery. In: ESANN

  56. Reyes A K, Caicedo J C, Camargo J E (2015) Fine-tuning deep convolutional networks for plant recognition. CLEF (Working Notes), 1391

  57. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  58. Sa I, Ge Z, Dayoub F, Upcroft B, Perez T, McCool C (2016) Deepfruits: a fruit detection system using deep neural networks. Sensors 16(8):1222

    Article  Google Scholar 

  59. Sameen M I, Pradhan B, Aziz O S (2018) Classification of very high resolution aerial photos using spectral-spatial convolutional neural networks. Journal of Sensors, 2018

  60. Scott G J, Marcum R A, Davis C H, Nivin T W (2017) Fusion of deep convolutional neural networks for land cover classification of high-resolution imagery. IEEE Geosci Remote Sens Lett 14(9):1638–1642

    Article  Google Scholar 

  61. Sengupta S, Lee W S (2014) Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosys Eng 117:51–61

    Article  Google Scholar 

  62. Senthilnath J, Dokania A, Kandukuri M, Ramesh KN, Anand G, Omkar SN (2016) Detection of tomatoes using spectral-spatial methods in remotely sensed rgb images captured by uav. Biosyst Eng 146:16–32

    Article  Google Scholar 

  63. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  64. Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D (2016) Deep neural networks based recognition of plant diseases by leaf image classification. Computational Intelligence and Neuroscience, 2016

  65. Sørensen R A, Rasmussen J, Nielsen J, Jørgensen R N (2017) Thistle detection using convolutional neural networks. In: 2017 EFITA WCCA CONGRESS, p 161

  66. Su Y-, Xu H, Yan L- (2017) Support vector machine-based open crop model (sbocm): case of rice production in china. Saudi J Biol Sci 24(3):537–547

    Article  Google Scholar 

  67. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: International conference on computer vision and pattern recognition, pp 2818–2826

  68. Tran T, Choi J, Le T H, Kim J (2019) A comparative study of deep cnn in forecasting and classifying the macronutrient deficiencies on development of tomato plant. Appl Sci 9(8):1601

  69. Xinshao W, Cheng C (2015) Weed seeds classification based on pcanet deep learning baseline. In: 2015 Asia-Pacific signal and information processing association annual summit and conference (APSIPA), pp 408–415

Download references

Funding

This work is supported by a project-grant (with file number: ECR/2016/001227) from Science and Engineering Research Board, Department of Science and Technology, Government of India.

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Indian Institute of Information Technology Guwahati, Bongora, Guwahati, 781015, Assam, India

    Indrajit Kalita, Gyan Prakash Singh & Moumita Roy

Authors
  1. Indrajit Kalita
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gyan Prakash Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moumita Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Moumita Roy.

Ethics declarations

Competing interests

The authors declare that they have no known competing interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Availability of data and material

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalita, I., Singh, G.P. & Roy, M. Crop classification using aerial images by analyzing an ensemble of DCNNs under multi-filter & multi-scale framework. Multimed Tools Appl 82, 18409–18433 (2023). https://doi.org/10.1007/s11042-022-13946-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13946-1

Keywords

Search

Navigation