Skip to main content
SpringerLink
Log in

A novel and high precision tomato maturity recognition algorithm based on multi-level deep residual network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Since the existing tomato picking system uses multispectral sensors, color and other passive sensors for tomato detection and recognition, its detection range is very small, anti-interference ability is also weak, and tomato maturity detection cannot be performed accurately in real-time. How to detect tomato information from the massive image data obtained from tomato picking equipment and improve the recognition accuracy is a challenging research topic at home and abroad. This paper proposes an improved DenseNet deep neural network architecture, and uses it to solve the detection problems of maturity tomato in complex images. In order to enhance the accuracy of feature propagation and reduce the amount of stored data, a structured sparse operation is proposed. By dividing the network convolution kernel into multiple groups, the unimportant parameter connections in each group are gradually reduced during the network training process. In addition, since the dataset constructed in the field of tomato picking has imbalance, we introduce the Focal loss function to identify the tomato in the classification layer so as to enhance the accuracy of the final classification prediction of the tomato detection system. A large number of qualitative and quantitative experiments show that our improved network in this paper is superior to other existing deep models in terms of detection rate and FPPI, and its computational complexity is lower than that of DenseNet algorithm 18% under the same hardware and software configuration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahlin K et al (2016) Autonomous leaf picking using deep learning and visual-Servoing. IFAC-PapersOnLine 49.16:177–183

    Article  Google Scholar 

  2. Alvaro F et al (2017) A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 17.9:2022

    Google Scholar 

  3. Brahimi M, Boukhalfa K, Moussaoui A (2017) Deep learning for tomato diseases: classification and symptoms visualization. Appl Artif Intell :1–17

  4. Cai, Ziyun, et al. () RGB-D datasets using microsoftkinect or similar sensors: a survey. Multimed Tools Appl 76.3 (2017):4313–4355.

  5. Chen TH, Wu PH, Chiou YC (2005) An early fire-detection method based on image processing [C]// international conference on image processing. IEEE(3): 1707–1710

  6. Dimatira J, Baez U et al. (2017) Application of fuzzy logic in recognition of tomato fruit maturity in smart farming. Region 10 Conference IEEE

  7. Dimitropoulos K, Barmpoutis P, Grammalidis N (2015) Spatio-temporal flame modeling and dynamic texture detection analysis for automatic video-based fire[J]. IEEE Trans Circ Syst Video Technol 25(2):339–351

    Article  Google Scholar 

  8. Ernfjäll J, Stigsohn G (2018) Fire detection system[J]. Optoelectronics Instrum Data Process 45(2):140–145

    Google Scholar 

  9. Grzeszick R et al. (2017) Deep neural network based human activity recognition for the order picking process.. Proceedings of the 4th international workshop on sensor-based activity recognition and interaction :1–6

  10. Gubbi J, Marusic S, Palaniswami M (2009) Smoke detection in video using wavelets and support vector machines[J]. Fire Saf J 44(8):1110–1115

    Article  Google Scholar 

  11. Han L et al. (2017) Green ripe tomato detection method based on machine vision in greenhouse. Transactions of the Chinese Society of Agricultural Engineering

  12. Huang L et al. (2017) Densely connected convolutional networks//IEEE The IEEE Conference on Computer Vision and Pattern Recognition (CPVR) Las Vegas IEEE :4700–4708

  13. Listyorini T (2018) A prototype fire detection implemented using the internet of things and fuzzy logic[J]. World Trans Eng Technol Educ 16(1):42–46

    Google Scholar 

  14. Liu W, Anguelov D, Erhan D et al. (2016) SSD: single shot MultiBox detector[C]. European conference on computer vision : 21–37

  15. Misra A, Gupta A, Hebert M (2017) [IEEE 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) - Honolulu, HI (2017.7.21–2017.7.26)] 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) - From Red Wine to Red Tomato: Composition with Context." IEEE Conference on Computer Vision & Pattern Recognition IEEE Computer Society : 1160–1169

  16. Sari YA, Adinugroho S (2017) Tomato ripeness clustering using 6-means algorithm based on v-channel otsu segmentation. International Symposium on Computational & Business Intelligence IEEE

  17. Schwarz M et al. (2017) NimbRo picking: versatile part handling for warehouse automation. IEEE International Conference on Robotics & Automation IEEE

  18. Sermanet SC, LeCun Y (2012) Convolutional neural networks applied to house numbers digit classification. ICPR, IEEE 5:3288–3291

    Google Scholar 

  19. Shimoda W, Yanai K (2015) CNN-based food image segmentation without pixel-wise annotation[J] 9281:449–457

  20. Szegedy VV, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. CVPR 2

  21. Tian L, Wang J, Zhou H et al (2018) Automatic detection of forest fire disturbance based on dynamic modelling from MODIS time-series observations[J]. Int J Remote Sens 39(12):3801–3815

    Article  Google Scholar 

  22. Toreyin DY et al (2006) Computer vision based method for real-time fire and flame detection.[J]. Pattern Recogn Lett 27(1):49–58

    Article  Google Scholar 

  23. Wada K et al. (2017) 3D object segmentation for shelf bin picking by humanoid with deep learning and occupancy voxel grid map. IEEE-RAS international conference on humanoid robots IEEE

  24. Wang L et al (2017) Target extraction method of ripe tomato in greenhouse based on Niblack self-adaptive adjustment parameter. Trans Chin Soc Agric Eng 33:322–327

    Google Scholar 

  25. Xie S, Girshick R B, Dollar P, et al. (2017) Aggregated residual transformations for deep neural networks[C]. Computer Vision and Pattern Recognition : 5987–5995

  26. Yuncheng Z et al (2017) Classification and recognition approaches of tomato main organs based on DCNN. Trans Chin Soc Agric Eng 3315:219–226

    Google Scholar 

  27. Zeng A et al. (2016) Multi-view self-supervised deep learning for 6D pose estimation in the Amazon picking challenge

  28. Zhang L et al (2018) Deep learning based improved classification system for designing tomato harvesting robot. IEEE Access 99:1–1

    Google Scholar 

Download references

Acknowledgements

This work was financially supported by Independent innovation fund of agricultural science and technology in Jiangsu (No. CX(16)1002).

Author information

Authors and Affiliations

  1. Institute of Agricultural Facilities and Equipment, Jiangsu Academy of Agricultural Sciences, Key Laboratory of Protected Agriculture Engineering in the Middle and Lower Reaches of Yangtze River, Ministry of Agriculture, Nanjing, 210014, China

    Jun Liu, Jie Pi & Liru Xia

Authors
  1. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Pi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liru Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liru Xia.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Pi, J. & Xia, L. A novel and high precision tomato maturity recognition algorithm based on multi-level deep residual network. Multimed Tools Appl 79, 9403–9417 (2020). https://doi.org/10.1007/s11042-019-7648-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7648-7

Keywords

Search

Navigation