Abstract
Computer vision finds wide range of applications in fruit processing industries, allowing the tasks to be done with automation. Classification of fruit’s quality and thereby gradation of the same is very important for the industry manufacture unit for production of best quality finished food products and the finest quality of the raw fruits to be sellable in the market. In the present paper, detection of rotten or fresh apple has been accomplished based on the defects present on the peel of the fruit. The work proposes a semantic segmentation of the rotten portion present in the apple’s RGB image based on deep learning architecture. UNet and a modified version of it, the Enhanced UNet (En-UNet) are implemented for segmentation yielding promising results. The proposed En-UNet model generated enhanced outputs than UNet with training and validation accuracies of 97.46% and 97.54% respectively while UNet as the base architecture attaining an accuracy of 95.36%. The best mean IoU score under a threshold of 0.95 attained by En-UNet is 0.866 while that of UNet is 0.66. The experimental results show that the proposed model is a better one to be used for segmentation, detection and categorization of the rotten or fresh apples in real time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Chen Y, Hou C, Tang Y, Zhuang J, Lin J, He Y, Luo S (2019) Citrus tree segmentation from UAV images based on monocular machine vision in a natural orchard environment. Sensors 19(24):5558
Dias PA, Tabb A, Medeiros H (2018) Multispecies fruit flower detection using a refined semantic segmentation network. IEEE Robot Autom Lett 3(4):3003–3010
Guan S, Khan AA, Sikdar S, Chitnis PV (2019)Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform 24(2):568–576
Häni N, Roy P, Isler V (2020) A comparative study of fruit detection and counting methods for yield mapping in apple orchards. J Field Robot 37(2):263–282
Jin S, Su Y, Gao S, Wu F, Ma Q, Xu K, Zhang J (2019) Separating the structural components of maize for field phenotyping using terrestrial lidar data and deep convolutional neural networks. IEEE Trans Geosci Remote Sens 58(4):2644–2658
Kang H, Chen C (2019) Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors 19(20):4599
Kang H, Chen C (2020) Fast implementation of real-time fruit detection in apple orchards using deep learning. Computers Electron Agricult 168:105108
Kumari N, Bhatt AK, Dwivedi RK, Belwal R (2020) Hybridized approach of image segmentation in classification of fruit mango using BPNN and discriminant analyzer. Multimedia Tools Applic 1–31
Li J, Lin X, Che H, Li H, Qian X (2019) Probability map guided bi-directional recurrent UNet for pancreas segmentation. arXiv preprint arXiv:1903.00923
Liu X, Zhao D, Jia W, Ji W, Ruan C, Sun Y (2019) Cucumber fruits detection in greenhouses based on instance segmentation. IEEE Access 7:139635–139642
Long J, Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440)
Luo Z, Zhang Y, Zhou L, Zhang B, Luo J, Wu H (2019) Micro-vessel image segmentation based on the AD-UNet model. IEEE Access 7:143402–143411
Majeed Y, Karkee M, Zhang Q, Fu L, Whiting MD (2020) Determining grapevine cordon shape for automated green shoot thinning using semantic segmentation-based deep learning networks. Computers Electron Agricult 171:105308
Mhapne NV, Harish SV, Kini AS, Narendra VG (2019) A comparative study to find an effective image segmentation technique using clustering to obtain the defective portion of an apple. In 2019 international conference on automation, computational and technology management (ICACTM) (pp. 304–309). IEEE
Nardari GV, Romero RA, Guizilini VC, Mareco WE, Milori DM, Villas-Boas PR, Santos IAD (2018) Crop anomaly identification with color filters and convolutional neural networks. In 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE) (pp. 363–369). IEEE
Ni X, Li C, Jiang H, Takeda F (2020) Deep learning image segmentation and extraction of blueberry fruit traits associated with harvestability and yield. Horticult Res 7(1):1–14
Ni X, Li C, Jiang H (2020) Blueberry harvestability trait extraction from 2D images and 3D point clouds based on deep learning and photogrammetric reconstruction. In 2020 ASABE Annual International Virtual Meeting (p. 1). American Society of Agricultural and Biological Engineers.
Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision (pp. 1520–1528)
Rong D, Xie L, Ying Y (2019) Computer vision detection of foreign objects in walnuts using deep learning. Computers Electron Agricult 162:1001–1010
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, pp.234–241
Roy K, Chaudhuri SS, Bhattacharjee S, Manna S, Chakraborty T (2019) Segmentation techniques for rotten fruit detection. In 2019 International Conference on Opto-Electronics and Applied Optics (Optronix) (pp. 1–4). IEEE
Roy K, Ghosh A, Saha D, Chatterjee J, Sarkar S, Chaudhuri SS (2019) Masking based Segmentation of Rotten Fruits. In 2019 International Conference on Opto-Electronics and Applied Optics (Optronix) (pp. 1–4). IEEE
Shafi ASM, Rahman MB, Rahman MM (2018) Fruit disease recognition and automatic classification using MSVM with multiple features. Int J ComputAppl 181(10):0975–8887
Wajid A, Singh NK, Junjun P, Mughal MA (2018) Recognition of ripe, unripe and scaled condition of orange citrus based on decision tree classification. In 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1–4). IEEE
www.kaggle.com/sriramr/fruits-fresh-and-rotten-for-classification
Yang C (2020) Colorful fruit image segmentation based on texture feature. Adv Intell Inform Hid Multimedia Sign Process. Springer, Singapore, pp 305–311
Yu Y, Zhang K, Yang L, Zhang D (2019) Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers Electron Agricul 163:104846
Zeng Z, Xie W, Zhang Y, Lu Y (2019) RIC-Unet: An improved neural network based on Unet for nuclei segmentation in histology images. Ieee Access 7:21420–21428
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, pp 3–11
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Roy, K., Chaudhuri, S.S. & Pramanik, S. Deep learning based real-time Industrial framework for rotten and fresh fruit detection using semantic segmentation. Microsyst Technol 27, 3365–3375 (2021). https://doi.org/10.1007/s00542-020-05123-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00542-020-05123-x