Abstract
Morphological features of fruit, such as size and shape, are essential in determining fruit quality. Given the limitations in accurately measuring precise morphological features using solely two-dimensional (2D) images, studies utilizing three-dimensional (3D) imaging techniques for measuring fruit morphology have been conducted. However, because of the time-consuming processes involved, measuring and processing 3D images in real time has thus far been impossible. Therefore, this study aimed to measure 3D images and extract the morphological features of fruits in real time. A measurement system with multiple RGB-D cameras was developed to enable real-time measurements by coordinate calibration among the cameras. Algorithms for real-time extraction of morphological features specific to oriental melon fruits were also developed. The prediction performances for the length, volume, and density of oriental melons showed determination coefficients of 0.9676, 0.9975, and 0.9057 and root-mean-squared errors of 2.08 mm, 3.77 cm3, and 10.73 kg/m3, respectively. In addition, predictive modeling was performed for their morphological grades by using parameters based on 3D morphology. The reference grade was determined by skilled workers at the processing center according to their produce classifying standards. The parameters were analyzed against their morphological grades, and the predictive model showed an accuracy of over 94%. The developed system and algorithms had a processing time of 40.28 ms for measuring and processing 3D images with an i5-16300KF CPU and 32 GB of RAM, indicating their potential application in phenotyping and as fruit-sorting machines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
References
Arjenaki, O. O., Moghaddam, P. A., & Motlagh, A. M. (2013). Online tomato sorting based on shape, maturity, size, and surface defects using machine vision. Turkish Journal of Agriculture and Forestry, 37(1), 62–68. https://doi.org/10.3906/tar-1201-10
Blasco, J., Aleixos, N., & Moltó, E. (2003). Machine vision system for automatic quality grading of fruit. Biosystems Engineering, 85(4), 415–423. https://doi.org/10.1016/S1537-5110(03)00088-6
Clement, J., Novas, N., Gazquez, J.-A., & Manzano-Agugliaro, F. (2012). High speed intelligent classifier of tomatoes by colour, size and weight. Spanish Journal of Agricultural Research, 10(2), 314–325. https://doi.org/10.5424/sjar/2012102-368-11
Crowe, T. G., & Delwiche, M. J. (1996). Real-time defect detection in fruit. I. Design concepts and development of prototype hardware. Transactions of the ASAE (USA), 39(6). https://doi.org/10.13031/2013.27740
Fadilah, N., Mohamad-Saleh, J., Halim, Z. A., Ibrahim, H., & Ali, S. S. S. (2012). Intelligent color vision system for ripeness classification of oil palm fresh fruit bunch. Sensors, 12(10), 14179–14195. https://doi.org/10.3390/s121014179
Feldmann, M. J., Hardigan, M. A., Famula, R. A., Lopez, C. M., Tabb, A., Cole, G. S., & Knapp, S. J. (2020). Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry. GigaScience, 9(5), giaa030. https://doi.org/10.1093/gigascience/giaa030
Fu, L., Sun, S., Li, R., & Wang, S. (2016). Classification of kiwifruit grades based on fruit shape using a single camera. Sensors, 16(7), 1012. https://doi.org/10.3390/s16071012
Ha, Y. S., & Kim, T. W. (2013). Design factor analysis of end-effector for oriental melon harvesting robot in greenhouse cultivation. Protected Horticulture and Plant Factory, 22(3), 284–290. https://doi.org/10.12791/KSBEC.2013.22.3.284
Hong, S.-J., Kim, S., Lee, C. H., Park, S., Kim, K.-C., Lee, A., & Kim, G. (2024). On-plant size and weight estimation of tomato fruits using deep neural networks and RGB-D imaging. Journal of the ASABE, 67(2), 439–450. https://doi.org/10.13031/ja.15746
Jadhav, T., Singh, K., & Abhyankar, A. (2019). Volumetric estimation using 3D reconstruction method for grading of fruits. Multimedia Tools and Applications, 78, 1613–1634. https://doi.org/10.1007/s11042-018-6271-3
Kavdir, I., & Guyer, D. E. (2003). Apple grading using fuzzy logic. Turkish Journal of Agriculture and Forestry, 27(6), 375–382. https://journals.tubitak.gov.tr/agriculture/vol27/iss6/8/
Kheiralipour, K., & Pormah, A. (2017). Introducing new shape features for classification of cucumber fruit based on image processing technique and artificial neural networks. Journal of Food Process Engineering, 40(6), e12558. https://doi.org/10.1111/jfpe.12558
Kim, S.-Y., Hong, S.-J., Kim, E., Lee, C.-H., & Kim, G. (2023). Application of ensemble neural-network method to integrated sugar content prediction model for citrus fruit using Vis/NIR spectroscopy. Journal of Food Engineering, 338, 111254. https://doi.org/10.1016/j.jfoodeng.2022.111254
Kim, E., Hong, S.-J., Kim, S.-Y., Lee, C.-H., Kim, S., Kim, H.-J., & Kim, G. (2022). CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques. Science and Reports, 12(1), 1–16. https://doi.org/10.1038/s41598-022-25260-9
Lefcourt, A. M., Narayanan, P., Tasch, U., Kim, M. S., Reese, D., Rostamian, R., & Lo, Y. M. (2009). Orienting apples for imaging using their inertial properties and random apple loading. Biosystems Engineering, 104(1), 64–71. https://doi.org/10.1016/j.biosystemseng.2009.06.002
Liming, X., & Yanchao, Z. (2010). Automated strawberry grading system based on image processing. Computers and Electronics in Agriculture, 71, S32–S39. https://doi.org/10.1016/j.compag.2009.09.013
Liu, J., Xu, X., Liu, Y., Rao, Z., Smith, M. L., Jin, L., & Li, B. (2021). Quantitative potato tuber phenotyping by 3D imaging. Biosystems Engineering, 210, 48–59. https://doi.org/10.1016/j.biosystemseng.2021.08.001
Mirbod, O., Choi, D., Heinemann, P. H., Marini, R. P., & He, L. (2023). On-tree apple fruit size estimation using stereo vision with deep learning-based occlusion handling. Biosystems Engineering, 226, 27–42. https://doi.org/10.1016/J.BIOSYSTEMSENG.2022.12.008
Nakano, K. (1997). Application of neural networks to the color grading of apples. Computers and Electronics in Agriculture, 18(2–3), 105–116. https://doi.org/10.1016/S0168-1699(97)00023-9
Narendra, V. G., & Hareesha, K. S. (2010). Quality inspection and grading of agricultural and food products by computer vision-a review. International Journal of Computers and Applications, 2(1), 43–65. https://doi.org/10.5120/612-863
Neupane, C., Koirala, A., Wang, Z., & Walsh, K. B. (2021). Evaluation of depth cameras for use in fruit localization and sizing: Finding a successor to kinect v2. Agronomy, 11(9), 1780. https://doi.org/10.3390/agronomy11091780
Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66. https://doi.org/10.1109/TSMC.1979.4310076
Parr, B., Legg, M., & Alam, F. (2022). Analysis of depth cameras for proximal sensing of grapes. Sensors, 22(11), 4179. https://doi.org/10.3390/s22114179
Pothula, A. K., Zhang, Z., & Lu, R. (2023). Evaluation of a new apple in-field sorting system for fruit singulation, rotation and imaging. Computers and Electronics in Agriculture, 208, 107789. https://doi.org/10.1016/j.compag.2023.107789
Saltveit, M. E. (2005). Fruit ripening and fruit quality. In Tomatoes (pp. 145–170). Cabi Publishing Wallingford UK. https://doi.org/10.1079/9780851993966.0145
Utai, K., Nagle, M., Hämmerle, S., Spreer, W., Mahayothee, B., & Müller, J. (2019). Mass estimation of mango fruits (Mangifera indica L., cv.‘Nam Dokmai’) by linking image processing and artificial neural network. Engineering in Agriculture, Environment and Food, 12(1), 103–110. https://doi.org/10.1016/j.eaef.2018.10.003
Venkatesh, V. G., Iqbal, S. M., Gopal, A., & Ganesan, D. (2015). Estimation of volume and mass of axi-symmetric fruits using image processing technique. International Journal of Food Properties, 18(3), 608–626. https://doi.org/10.1080/10942912.2013.831444
Ventura, M., de Jager, A., de Putter, H., & Roelofs, F. P. M. M. (1998). Non-destructive determination of soluble solids in apple fruit by near infrared spectroscopy (NIRS). Postharvest Biology and Technology, 14(1), 21–27. https://doi.org/10.1016/S0925-5214(98)00030-1
Walsh, K. B., Guthrie, J. A., & Burney, J. W. (2000). Application of commercially available, low-cost, miniaturised NIR spectrometers to the assessment of the sugar content of intact fruit. Functional Plant Biology, 27(12), 1175–1186. https://doi.org/10.1071/PP99111
Wang, W., & Li, C. (2014). Size estimation of sweet onions using consumer-grade RGB-depth sensor. Journal of Food Engineering, 142, 153–162. https://doi.org/10.1016/j.jfoodeng.2014.06.019
Wang, Z., Walsh, K. B., & Verma, B. (2017). On-tree mango fruit size estimation using RGB-D images. Sensors, 17(12), 2738. https://doi.org/10.3390/S17122738
Xiao, D., Liu, J., Hu, T., Shah Nayaz, B. M., Jiang, X., Zhang, F., & Yan, P. (2021). Simple ways to estimate meningioma volume: can abc-and sh-derived methods be used in clinical practice reliably? Journal of Oncology, 2021. https://doi.org/10.1155/2021/9712287
Xie, W., Wei, S., & Yang, D. (2023). Morphological measurement for carrot based on three-dimensional reconstruction with a ToF sensor. Postharvest Biology and Technology, 197, 112216. https://doi.org/10.1016/J.POSTHARVBIO.2022.112216
Yamamoto, S., Yamamoto, S., Karkee, M., Kobayashi, Y., Nakayama, N., Tsubota, S., Thanh, L. N. T., & Konya, T. (2018). 3D reconstruction of apple fruits using consumer-grade RGB-depth sensor. Engineering in Agriculture, Environment and Food, 11(4), 159–168. https://doi.org/10.1016/j.eaef.2018.02.005
Zhou, Q.-Y., Park, J., & Koltun, V. (2018). Open3D: A modern library for 3D data processing. ArXiv Preprint ArXiv:1801.09847. https://doi.org/10.48550/arXiv.1801.09847
Funding
This study was conducted with the “Research Program for Agricultural Science & Technology Development (Project No. PJ01675503)”, National Institute of Agricultural Science, Rural Development Administration, Republic of Korea.
Author information
Authors and Affiliations
Contributions
Suk-Ju Hong: Writing the Original Draft, Conceptualization, Methodology, Software, Formal analysis, and Project administration. Jinse Kim: Validation, Investigation, Data Curation. Ahyeong Lee: Writing-Review & Editing, Visualization, Formal analysis.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hong, SJ., Kim, J. & Lee, A. Real-Time Morphological Measurement of Oriental Melon Fruit Through Multi-Depth Camera Three-Dimensional Reconstruction. Food Bioprocess Technol (2024). https://doi.org/10.1007/s11947-024-03435-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11947-024-03435-8