Skip to main content
SpringerLink
Log in

Cherry tomato firmness detection and prediction using a vision-based tactile sensor

  • Original Paper
  • Published:
Journal of Food Measurement and Characterization Aims and scope Submit manuscript

Abstract

Since fresh cherry tomatoes have a high likelihood of spoilage during post-harvest transport and storage, assessing the fruit grade based on its firmness or predicting its potential storage time becomes necessary. The study proposes a method for non-destructive fruit firmness detection using a photometric stereo vision-based tactile sensor. The tactile sensor can measure the three-dimensional deformation of cherry tomatoes by mapping between pixel color and surface normal using a gradient prediction network. The summation of the deformation heights measured by a tactile sensor pressing on the cherry tomatoes can accurately describe fruit firmness, which was also verified by a comparative experiment with the Texture Analyzer. To create a model of firmness variation over time, a group of tomatoes of the same variety and origin were measured for firmness at various storage dates following harvest in subsequent experiments. An experiment was conducted to measure and predict the variation in the firmness of cherry tomatoes using a tactile sensor. The results indicate that the firmness of cherry tomatoes decreases linearly over time under similar conditions. The relative prediction errors were found to be mostly within ± 15% and ± 25% during 1 and 2 weeks of storage, respectively. All findings indicate that the proposed method can effectively measure the firmness of cherry tomatoes and accurately predict their short-term variations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. A.M. Opiyo, T.J. Ying, The effects of 1-methylcyclopropene treatment on the shelf life and quality of cherry tomato (Lycopersicon esculentum var. cerasiforme) fruit. Int. J. Food Sci. Technol. 40(6), 665–673 (2005). https://doi.org/10.1111/j.1365-2621.2005.00977.x

    Article  CAS  Google Scholar 

  2. L. Aragüez, A. Colombo, R. Borneo, A. Aguirre, Active packaging from triticale flour films for prolonging storage life of cherry tomato. Food Packag. Shelf Life 25, 100520 (2020). https://doi.org/10.1016/j.fpsl.2020.100520

    Article  Google Scholar 

  3. C. Fagundes, K. Moraes, M.B. Pérez-Gago, L. Palou, M. Maraschin, A.R. Monteiro, Effect of active modified atmosphere and cold storage on the postharvest quality of cherry tomatoes. Postharvest Biol. Technol. 109, 73–81 (2015). https://doi.org/10.1016/j.postharvbio.2015.05.017

    Article  Google Scholar 

  4. P. Barreiro, V. Steinmetz, M. Ruiz-Altisent, Neural bruise prediction models for fruit handling and machinery evaluation. Comput. Electron. Agric. 18(2–3), 91–103 (1997). https://doi.org/10.1016/s0168-1699(97)00022-7

    Article  Google Scholar 

  5. P.P. Subedi, K.B. Walsh, Non-invasive techniques for measurement of fresh fruit firmness. Postharvest Biol. Technol. 51(3), 297–304 (2009). https://doi.org/10.1016/j.postharvbio.2008.03.004

    Article  Google Scholar 

  6. J.A. Caladcad, S. Cabahug, M.R. Catamco, P.E. Villaceran, L. Cosgafa, K.N. Cabizares et al., Determining Philippine coconut maturity level using machine learning algorithms based on acoustic signal. Comput. Electron. Agric. 172, 105327 (2020). https://doi.org/10.1016/j.compag.2020.105327

    Article  Google Scholar 

  7. S. Sohaib Ali Shah, A. Zeb, W.S. Qureshi, M. Arslan, A. Ullah Malik, W. Alasmary, E. Alanazi, Towards fruit maturity estimation using NIR spectroscopy. Infrared Phys. Technol. 111, 103479 (2020). https://doi.org/10.1016/j.infrared.2020.103479

    Article  CAS  Google Scholar 

  8. A. Tugnolo, V. Giovenzana, R. Beghi, S. Grassi, C. Alamprese, A. Casson et al., A diagnostic visible/near infrared tool for a fully automated olive ripeness evaluation in a view of a simplified optical system. Comput. Electron. Agric. 180, 105887 (2021). https://doi.org/10.1016/j.compag.2020.105887

    Article  Google Scholar 

  9. S. Nie, D.F. Al Riza, Y. Ogawa, T. Suzuki, M. Kuramoto, N. Miyata, N. Kondo, Potential of a double lighting imaging system for characterization of “Hayward” kiwifruit harvest indices. Postharvest Biol. Technol. 162, 111113 (2020). https://doi.org/10.1016/j.postharvbio.2019.111113

    Article  CAS  Google Scholar 

  10. A. Scalisi, D. Pelliccia, M.G. O’connell, Maturity prediction in yellow peach (Prunus persica l.) cultivars using a fluorescence spectrometer. Sensors 20(22), 1–17 (2020). https://doi.org/10.3390/s20226555

    Article  CAS  Google Scholar 

  11. S. Srivastava, S. Sadistap, Data fusion for fruit quality authentication: combining non-destructive sensing techniques to predict quality parameters of citrus cultivars. J. Food Meas. Charact. 16(1), 344–365 (2022). https://doi.org/10.1007/s11694-021-01165-5

    Article  Google Scholar 

  12. M. Arunkumar, A. Rajendran, S. Gunasri, M. Kowsalya, C.K. Krithika, Non-destructive fruit maturity detection methodology—a review. Mater. Today: Proc. (2021). https://doi.org/10.1016/j.matpr.2020.12.1094

    Article  Google Scholar 

  13. M. Zude, B. Herold, J.M. Roger, V. Bellon-Maurel, S. Landahl, Non-destructive tests on the prediction of apple fruit flesh firmness and soluble solids content on tree and in shelf life. J. Food Eng. 77(2), 254–260 (2006). https://doi.org/10.1016/j.jfoodeng.2005.06.027

    Article  Google Scholar 

  14. Y. Huang, J. Xiong, X. Jiang, K. Chen, D. Hu, Assessment of firmness and soluble solids content of peaches by spatially resolved spectroscopy with a spectral difference technique. Comput. Electron. Agric. 200, 107212 (2022). https://doi.org/10.1016/j.compag.2022.107212

    Article  Google Scholar 

  15. C. Ortiz, C. Blanes, M. Mellado, An ultra-low pressure pneumatic jamming impact device to non-destructively assess cherimoya firmness. Biosys. Eng. 180, 161–167 (2019). https://doi.org/10.1016/j.biosystemseng.2019.02.003

    Article  Google Scholar 

  16. C. Valero, C.H. Crisosto, D. Slaughter, Relationship between nondestructive firmness measurements and commercially important ripening fruit stages for peaches, nectarines and plums. Postharvest Biol. Technol. 44(3), 248–253 (2007). https://doi.org/10.1016/j.postharvbio.2006.12.014

    Article  Google Scholar 

  17. L. Scimeca, P. Maiolino, D. Cardin-Catalan, A.P.D. Pobil, A. Morales, F. Iida, Non-destructive robotic assessment of mango ripeness via multi-point soft haptics, in Proceedings—IEEE International Conference on Robotics and Automation. 2019-May, 1821–1826. https://doi.org/10.1109/ICRA.2019.8793956 (2019)

  18. R.V. Aroca, R.B. Gomes, R.R. Dantas, A.G. Calbo, L.M.G. Gonçalves, A wearable mobile sensor platform to assist fruit grading. Sensors 13(5), 6109–6140 (2013). https://doi.org/10.3390/s130506109

    Article  PubMed  PubMed Central  Google Scholar 

  19. Z. Zhang, J. Zhou, Z. Yan, K. Wang, J. Mao, Z. Jiang, Hardness recognition of fruits and vegetables based on tactile array information of manipulator. Comput. Electron. Agric. 181, 105959 (2021). https://doi.org/10.1016/j.compag.2020.105959

    Article  Google Scholar 

  20. M. Kielar, T. Hamid, L. Wu, F. Windels, P. Sah, A.K. Pandey, Organic optoelectronic diodes as tactile sensors for soft-touch applications. ACS Appl. Mater. Interfaces 11(24), 21775–21783 (2019). https://doi.org/10.1021/acsami.9b04671

    Article  CAS  PubMed  Google Scholar 

  21. R.L. Truby, R.K. Katzschmann, J.A. Lewis, D. Rus, Soft robotic fingers with embedded ionogel sensors and discrete actuation modes for somatosensitive manipulation. RoboSoft 2019–2019 IEEE International Conference on Soft Robotics, pp. 322–329 (2019). https://doi.org/10.1109/ROBOSOFT.2019.8722722

  22. L. Viry, A. Levi, M. Totaro, A. Mondini, V. Mattoli, B. Mazzolai, L. Beccai, Flexible three-axial force sensor for soft and highly sensitive artificial touch. Adv. Mater. 26(17), 2659–2664 (2014). https://doi.org/10.1002/adma.201305064

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. P. Roberts, D.D. Damian, W. Shan, T. Lu, C. Majidi, Soft-matter capacitive sensor for measuring shear and pressure deformation, in Proceedings—IEEE International Conference on Robotics and Automation, pp. 3529–3534 (2013). https://doi.org/10.1109/ICRA.2013.6631071

  24. J.A. Fishel, G.E. Loeb, Bayesian exploration for intelligent identification of textures. Front. Neurorobot. 6, 1–20 (2012). https://doi.org/10.3389/fnbot.2012.00004

    Article  Google Scholar 

  25. J.A. Fishel, G.E. Loeb, Sensing tactile microvibrations with the BioTac comparison with human sensitivity, in Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 1122–1127 (2012). https://doi.org/10.1109/BioRob.2012.6290741

  26. W. Yuan, Y. Mo, S. Wang, E.H. Adelson, Active clothing material perception using tactile sensing and deep learning. Proc. IEEE Int. Conf. Robot. Autom. 1, 4842–4849 (2018). https://doi.org/10.1109/ICRA.2018.8461164

    Article  Google Scholar 

  27. A. Amini, J.I. Lipton, D. Rus, Uncertainty aware texture classification and mapping using soft tactile sensors, in IEEE International Conference on Intelligent Robots and Systems, pp. 4249–4256 (2020). https://doi.org/10.1109/IROS45743.2020.9341045

  28. C. Wang, S. Wang, B. Romero, F. Veiga, E. Adelson, SwingBot: learning physical features from in-hand tactile exploration for dynamic swing-up manipulation, in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5633–5640 (2020)

  29. W. Yuan, M.A. Srinivasan, E.H. Adelson, Estimating object hardness with a GelSight touch sensor, in IEEE International Conference on Intelligent Robots and Systems, 2016-Novem, pp. 208–215 (2016). https://doi.org/10.1109/IROS.2016.7759057

  30. W. Yuan, C. Zhu, A. Owens, M.A. Srinivasan, E.H. Adelson, Shape-independent hardness estimation using deep learning and a GelSight tactile sensor, in Proceedings—IEEE International Conference on Robotics and Automation, pp. 951–958 (2017). https://doi.org/10.1109/ICRA.2017.7989116

  31. Y. Chen, J. Lin, X. Du, B. Fang, F. Sun, S. Li, Non-destructive fruit firmness evaluation using vision-based tactile information, in Proceedings—IEEE international conference on robotics and automation, pp. 2303–2309 (2022). https://doi.org/10.1109/ICRA46639.2022.9811920

  32. W. Yuan, S. Dong, E.H. Adelson, GelSight: high-resolution robot tactile sensors for estimating geometry and force. Sensors (2017). https://doi.org/10.3390/s17122762

    Article  PubMed  PubMed Central  Google Scholar 

  33. J. Li, S. Dong, E.H. Adelson, End-to-end pixelwise surface normal estimation with convolutional neural networks and shape reconstruction using GelSight sensor, in 2018 IEEE International Conference on Robotics and Biomimetics, ROBIO 2018. 1292–1297 (2017) (2018). https://doi.org/10.1109/ROBIO.2018.8665351

  34. A. Newell, K. Yang, J. Deng, Stacked hourglass networks for human pose estimation. Lect. Notes Comput. Sci. (Incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinform.) 9912, 483–499 (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Article  Google Scholar 

  35. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2015). https://doi.org/10.1109/CVPR.2016.90

  36. P. Biswas, A.R. East, E.W. Hewett, J.A. Heyes, Interpreting textural changes in low temperature stored tomatoes. Postharvest Biol. Technol. 87, 140–143 (2014). https://doi.org/10.1016/j.postharvbio.2013.08.018

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LGN22C130006

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou, China

    Leiying He, Lei Tao, Zenghong Ma & Xiaoqiang Du

  2. Key Laboratory of Transplanting Equipment and Technology of Zhejiang Province, Hangzhou, China

    Leiying He, Zenghong Ma & Xiaoqiang Du

  3. Graduate School of Engineering Science Department of Systems Innovation, Osaka University, Osaka, Japan

    Weiwei Wan

Authors
  1. Leiying He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zenghong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoqiang Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weiwei Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leiying He.

Ethics declarations

Competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, L., Tao, L., Ma, Z. et al. Cherry tomato firmness detection and prediction using a vision-based tactile sensor. Food Measure 18, 1053–1064 (2024). https://doi.org/10.1007/s11694-023-02249-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11694-023-02249-0

Keywords

Search

Navigation