Abstract
A procedure that is currently mostly handled by manual labor is olive fruit sorting by ripeness, as indicated by the color of each individual olive. Small and medium olive tree owners in olive-producing countries have to employee a significant number of workers for several days to perform this tedious task. Big industrial machines, capable of sorting do exist, but their cost is prohibitively high. These machines consist of two main components (a) the computer vision algorithm that is responsible for the detection of the olive fruit that are moving on a conveyor belt in lines and (b) a mechanical part that is responsible to sort the olives. However, advancements in computer vision allow for the implementation of low-cost solutions for the detection of olives. In this work, we present an automated solution that, by using computer vision, can perform the task of detecting the moving olive fruit effectively. Centroids of moving olive fruit are extracted through the Watershed Transform and by employing an Unscented Kalman filter, their position is estimated and tracked in consecutive video frames. Simulation experiments are designed, and the method is tested in various conditions to study its performance. Results suggest that the proposed approach tracks individual olives in synthetic videos created by scrolling images of olives with high accuracy. Even in the presence of induced noise, that resembles motion traces in images, the procedure remains capable of detecting and tracking olive fruit that are not trivially detected by the human eye.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
A symmetric nxn matrix A is positive semi-definite, if and only if xTAx ≥ 0 for all vectors x ∈ ℝn.
- 6.
An overview of the centroid extraction process can be accessed through https://www.youtube.com/watch?v=LgUJuib5yq0
- 7.
A video of the Kalman filtering results can be found in (https://youtu.be/q4T-poMhXlA)
References
Díaz, R. (2016). Classification and quality evaluation of table olives. In Computer vision technology for food quality evaluation (pp. 351–367). Academic Press.
Guzmán, E., Baeten, V., Pierna, J. A. F., & García-Mesa, J. A. (2015). Determination of the olive maturity index of intact fruit using image analysis. Journal of Food Science and Technology, 52(3), 1462–1470.
Puerto, D. A., Gila, D. M. M., García, J. G., & Ortega, J. G. (2015). Sorting olive batches for the milling process using image processing. Sensors, 15(7), 15738–15754.
Kavdır, İ., Büyükcan, M. B., & Kurtulmuş, F. (2018). Classification of olives using FT-NIR spectroscopy, neural networks and statistical classifiers. Journal of Food Measurement and Characterization, 12(4), 2493–2502.
Ponce, J. M., Aquino, A., & Andújar, J. M. (2019). Olive-fruit variety classification by means of image processing and convolutional neural networks. IEEE Access, 7, 147629–147641.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2016). Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv:1602.07261 preprint.
Ponce, J. M., Aquino, A., Millan, B., & Andújar, J. M. (2019). Automatic counting and individual size and mass estimation of olive-fruit through computer vision techniques. IEEE Access, 7, 59451–59465.
Vincent, L., & Soille, P. (1991). Watershed in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 583–598.
Haris, K., Efstratiadis, S. N., Maglaveras, N., Pappas, C., Gourassas, J., & Louridas, G. (1999). Model-based morphological segmentation and labelling of coronary angiograms. IEEE Transactions on Medical Imaging, 18(10), 1003–1015.
Winter, M., Mankowski, W., Wait, E., De La Hoz, E. C., Aguinaldo, A., & Cohen, A. R. (2019). Separating touching cells using pixel replicated elliptical shape models. IEEE Transactions on Medical Imaging, 38(4), 883–893.
Lin, G., Adiga, U., Olson, K., Guzowski, J. F., Barnes, C. A., & Roysam, B. (2003). A hybrid 3D watershed algorithm incorporating gradient cues and object models for automatic segmentation of nuclei in confocal image stacks. Cytometry Part A, 56(1), 23–36.
Lu, N., & Ke, X. (2007). A segmentation method based on gray-scale mor-phological filter and watershed algorithm for touching objects image. In Fourth international conference on fuzzy systems and knowledge discovery (FSKD 2007) (pp. 474–478).
Otsu, N. (1979). A threshold selection method for gray-levels histograms. IEEE Transactions on Pattern Analysis Machine Intelligent, 13, 583–598.
Karvelis, P., Tzallas, A., Fotiadis, D., & Georgiou, I. (2008). A multichannel watershed-based segmentation method for multispectral chromosome classification. IEEE Transactions on Medical Imaging, 2(5), 697–708.
Gonzalez, R., & Woods, R. (2017). Digital image processing (4th ed.). Addison-Wesley Longman Publishing.
Vincent, L. (1993). Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms. IEEE Transactions on Image Processing, 2, 176–201.
Soille, P. (1999). Morphological image analysis: Principles and applications. Springer-Verlag.
Karvelis, P., Likas, A., & Fotiadis, D. (2010). Identifying touching and overlapping chromosomes using the watershed transform and gradient paths. Pattern Recognition Letters, 31(16), 2474–2488.
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X. & Kim T. (2014). Multiple object tracking: A literature review. arXiv: Computer Vision and Pattern Recognition.
Ciaparrone, G., Luque Sánchez, F., Tabik, S., Troiano, L., Tagliaferri, R., & Herrera, F. (2020). Deep learning in video multi-object tracking: A survey. Neurocomputing, 381, 61–88.
Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., & McCool, C. (2016). DeepFruit: A fruit detection system using deep neural networks. Sensors, 16, 1222.
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. ASME-Journal of Basic Engineering, 382(1), 35–45.
Julier, S. J. & Uhlmann, J. K. (1997). A new extension of the Kalman filter to nonlinear systems. In Proceedings of SPIE: The International Society for Optical Engineering, Orlando, FL, 3068, 182–193.
Julier, S. J., & Uhlmann, J. K. (2004). Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3), 401–422.
van der Merwe, R., Wan, E., & Julier, S. (2004). Sigma-point Kalman filters for nonlinear estimation and sensor-fusion: Applications to integrated navigation. AIAA Guidance, Navigation, and Control Conference, Providence.
Julier, S. J. (2002). The scaled unscented transformation. Proceedings of the 2002 American Control Conference, Anchorage, AK, USA, 6, 4555–4559.
Wang, X., Li, T., Sun, S., & Corchad, J. (2017). A survey of recent advances in particle filters and remaining challenges for multitarget tracking. Sensors, 17(12), 2707.
Guo, L., Ge, P., He, D., & Wang, D. (2019). Multi-vehicle detection and tracking based on Kalman filter and data association. In H. Yu, J. Liu, L. Liu, Z. Ju, Y. Liu, & D. Zhou (Eds.), Intelligent robotics and applications, ICIRA 2019, lecture notes in computer science, 11744. Springer.
Chen, X., Wang, X. & Xuan, J. (2018). Tracking multiple moving objects using Unscented Kalman Filtering techniques. arXiv:1802.01235.
Kuhn, H. W. (1955). The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2, 83–97.
Munkres, J. (1957). Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32–38.
Bourgeois, F., & Lassalle, J. (1971). An extension of the Munkres algorithm for the assignment problem to rectangular matrices. Communications of the ACM, 14(12), 802.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Georgiou, G., Karvelis, P., Gogos, C. (2022). Computer Vision-based Detection and Tracking in the Olive Sorting Pipeline. In: Bochtis, D.D., Moshou, D.E., Vasileiadis, G., Balafoutis, A., Pardalos, P.M. (eds) Information and Communication Technologies for Agriculture—Theme II: Data. Springer Optimization and Its Applications, vol 183. Springer, Cham. https://doi.org/10.1007/978-3-030-84148-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-84148-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84147-8
Online ISBN: 978-3-030-84148-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)