Detection and counting of immature green citrus fruit based on the Local Binary Patterns (LBP) feature using illumination-normalized images

Article
  • 20 Downloads

Abstract

Early detection and counting of immature green citrus fruit using computer vision can help growers produce a predictive yield map which could be used to adjust management practices during the fruit maturing stages. However, such detecting and counting is difficult because of varying illumination, random occlusion and color similarity with leaves. An immature fruit detection algorithm was developed with the aim of identifying and counting fruit in a citrus grove under varying illumination environments and random occlusions using images acquired by a regular red–green–blue (RGB) color camera. Acquired citrus images included front-lighting and back-lighting illumination conditions. The Retinex image enhancement algorithm and the two-dimensional discrete wavelet transform were used for image illumination normalization. Color-based K-means clustering and circular hough transform (CHT) were applied in order to detect potential fruit regions. A Local Binary Patterns feature-based Adaptive Boosting (AdaBoost) classifier was built for removing false positives. A sub-window was used to scan the difference image between the illumination-normalized image and the resulting image from CHT detection in order to detect small areas and partially occluded fruit. An overall accuracy of 85.6% was achieved for the validation set which showed promising potential for the proposed method.

Keywords

Circular hough transform Immature green citrus Varying illumination Yield mapping 

Abbreviations

LBP

Local Binary Patterns

RGB

Red, green and blue

CHT

Circle hough transform

FFT

Fast Fourier transform

ARB

Adaptive red and blue chromatic map

FNCC

Fast normalized cross correlation

Retinex

A compound word from ‘retina’ and ‘cortex’

SVM

Support vector machines

LS- SVM

The least squares support vector machine

AdaBoost

Abbreviations of adaptive boosting

Notes

Acknowledgements

The authors would like to thank the National Natural Science Foundation of China and Science, technology project of Guangdong Province, technology project of Huizhou (Nos. 31571568, 2015A020209111, 2014B040008006), and support from the University of Florida.

References

  1. Bansal, R., Lee, W. S., & Satish, S. (2013). Green citrus detection using fast Fourier transform (FFT) leakage. Precision Agriculture, 14(1), 59–70.CrossRefGoogle Scholar
  2. Cheng, Y., Hou, Y., Zhao, C., Li, Z., Hu, Y., & Wang, C. (2010). Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain. Neurocomputing, 73, 2217–2224.CrossRefGoogle Scholar
  3. Du, S., & Ward, R. (2005). Wavelet-based illumination normalization for face recognition. In 2005 International Conference on Image Processing (Vol. 3, pp. 954–957). IEEE Computer Society, Genova, Italy.Google Scholar
  4. Duan, J., Zhou, C. G., Liu, X. H., Zhang, L. B., & Liu, M. (2004). The methods of improving variable illumination for face recognition. In Proceedings of 2004 International Conference on Machine Learning and Cybernetics (pp. 3918–3923). IEEE Computer Society, Shanghai City, China.Google Scholar
  5. Duda, R. O., & Hart, P. E. (1972). Use of the Hough transform to detect lies and curves in pictures. Communications of the ACM, 15(1), 11–15.CrossRefGoogle Scholar
  6. Durand, F., & Dorsey, J. (2002). Fast bilateral filtering for the display of high dynamic range images. SIGGRAPH, 2002, 257–266.Google Scholar
  7. Edwin, H. Land. (1977). The retinex theory of color vision. Scientific American, 237(6), 108–129.CrossRefGoogle Scholar
  8. Emadi, M., Khalid, M., Yusof, R., & Navabifar, F. (2012). Illumination Normalization using 2D Wavelet. Procedia Engineering, 41, 854–859.CrossRefGoogle Scholar
  9. Freund, Y., & Schapire, R. E. (1995). A decision-theoretic generalization of on-line learning and an application to boosting. In: Vitányi P. (Ed) Computational Learning Theory. EuroCOLT 1995. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence) (Vol. 904). Berlin, Heidelberg, Germany: Springer.Google Scholar
  10. Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.CrossRefGoogle Scholar
  11. Kane, K. E., & Lee, W. S. (2007). Multispectral imaging for in-field green citrus identification. ASABE Paper No. 073025. St. Joseph, MI, USA: ASABE.Google Scholar
  12. Khammari, A., Nashashibi, F., Abramson, Y., & Laurgeau, C. (2005). Vehicle detection combining gradient analysis and AdaBoost classification. In Proceedings of 2005 Intelligent Transportation Systems (pp. 66–71). IEEE Computer Society, Vienna, Austria.Google Scholar
  13. Kurtulmus, F., Lee, W. S., & Vardar, A. (2011). Green citrus detection using “eigenfruit”, color and circular Gabor texture features under natural outdoor conditions. Computers and Electronics in Agriculture, 78(2), 140–149.CrossRefGoogle Scholar
  14. Kurtulmus, F., Lee, W. S., & Vardar, A. (2014). Immature peach detection in color images acquired in natural illumination conditions using statistical classifiers and neural network. Precision Agriculture, 15, 57–79.CrossRefGoogle Scholar
  15. Lei, J., Han, H., & Tong, M. (2012). Efficient visual tracking by using LBP descriptor. International Conference on Artificial Intelligence and Computer Engineering, 7530, 391–399.CrossRefGoogle Scholar
  16. Li, H., Lee, W. S., & Wang, K. (2016). Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor color images. Precision Agriculture, 17, 678–697.CrossRefGoogle Scholar
  17. Linker, R., Cohen, O., & Naor, A. (2012). Determination of the number of green apples in RGB images recorded in orchards. Computers and Electronics in Agriculture, 81(2), 45–57.CrossRefGoogle Scholar
  18. Luo, L. F., Tang, Y. C., Zou, X. J., Wang, C. L., Zhang, P., & Feng, W. X. (2016). Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components. Sensors, 16(12), 1–20.CrossRefGoogle Scholar
  19. Mazo, C., Alegre, E., & Trujillo, M. (2017). Classification of cardiovascular tissues using LBP based descriptors and a cascade SVM. Computer Methods and Programs in Biomedicine, 147, 1–10.CrossRefPubMedGoogle Scholar
  20. Ojala, T., Pietikäinen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on feature distributions. Pattern Recognition, 29, 51–59.CrossRefGoogle Scholar
  21. Okamoto, H., & Lee, W. S. (2009). Green citrus detection using hyperspectral imaging. Computers and Electronics in Agriculture, 66, 201–208.CrossRefGoogle Scholar
  22. Savvides, M., & Kumar, V. (2003). Illumination normalization using logarithm transforms for face authentication. In Proceedings of the 4th International Conference on Audio- and Video-Based Biometric Person Authentication (pp. 549–556). Guildford, UK: Springer.Google Scholar
  23. Sengupta, S., & Lee, W. S. (2014). Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosystems Engineering, 117, 51–61.CrossRefGoogle Scholar
  24. Stajnko, D., Lakota, M., & Hoevar, M. (2004). Estimation of number and diameter of apple fruit in an orchard during the growing season by thermal imaging. Computers and Electronics in Agriculture, 42, 31–42.CrossRefGoogle Scholar
  25. Sun, J. G., Li, Y., Yang, X. N., & Wang, J. T. (2011). Face recognition based on improved LBP and LS-SVM. Advanced Materials Research, 403–408, 3249–3252.CrossRefGoogle Scholar
  26. United States Department of Agriculture-National Agricultural Statistics Service (USDA-NASS). (2015). Forecasting Florida citrus production (brochure). Retrieved June 28, 2017 from http://usda.mannlib.cornell.edu/usda/nass/CitrFrui//2010s/2015/CitrFrui-09-17-2015.pdf.
  27. Vishwakarma, V. P., Pandey, S., & Gupta, M. N. (2009). Adaptive histogram equalization and logarithm transform with rescaled low-frequency DCT coefficients for illumination normalization. International Journal of Recent Trends in Engineering, Issue on Computer Science, 1, 318–322.Google Scholar
  28. Wachs, J. P., Stern, H. I., Burks, T., & Alchanatis, V. (2010). Low and high-level visual feature-based apple detection from multi-modal images. Precision Agriculture, 11, 717–735.CrossRefGoogle Scholar
  29. Wang, Y. B., Ai, H. Z., Wu, B., & Huang, C. (2004). Real time facial expression recognition with AdaBoost. In Proceedings of the 17th International Conference on Pattern Recognition (pp. 926–929). IEEE Computer Society, Cambridge, UK.Google Scholar
  30. Yang, Y. N., Jiang, Z. H., Yang, C. H., Xia, Z. Q., & Liu, F. (2015). Improved retinex image enhancement algorithm based on bilateral filtering. In Proceedings of the 4th International Conference on Mechatronics, Materials, Chemistry and Computer Engineering (Vol. 39, pp. 2250–2256). Xi’an, China: Atlantis Press.Google Scholar
  31. Zhao, C. Y., Lee, W. S., & He, D. (2016). Immature green citrus detection based on color feature and sum of absolute transformed difference (SATD) using color images in the citrus grove. Computers and Electronics in Agriculture, 124, 243–253.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electronic Information EngineeringChina West Normal UniversityNanchongChina
  2. 2.Department of Agricultural & Biological EngineeringUniversity of FloridaGainesvilleUSA
  3. 3.College of EngineeringSouth China Agricultural UniversityGuangzhouChina
  4. 4.Department of Agricultural & Biological EngineeringPenn State UniversityUniversity ParkUSA

Personalised recommendations