Skip to main content

Localization and Classification of Rice-grain Images Using Region Proposals-based Convolutional Neural Network

Abstract

This paper proposes a solution to localization and classification of rice grains in an image. All existing related works rely on conventional based machine learning approaches. However, those techniques do not do well for the problem designed in this paper, due to the high similarities between different types of rice grains. The deep learning based solution is developed in the proposed solution. It contains pre-processing steps of data annotation using the watershed algorithm, auto-alignment using the major axis orientation, and image enhancement using the contrast-limited adaptive histogram equalization (CLAHE) technique. Then, the mask region-based convolutional neural networks (R-CNN) is trained to localize and classify rice grains in an input image. The performance is enhanced by using the transfer learning and the dropout regularization for overfitting prevention. The proposed method is validated using many scenarios of experiments, reported in the forms of mean average precision (mAP) and a confusion matrix. It achieves above 80% mAP for main scenarios in the experiments. It is also shown to perform outstanding, when compared to human experts.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    K. M. He, G. Gkioxari, P. Dollar, R. Girshick. Mask RCNN. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2961–2969, 2017. DOI: https://doi.org/10.1109/ICCV.2017.322.

    Google Scholar 

  2. [2]

    B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning-based fine-grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 119–135, 2017. DOI: https://doi.org/10.1007/s11633-017-1053-3.

    Article  Google Scholar 

  3. [3]

    Z. Y. Liu, F. Cheng, Y. B. Ying, X. Q. Rao. Identification of rice seed varieties using neural network. Journal of Zhejiang University Science B, vol. 6, no. 11, pp. 1095–1100, 2005.

    Article  Google Scholar 

  4. [4]

    A. H. Bhensjaliya, H. D. Vasava. Survey on classification of rice grains using neural network. International Journal of Computer Sciences and Engineering, vol. 7, pp. 828–831, 2019.

    Article  Google Scholar 

  5. [5]

    T. Y. Kuo, C. L. Chung, S. Y. Chen, H. A. Lin, Y. F. Kuo. Identifying rice grains using image analysis and sparse-representation-based classification. Computers and Electronics in Agriculture, vol. 127, pp. 716–725, 2016. DOI: https://doi.org/10.1016/j.compag.2016.07.020.

    Article  Google Scholar 

  6. [6]

    X. Yi, M. Eramian, R. J. Wang, E. Neufeld. Identification of morphologically similar seeds using multi-kernel learning. In Proceedings of Canadian Conference on Computer and Robot Vision, IEEE, Montreal, Canada, pp. 143–150, 2014. DOI: https://doi.org/10.1109/CRV.2014.27.

    Google Scholar 

  7. [7]

    X. S. Wang, C. Cai. Weed seeds classification based on pcanet deep learning baseline. In Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, IEEE, Hong Kong, China, pp. 408–415, 2015. DOI: https://doi.org/10.1109/APSIPA.2015.7415304.

    Google Scholar 

  8. [8]

    J. Rexce, K. D. K. Usha. Classification of milled rice using image processing. International Journal of Scientific & Engineering Research, vol. 8, no. 2, pp. 10–14, 2017.

    Google Scholar 

  9. [9]

    J. B. T. M. Roerdink, A. Meijster. The watershed transform: Definitions, algorithms and parallelization strategies. Fundamenta Informaticae, vol. 41, no. 1–2, pp. 187–228, 2000. DOI: https://doi.org/10.3233/FI-2000-411207.

    MathSciNet  Article  Google Scholar 

  10. [10]

    X. Y. Gong, H. Su, D. Xu, Z. T. Zhang, F. Shen, H. B. Yang. An overview of contour detection approaches. International Journal of Automation and Computing, vol. 15, no. 6, pp. 656–672, 2018. DOI: https://doi.org/10.1007/s11633-018-1117-z.

    Article  Google Scholar 

  11. [11]

    J. Yosinski, J. Clune, Y. Bengio, H. Lipson. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 3320–3328, 2014.

    Google Scholar 

  12. [12]

    S. Hantke, T. Olenyi, C. Hausner, T. Appel, B. Schuller. Large-scale data collection and analysis via a gamified intelligent crowdsourcing platform. International Journal of Automation and Computing, vol. 16, no. 4, pp. 427–436, 2019. DOI: https://doi.org/10.1007/s11633-019-1180-0.

    Article  Google Scholar 

  13. [13]

    S. Q. Ren, K. M. He, R. Girshick, J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 91–99, 2015.

    Google Scholar 

  14. [14]

    K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90

    Google Scholar 

  15. [15]

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014.

    MathSciNet  MATH  Google Scholar 

  16. [16]

    W. Abdulla. Mask R-CNN for object detection and instance segmentation on Keras and tensorflow, [Online], Available: https://github.com/matterport/Mask_RCNN, 2017.

    Google Scholar 

  17. [17]

    T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick. Microsoft coco: Common objects in context. In Proceedings of the 13th European Conference on Computer Vision, Springer, Zurich, Switzerland, pp. 740–755, 2014. DOI: https://doi.org/10.1007/978-3-319-10602-148.

    Google Scholar 

  18. [18]

    J. Majumdar, K. L. Santhosh Kumar. Modified clahe: An adaptive algorithm for contrast enhancement of aerial, medical and underwater images. International Journal of Computer Engineering & Technology, vol. 5, no. 11, pp. 32–47, 2014.

    Google Scholar 

  19. [19]

    W. Kusakunniran, A. Wiratsudakul, U. Chuachan, S. Kanchanapreechakorn, T. Imaromkul. Automatic cattle identification based on fusion of texture features extracted from muzzle images. In Proceedings of IEEE International Conference on Industrial Technology, IEEE, Lyon, France, pp. 1484–1489, 2018. DOI: https://doi.org/10.1109/ICIT.2018.8352400.

    Google Scholar 

  20. [20]

    Y. S. Yue, T. Finley, F. Radlinski, T. Joachims. A support vector method for optimizing average precision. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Amsterdam, The Netherlands, pp. 271–278, 2007. DOI: https://doi.org/10.1145/1277741.1277790.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Worapan Kusakunniran.

Additional information

Recommended by Associate Editor Zhi-Jie Xu

Kittinun Aukkapinyo received the B. Sc. degree in information and communication technology from Faculty of Information and Communication Technology, Mahidol University, Thailand in 2019. He is currently a Data Scientist with Wongnai Media Co., Ltd, Bangkok, Thailand.

His research interests include pattern recognition, computer vision, multimedia information retrieval, and machine learning.

Suchakree Sawangwong received the B. Sc degree in information and communication technology from University of Mahidol, Thailand in 2019. He is currently a Unity Developer with Proudia company, Bangkok, Thailand.

His research interests include image processing, computer vision, multimedia, and machine learning.

Parintorn Pooyoi received the B. Sc degree in information and communication technology from University of Mahidol, Thailand in 2019. He is currently a Java Developer with Siam commercial bank, Bangkok, Thailand.

His research interests include image processing, computer vision, multi-thread programming, machine learning, and deep learning.

Worapan Kusakunniran received the B. Eng. degree in computer engineering from the University of New South Wales (UNSW), Australia in 2008, and the Ph.D. degree in computer science and engineering from UNSW, in cooperation with the Neville Roach Laboratory, National ICT Australia, Australia in 2013. He is currently a lecturer with the Faculty of Information and Communication Technology, Mahidol University, Thailand. He is the author of several papers in top international conferences and journals. He served as a program committee member for many international conferences and workshops. Also, he has served as a reviewer for several international conferences and journals, such as International Conference on Pattern Recognition, IEEE International Conference on Image Processing, IEEE International Conference on Advanced Video and Signal based Surveillance, Pattern Recognition, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on Image Processing, IEEE Transactions on Information Forensics and Security, and IEEE Signal Processing Letters. He was a recipient of the ICPR Best Biometric Student Paper Award in 2010, and also a winner of several national and international innovation contests.

His research interests include biometrics, pattern recognition, medical image processing, computer vision, multimedia, and machine learning.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aukkapinyo, K., Sawangwong, S., Pooyoi, P. et al. Localization and Classification of Rice-grain Images Using Region Proposals-based Convolutional Neural Network. Int. J. Autom. Comput. 17, 233–246 (2020). https://doi.org/10.1007/s11633-019-1207-6

Download citation

Keywords

  • Mask region-based convolutional neural networks (R-CNN)
  • computer vision
  • deep learning
  • rice grain classification
  • transfer learning