Abstract
This paper proposes a solution to localization and classification of rice grains in an image. All existing related works rely on conventional based machine learning approaches. However, those techniques do not do well for the problem designed in this paper, due to the high similarities between different types of rice grains. The deep learning based solution is developed in the proposed solution. It contains pre-processing steps of data annotation using the watershed algorithm, auto-alignment using the major axis orientation, and image enhancement using the contrast-limited adaptive histogram equalization (CLAHE) technique. Then, the mask region-based convolutional neural networks (R-CNN) is trained to localize and classify rice grains in an input image. The performance is enhanced by using the transfer learning and the dropout regularization for overfitting prevention. The proposed method is validated using many scenarios of experiments, reported in the forms of mean average precision (mAP) and a confusion matrix. It achieves above 80% mAP for main scenarios in the experiments. It is also shown to perform outstanding, when compared to human experts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
K. M. He, G. Gkioxari, P. Dollar, R. Girshick. Mask RCNN. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2961–2969, 2017. DOI: https://doi.org/10.1109/ICCV.2017.322.
B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning-based fine-grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 119–135, 2017. DOI: https://doi.org/10.1007/s11633-017-1053-3.
Z. Y. Liu, F. Cheng, Y. B. Ying, X. Q. Rao. Identification of rice seed varieties using neural network. Journal of Zhejiang University Science B, vol. 6, no. 11, pp. 1095–1100, 2005.
A. H. Bhensjaliya, H. D. Vasava. Survey on classification of rice grains using neural network. International Journal of Computer Sciences and Engineering, vol. 7, pp. 828–831, 2019.
T. Y. Kuo, C. L. Chung, S. Y. Chen, H. A. Lin, Y. F. Kuo. Identifying rice grains using image analysis and sparse-representation-based classification. Computers and Electronics in Agriculture, vol. 127, pp. 716–725, 2016. DOI: https://doi.org/10.1016/j.compag.2016.07.020.
X. Yi, M. Eramian, R. J. Wang, E. Neufeld. Identification of morphologically similar seeds using multi-kernel learning. In Proceedings of Canadian Conference on Computer and Robot Vision, IEEE, Montreal, Canada, pp. 143–150, 2014. DOI: https://doi.org/10.1109/CRV.2014.27.
X. S. Wang, C. Cai. Weed seeds classification based on pcanet deep learning baseline. In Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, IEEE, Hong Kong, China, pp. 408–415, 2015. DOI: https://doi.org/10.1109/APSIPA.2015.7415304.
J. Rexce, K. D. K. Usha. Classification of milled rice using image processing. International Journal of Scientific & Engineering Research, vol. 8, no. 2, pp. 10–14, 2017.
J. B. T. M. Roerdink, A. Meijster. The watershed transform: Definitions, algorithms and parallelization strategies. Fundamenta Informaticae, vol. 41, no. 1–2, pp. 187–228, 2000. DOI: https://doi.org/10.3233/FI-2000-411207.
X. Y. Gong, H. Su, D. Xu, Z. T. Zhang, F. Shen, H. B. Yang. An overview of contour detection approaches. International Journal of Automation and Computing, vol. 15, no. 6, pp. 656–672, 2018. DOI: https://doi.org/10.1007/s11633-018-1117-z.
J. Yosinski, J. Clune, Y. Bengio, H. Lipson. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 3320–3328, 2014.
S. Hantke, T. Olenyi, C. Hausner, T. Appel, B. Schuller. Large-scale data collection and analysis via a gamified intelligent crowdsourcing platform. International Journal of Automation and Computing, vol. 16, no. 4, pp. 427–436, 2019. DOI: https://doi.org/10.1007/s11633-019-1180-0.
S. Q. Ren, K. M. He, R. Girshick, J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 91–99, 2015.
K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014.
W. Abdulla. Mask R-CNN for object detection and instance segmentation on Keras and tensorflow, [Online], Available: https://github.com/matterport/Mask_RCNN, 2017.
T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick. Microsoft coco: Common objects in context. In Proceedings of the 13th European Conference on Computer Vision, Springer, Zurich, Switzerland, pp. 740–755, 2014. DOI: https://doi.org/10.1007/978-3-319-10602-148.
J. Majumdar, K. L. Santhosh Kumar. Modified clahe: An adaptive algorithm for contrast enhancement of aerial, medical and underwater images. International Journal of Computer Engineering & Technology, vol. 5, no. 11, pp. 32–47, 2014.
W. Kusakunniran, A. Wiratsudakul, U. Chuachan, S. Kanchanapreechakorn, T. Imaromkul. Automatic cattle identification based on fusion of texture features extracted from muzzle images. In Proceedings of IEEE International Conference on Industrial Technology, IEEE, Lyon, France, pp. 1484–1489, 2018. DOI: https://doi.org/10.1109/ICIT.2018.8352400.
Y. S. Yue, T. Finley, F. Radlinski, T. Joachims. A support vector method for optimizing average precision. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Amsterdam, The Netherlands, pp. 271–278, 2007. DOI: https://doi.org/10.1145/1277741.1277790.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Zhi-Jie Xu
Kittinun Aukkapinyo received the B. Sc. degree in information and communication technology from Faculty of Information and Communication Technology, Mahidol University, Thailand in 2019. He is currently a Data Scientist with Wongnai Media Co., Ltd, Bangkok, Thailand.
His research interests include pattern recognition, computer vision, multimedia information retrieval, and machine learning.
Suchakree Sawangwong received the B. Sc degree in information and communication technology from University of Mahidol, Thailand in 2019. He is currently a Unity Developer with Proudia company, Bangkok, Thailand.
His research interests include image processing, computer vision, multimedia, and machine learning.
Parintorn Pooyoi received the B. Sc degree in information and communication technology from University of Mahidol, Thailand in 2019. He is currently a Java Developer with Siam commercial bank, Bangkok, Thailand.
His research interests include image processing, computer vision, multi-thread programming, machine learning, and deep learning.
Worapan Kusakunniran received the B. Eng. degree in computer engineering from the University of New South Wales (UNSW), Australia in 2008, and the Ph.D. degree in computer science and engineering from UNSW, in cooperation with the Neville Roach Laboratory, National ICT Australia, Australia in 2013. He is currently a lecturer with the Faculty of Information and Communication Technology, Mahidol University, Thailand. He is the author of several papers in top international conferences and journals. He served as a program committee member for many international conferences and workshops. Also, he has served as a reviewer for several international conferences and journals, such as International Conference on Pattern Recognition, IEEE International Conference on Image Processing, IEEE International Conference on Advanced Video and Signal based Surveillance, Pattern Recognition, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on Image Processing, IEEE Transactions on Information Forensics and Security, and IEEE Signal Processing Letters. He was a recipient of the ICPR Best Biometric Student Paper Award in 2010, and also a winner of several national and international innovation contests.
His research interests include biometrics, pattern recognition, medical image processing, computer vision, multimedia, and machine learning.
Rights and permissions
About this article
Cite this article
Aukkapinyo, K., Sawangwong, S., Pooyoi, P. et al. Localization and Classification of Rice-grain Images Using Region Proposals-based Convolutional Neural Network. Int. J. Autom. Comput. 17, 233–246 (2020). https://doi.org/10.1007/s11633-019-1207-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-019-1207-6