Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 22, pp 29779–29797 | Cite as

Plant identification based on very deep convolutional neural networks

  • Heyan Zhu
  • Qinglin Liu
  • Yuankai Qi
  • Xinyuan Huang
  • Feng Jiang
  • Shengping Zhang
Article
  • 304 Downloads

Abstract

Plant identification is a critical step in protecting plant diversity. However, many existing identification systems prohibitively rely on hand-crafted features for plant species identification. In this paper, a deep learning method is employed to extract discriminative features from plant images along with a linear SVM for plant identification. To offer a self-learning feature representation for different plant organs, we choose a very deep convolutional neural networks (CNNs), which consists of sixteen convolutional layers followed by three Fully-Connected (FC) layers and a final soft-max layer. Five max-pooling layers are performed over a 2×2 pixel window with stride 2. Extensive experiments on several plant datasets demonstrate the remarkable performance of the very deep neural network compared to the hand-crafted features.

Keywords

Plant identification CNN Linear SVM 

Notes

Acknowledgments

This work is supported by the key R&D program of Yantai City (No. 2016YT06000609).

References

  1. 1.
    Nguyen QK, Le TL, Pham NH (2013) Leaf based plant identification system for android using surf features in combination with bag of words model and supervised learning. In: Proceedings of international conference on advanced technologies for communications, pp 404–407Google Scholar
  2. 2.
    Ahmed N, Ghani U, Asif S (2016) An automatic leaf based plant identification system. In: Proceedings of the 5th international multidisciplinary conference, pp 29–31Google Scholar
  3. 3.
    Arora A, Gupta A, Bagmar N, Mishra S, Bhattacharya A (2012) A plant identification system using shape and morphological features on segmented leaflets. CLEFGoogle Scholar
  4. 4.
    Barré P, Stöver BC, Müller KF, Steinhage V (2017) A computer vision system for automatic plant species identification. Ecological Informatics 40:50–56CrossRefGoogle Scholar
  5. 5.
    Belhumeur PN, Chen D, Feiner S, Jacobs D, Kress W, Ling H, Lopez I, Ramamoorthi R, Sheorey S, White S, Zhang L (2008) Searching the world’s herbaria: A system for visual identification of plant species. In: Proceedings of 10th european conference on computer vision, vol 4, pp 116–129Google Scholar
  6. 6.
    Bo L, Ren X, Fox D (2013) Multipath sparse coding using hierarchical matching pursuit. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 660–667Google Scholar
  7. 7.
    Boureau Y-L, Bach F, LeCun Y, Ponce J (2010) Learning mid-level features for recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 2559–2566Google Scholar
  8. 8.
    Caballero C, Aranda MC (2010) Plant species identification using leaf image retrieval. In: Proceedings of the ACM international conference on image and video retrieval, pp 327–334Google Scholar
  9. 9.
    Chaki J, Parekh R (2011) Plant leaf recognition using shape based features and neural network classifiers. Int J Adv Comput Sci Appl 2(10):41–47Google Scholar
  10. 10.
    Chopra M (2015) Treeid: An image recognition system for plant species identification. Report in cs231n of Stanford UniversityGoogle Scholar
  11. 11.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297zbMATHGoogle Scholar
  12. 12.
    Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st international conference on machine learning, vol 32, pp 647–655Google Scholar
  13. 13.
    Du J-X, Wang X-F, Zhang G-J (2007) Leaf shape based plant species recognition. Appl Math Comput 185(2):883–893zbMATHGoogle Scholar
  14. 14.
    Zeiler MD, Taylor GW, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. In: Proceedings of IEEE international conference on computer vision, pp 2018–2025Google Scholar
  15. 15.
    Fiel S, Sablatnig R (2011) Automated identification of tree species from images of the bark, leaves and needles. In: Proceedings of 16th computer vision winter workshopGoogle Scholar
  16. 16.
    Goau H, Bonnet P, Barbe J, Bakic V, Joly A, Molino J-F, Barthelemy D, Boujemaa N (2012) Multi-organ plant identification. In: Proceedings of the 1st ACM international workshop on Multimedia analysis for ecological data, pp 41–44Google Scholar
  17. 17.
    Goau H, Bonnet P, Joly A (2015) Lifeclef plant identification task 2015. CLEFGoogle Scholar
  18. 18.
    Goau H, Bonnet P, Joly A (2016) Plant identi cation in an open-world (lifeclef 2016). In: Proceedings of conference and labs of the evaluation forum, pp 428–439Google Scholar
  19. 19.
    Guo Y, Ding G, Han J, Gao Y (2017) Zero-shot learning with transferred samples. IEEE Trans Image Process 26(7):3277–3290MathSciNetCrossRefGoogle Scholar
  20. 20.
    Guo Y, Ding G, Li L, Han J, Shao L (2017) Learning to hash with optimized anchor embedding for scalable retrieval. IEEE Trans Image Process 26 (3):1344–1354MathSciNetCrossRefGoogle Scholar
  21. 21.
    Guru DS, Sharath Kumar YH, Shantharamu M (2010) Texture features and knn in classification of flower images. Recent Trends in Image Processing and Pattern Recognition 37(1):21–29Google Scholar
  22. 22.
    Hsiao J-K, Kang L-W, Chang C-L, Hsu C-Y, Chen C-Y (2014) Learning sparse representation for leaf image recognition. In: Proceedings of IEEE conference on consumer electronics, pp 209–210Google Scholar
  23. 23.
    Hsiao J-K, Kang L-W, Chang CL, Lin CY (2014) Comparative study of leaf image recognition with a novel learning-based approach. In: Proceedings of science and information conference, pp 389–393Google Scholar
  24. 24.
    Jiang F, Zhang S, Wu S, Gao Y, Zhao D (2015) Multi-layered gesture recognition with kinect. J Mach Learn Res 16:227–254MathSciNetzbMATHGoogle Scholar
  25. 25.
    Kim S-J, Kim B-W, Kim D-P (2011) Tree recognition for landscape using by combination of features of its leaf, flower and bark. In: Proceedings of SICE annual conference, pp 1147–1151Google Scholar
  26. 26.
    Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90CrossRefGoogle Scholar
  27. 27.
    Kulkarni T, Uke NJ (2014) Implementation of image based flower classification system. International Journal of Computer Science and Business Informatics 13(1):35–44Google Scholar
  28. 28.
    Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez IC, Soares JVB (2012) Leafsnap: A computer vision system for automatic plant species identification. In: Proceedings of European Conference on Computer Vision, pp 502–516CrossRefGoogle Scholar
  29. 29.
    Kumar TP, Veera Prasad Reddy M, Bora PK (2016) Leaf identification using shape and texture features. In: Proceedings of international conference on computer vision and image processing, vol 460, pp 531–541Google Scholar
  30. 30.
    Kumar TP, Veera Prasad Reddy M, Bora PK (2017) Leaf identification using shape and texture features. In: Proceedings of international conference on computer vision and image processing, pp 531–541Google Scholar
  31. 31.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, vol 2, pp 2169–2178Google Scholar
  32. 32.
    Lee H, Grosse R, Ranganath R, Ng AY (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th annual international conference on machine learning, pp 609–616Google Scholar
  33. 33.
    Lee K-B, Hong K-S (2013) An implementation of leaf recognition system using leaf vein and shape. International Journal of Bio-Science and Bio-Technology 5(2):57–66CrossRefGoogle Scholar
  34. 34.
    Lee SH, Chan CS, Wilkin P, Remagnino P (2015) Deep-plant: Plant identification with convolutional neural networks. In: Proceedings of IEEE international conference on image processing, pp 452–456Google Scholar
  35. 35.
    Lin Z, Ding G, Han J, Wang J (2017) Cross-view retrieval via probability-based semantics-preserving hashing. IEEE Transactions on Cybernetics 47(12):4342–4355CrossRefGoogle Scholar
  36. 36.
    Liu Q, Lu X, He Z, Zhang C, Chen W-S (2017) Deep convolutional neural networks for thermal infrared object tracking. Knowl-Based Syst 134:189–198CrossRefGoogle Scholar
  37. 37.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110MathSciNetCrossRefGoogle Scholar
  38. 38.
    Ma L-H, Zhao Z-Q, Wang J (2013) Apleafis: an android-based plant leaf identification system. In: Proceedings of intelligent computing theories, pp 106–111CrossRefGoogle Scholar
  39. 39.
    Mabrouk AB, Najjar A, Zagrouba E (2014) Image flower recognition based on a new method for color feature extraction. In: Proceedings of the 9th international conference on computer vision theory and applications, vol 2, pp 201–206Google Scholar
  40. 40.
    Metre V, Ghorpade J (2013) An overview of the research on texture based plant leaf classification. International Journal of Computer Science and Network 2(3):25–36Google Scholar
  41. 41.
    Mouine S, Yahiaoui I, Verroust-Blondet A (2012) Advanced shape context for plant species identification using leaf image retrieval. In: Proceedings of the 2nd ACM international conference on multimedia retrievalGoogle Scholar
  42. 42.
    Mouine S, Yahiaoui I, Verroust-Blondet A, Joyeux L, Selmi S, Goau H (2013) An android application for leaf-based plant identification. In: Proceedings of the 3rd ACM conference on International conference on multimedia retrieval, pp 309–310Google Scholar
  43. 43.
    Nilsback M-E, Zisserman A (2008) Automated flower classification over a large number of classes. In: Proceedings of indian conference on computer vision, graphics and image processing, pp 722–729Google Scholar
  44. 44.
    Pallavi P, Veena Devi VS (2014) Leaf recognition based on feature extraction and zernike moments. In: Proceedings of international conference on advances in computer & communication engineering, vol 2, pp 67–73Google Scholar
  45. 45.
    Patel HN, Jain RK, Joshi MV (2011) Fruit detection using improved multiple features based algorithm. Int J Comput Appl 13(2):1–5Google Scholar
  46. 46.
    Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M-H Hedging deep features for visual tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence. In PressGoogle Scholar
  47. 47.
    Ren X-M, Wang X-F, Zhao Y (2012) An efficient multi-scale overlapped block lbp approach for leaf image recognition. In: Proceedings of the 8th international conference on intelligent computing theories and applications, pp 237–243Google Scholar
  48. 48.
    Satti V, Satya A, Sharma S (2013) An automatic leaf recognition system for plant identification using machine vision technology. International Journal of Engineering Science and Technology 5(4):874– 879Google Scholar
  49. 49.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: Proceedings of computer vision and pattern recognitionGoogle Scholar
  50. 50.
    Singh K, Gupta I, Gupta S (2010) Svm-bdt pnn and fourier moment technique for of leaf shape. International journal of signal processing, Image Processing and Pattern Recognition 3(4):67–78Google Scholar
  51. 51.
    Sünderhauf N, McCool C, Upcroft B, Perez T (2014) Fine-grained plant classification using convolutional neural networks for feature extraction. CLEFGoogle Scholar
  52. 52.
    Sun B-Y, Huang D-S, Guo L, Zhao Z-Q (2004) Support vector machine committee for classification. In: Proceedings of international symposium on neural networks, pp 648–653Google Scholar
  53. 53.
    Tsolakidis DG, Kosmopoulos DI, Papadourakis G (2014) Plant leaf recognition using zernike moments and histogram of oriented gradients. In: Proceedings of artificial intelligence: methods and applications, pp 406–417CrossRefGoogle Scholar
  54. 54.
    Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3360–3367Google Scholar
  55. 55.
    Wang X, Huang D-S, Du J-X, Xu H, Heutte L (2008) Classification of plant leaf images with complicated background. Appl Math Comput 205(2):916–926MathSciNetzbMATHGoogle Scholar
  56. 56.
    Wang Z, Chi Z, Feng D (2003) Shape based leaf image retrieval. IEE Proceedings on Vision, Image and Signal Processing 150(1):34–43CrossRefGoogle Scholar
  57. 57.
    Wang Z, Lu B, Chi Z, Feng D (2011) Leaf image classification with shape context and sift descriptors. In: Proceedings of international conference on digital image computing: techniques and applications, pp 650–654Google Scholar
  58. 58.
    Wilf P, Zhang S, Chikkerur S, Little SA, Wing SL, Serre T (2016) Computer vision cracks the leaf code. Proc Natl Acad Sci USA 113(12):3305–3310CrossRefGoogle Scholar
  59. 59.
    Xiao X-Y, Hu R, Zhang S-W, Wang X-F (2010) Hog-based approach for leaf classification. In: Proceedings of lecture notes in computer science, vol 6216, pp 149–155Google Scholar
  60. 60.
    Song Y, Glasbey CA, Horgan GW, Polder G, Dieleman JA, Van der Heijden GWAM (2014) Automatic fruit recognition and counting from multiple images. Biosyst Eng 118(1):203–215CrossRefGoogle Scholar
  61. 61.
    Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1794–1801Google Scholar
  62. 62.
    Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of european conference on computer vision, pp 818–833Google Scholar
  63. 63.
    Zhang B, Yang Y, Chen C, Yang L, Han J, Shao L (2017) Action recognition using 3d histograms of texture and a multi-class boosting classifier. IEEE Trans Image Process 26(10):4648–4660MathSciNetCrossRefGoogle Scholar
  64. 64.
    Zhang C, Zhou P, Li C, Liu L (2015) A convolutional neural network for leaves recognition using data augmentation. In: Proceedings of IEEE international conference on computer and information technology, pp 2143–2150Google Scholar
  65. 65.
    Zhang S, Lan X, Qi Y, Yuen PC (2017) Robust visual tracking via basis matching. IEEE Trans Circuits and Systems for Video Technology 27(3):421–430CrossRefGoogle Scholar
  66. 66.
    Zhang S, Lan X, Yao H, Zhou H, Tao D, Li X (2017) A biologically inspired appearance model for robust visual tracking. IEEE Trans Neural Networks and Learning Systems 28(10):2357–2370MathSciNetCrossRefGoogle Scholar
  67. 67.
    Zhang S, Qi Y, Jiang F, Lan X, Yuen PC, Zhou H Point-to-set distance metric learning on deep representations for visual tracking. IEEE Trans. on Intelligent Transportation Systems.  https://doi.org/10.1109/TITS.2017.2766093 CrossRefGoogle Scholar
  68. 68.
    Zhang S, Zhou H, Jiang F, Li X (2015) Robust visual tracking using structurally random projection and weighted least squares. IEEE Transactions on Circuits and Systems for Video Technology 25(11):1749–1760CrossRefGoogle Scholar
  69. 69.
    Zhao C, Chan SSF, Cham W-K, Chu LM (2015) Plant identification using leaf shapes - a pattern counting approach. Pattern Recogn 48(10):3203–3215CrossRefGoogle Scholar
  70. 70.
    Zhao Z, Huang X, Yang G (2015) Plant recognition based on leaf and bark images. Journal of Computational Information Systems 11(3):857–864Google Scholar
  71. 71.
    Zhao Z-Q, Xie B-J, Cheung YM, Wu X (2014) Plant leaf identification via a growing convolution neural network with progressive sample learning. In: Proceedings of asian conference on computer vision, vol 2, pp 348–361Google Scholar
  72. 72.
    Zhu H, Huang X, Zhang S, Yuen PC (2017) Plant identification via multipath sparse coding. Multimedia Tools and Applications 76(3):4599–4615CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of InformationBeijing Forestry UniversityBeijingChina
  2. 2.School of Opto-Electronic InformationYantai UniversityYantaiChina
  3. 3.School of Computer Science and TechnologyHarbin Institute of TechnologyWeihaiChina
  4. 4.School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  5. 5.Institute of Animation and digital artCommunication University of ChinaBeijingChina

Personalised recommendations