Advertisement

Genetic programming with transfer learning for texture image classification

  • Muhammad Iqbal
  • Harith Al-SahafEmail author
  • Bing Xue
  • Mengjie Zhang
Methodologies and Application
  • 171 Downloads

Abstract

Genetic programming (GP) represents a well-known and widely used evolutionary computation technique that has shown promising results in optimisation, classification, and symbolic regression problems. However, similar to many other techniques, the performance of GP deteriorates for solving highly complex tasks. Transfer learning can improve the learning ability of GP, which can be seen from previous research on including, but not limited to, symbolic regression and Boolean problems. However, utilising transfer learning to tackle image-related, specifically, image classification, problems in GP is limited. This paper aims at proposing a new method for employing transfer learning in GP to extract and transfer knowledge in order to tackle complex texture image classification problems. To assess the improvement gained from using the extracted knowledge, the proposed method is examined and compared against the baseline GP method and a state-of-the-art method on three publicly available and commonly used texture image classification datasets. The obtained results indicate that the reuse of the extracted knowledge from an image dataset has significant impact on improving the performance in learning different rotated versions of the same dataset, as well as other related image datasets. Further, it is found that the proposed approach in the very first generation of the evolutionary process produces better classification accuracy than the final classification accuracy obtained by the baseline method after 50 generations.

Keywords

Genetic programming Transfer learning Image classification Code fragments Evolutionary computation 

Notes

Acknowledgements

This work is supported in part by the Marsden Fund (Contract Numbers VUW1509, VUW1615) of New Zealand, and the University research grant of Victoria University of Wellington (Grant Numbers 213150 and 216137). Bing Xue received research grants from Marsden Fund (VUW1615) of New Zealand and from Victoria University of Wellington (213150). Mengjie Zhang received the research grants from Marsden Fund of New Zealand (VUW1509) and Victoria University of Wellington (216137).

Compliance with ethical standards

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. Al-Sahaf H, Song A, Neshatian K, Zhang M (2012) Two-tier genetic programming: towards raw pixel-based image classification. Expert Syst Appl 39(16):12291–12301CrossRefGoogle Scholar
  2. Al-Sahaf H, Zhang M, Johnston M, Verma B (2015)Image descriptor: a genetic programming approach to multiclass texture classification. In: Proceedings of 2015 IEEE congress on evolutionary computation. IEEE, pp 2460–2467Google Scholar
  3. Al-Sahaf H, Al-Sahaf A, Xue B, Johnston M, Zhang M (2017) Automatically evolving rotation-invariant texture image descriptors by genetic programming. IEEE Trans Evol Comput 21(1):83–101Google Scholar
  4. Blanchard G, Lee G, Scott C (2011) Generalizing from several related classification tasks to a new unlabeled sample. In: Proceedings of 2011 advances in neural information processing systems. Curran Associates, pp 2178–2186Google Scholar
  5. Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 120–128Google Scholar
  6. Brodatz P (1999) Textures: a photographic album for artists and designers. Dover Publications, New YorkGoogle Scholar
  7. Cha SH (2007) Comprehensive survey on distance/similarity measures between probability density functions. Int J Math Models Methods Appl Sci 1(4):300–307Google Scholar
  8. Chang BM, Tsai HH, Yen CY (2016) SVM-PSO based rotation-invariant image texture classification in SVD and DWT domains. Eng Appl Artif Intell 52:96–107CrossRefGoogle Scholar
  9. Chen Q, Xue B, Zhang M (2015) Generalisation and domain adaptation in GP with gradient descent for symbolic regression. In: Proceedings of 2015 IEEE Congress on Evolutionary Computation. IEEE, pp 1137–1144Google Scholar
  10. Christodoulidis S, Anthimopoulos M, Ebner L, Christe A, Mougiakakou S (2017) Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J Biomed Health Inform 21(1):76–84CrossRefGoogle Scholar
  11. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st international conference in machine learning, international conference on machine learning, vol 32. PMLR, pp 647–655Google Scholar
  12. Eiben AE, Smith J (2015) From evolutionary computation to the evolution of things. Nature 521(7553):476–482CrossRefGoogle Scholar
  13. Espejo PG, Ventura S, Herrera F (2010) A survey on the application of genetic programming to classification. IEEE Trans Syst Man Cybern C Appl Rev 40(2):121–144CrossRefGoogle Scholar
  14. Fogel DB (2007) Introduction to evolutionary computation, chap 1. Wiley, New York, pp 1–23Google Scholar
  15. Fu W, Johnston M, Zhang M (2014) Low-level feature extraction for edge detection using genetic programming. IEEE Trans Cybern 44(8):1459–1472CrossRefGoogle Scholar
  16. Galitsky BA (2013) Transfer learning of syntactic structures for building taxonomies for search engines. Eng Appl Artif Intell 26:2504–2515CrossRefGoogle Scholar
  17. Ghifary M, Kleijn WB, Zhang M (2014) Domain adaptive neural networks for object recognition. In: Proceedings of the 13th Pacific Rim international conference on artificial intelligence. Springer, pp 898–904Google Scholar
  18. Ghifary M, Kleijn WB, Zhang M, Balduzzi D (2015) Domain generalization for object recognition with multi-task autoencoders. In: Proceedings of 2015 IEEE international conference on computer vision. IEEE, pp 2551–2559Google Scholar
  19. Hafemann LG, Oliveira LS, Cavalin PR, Sabourin R (2015) Transfer learning between texture classification tasks using convolutional neural networks. In: Proceedings of the 2015 international joint conference on neural networks. IEEE, pp 1–7Google Scholar
  20. Hien NT, Hoai NX, McKay B (2011) A study on genetic programming with layered learning and incremental sampling. In: Proceedings of 2011 IEEE congress on evolutionary computation. IEEE, pp 1179–1185Google Scholar
  21. Hoang TH, McKay RIB, Essam D, Hoai NX (2011) On synergistic interactions between evolution, development and layered learning. IEEE Trans Evol Comput 15(3):287–312CrossRefGoogle Scholar
  22. Hosseinzadeh H, Razzazi F (2016) LMDT: a weakly-supervised large-margin-domain-transfer for handwritten digit recognition. Eng Appl Artif Intell 52:119–125CrossRefGoogle Scholar
  23. Iqbal M, Browne WN, Zhang M (2014) Reusing building blocks of extracted knowledge to solve complex, large-scale boolean problems. IEEE Trans Evol Comput 18(4):465–480CrossRefGoogle Scholar
  24. Iqbal M, Xue B, Zhang M (2016a) Reusing extracted knowledge in genetic programming to solve complex texture image classification problems. In: Proceedings of the 20th Pacific Asia knowledge discovery and data mining conference, Part II. Springer, pp 117–129Google Scholar
  25. Iqbal M, Zhang M, Xue B (2016b) Improving classification on images by extracting and transferring knowledge in genetic programming. In: Proceedings of 2016 IEEE congress on evolutionary computation. IEEE, pp 3582–3589Google Scholar
  26. Iqbal M, Xue B, Al-Sahaf H, Zhang M (2017) Cross-domain reuse of extracted knowledge in genetic programming for image classification. IEEE Trans Evol Comput 21(4):569–587CrossRefGoogle Scholar
  27. Jackson D, Gibbons AP (2007) Layered learning in boolean GP problems. In: Proceedings of the European conference on genetic programming, lecture notes in computer science, vol 4445. Springer, pp 148–159Google Scholar
  28. Jaśkowski W, Krawiec K, Wieloch B (2014) Cross-task code reuse in genetic programming applied to visual learning. Int J Appl Math Comput Sci 24(1):183–197CrossRefzbMATHGoogle Scholar
  29. Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, CambridgezbMATHGoogle Scholar
  30. Kylberg G (2011) The Kylberg texture dataset v. 1.0. External report (Blue series) 35, centre for image analysis, Swedish University of Agricultural Sciences and Uppsala University, Uppsala, SwedenGoogle Scholar
  31. Lensen A, Al-Sahaf H, Zhang M, Xue B (2015) A hybrid genetic programming approach to feature detection and image classification. In: Proceedings of the 30th international conference on image and vision computing New Zealand. IEEE, pp 1–6Google Scholar
  32. Lensen A, Al-Sahaf H, Zhang M, Xue B (2016) Genetic programming for region detection, feature extraction, feature construction and classification in image data. In Proceedings of the 19th European conference on genetic programming, lecture notes in computer science, vol 9594. Springer, pp 49–64Google Scholar
  33. Li Y, Ma J, Zhao Q (2008) Two improvements in genetic programming for image classification. In: Proceedings of the IEEE congress on evolutionary computation, pp 2492–2497Google Scholar
  34. Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowledge-Based Syst 80(C):14–23CrossRefGoogle Scholar
  35. Luke S (2013) Essentials of metaheuristics, 2nd edn. Lulu. https://cs.gmu.edu/~sean/book/metaheuristics/
  36. Muandet K, Balduzzi D, Schölkopf B (2013) Domain generalization via invariant feature representation. In: Proceedings of the international conference on machine learning, pp 10–18. JMLR.orgGoogle Scholar
  37. Muhammad G (2015) Date fruits classification using texture descriptors and shape-size features. Eng Appl Artif Intell 37:361–367CrossRefGoogle Scholar
  38. Ojala T, Mäenpää T, Pietikäinen M, Viertola J, Kyllonen J, Huovinen S (2002) Outex—new framework for empirical evaluation of texture analysis algorithms. In: Proceedings of the 16th international conference on pattern recognition, vol 1. IEEE, pp 701–706Google Scholar
  39. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359CrossRefGoogle Scholar
  40. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210CrossRefGoogle Scholar
  41. Patel VM, Gopalan R, Li R, Chellapa R (2015) Visual domain adaptation: a survey of recent advances. IEEE Signal Process Mag 32(3):53–69CrossRefGoogle Scholar
  42. Perez CB, Olague G (2009) Evolutionary learning of local descriptor operators for object recognition. In: Proceedings of the 11th annual conference on genetic and evolutionary computation. ACM, pp 1051–1058Google Scholar
  43. Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming (with contributions by Koza JR). http://www.gp-field-guide.org.uk/
  44. Sharif M, Jaffar MA, Mahmood MT (2014) Optimal composite morphological supervised filter for image denoising using genetic programming: application to magnetic resonance images. Eng Appl Artif Intell 31:78–89CrossRefGoogle Scholar
  45. Xue B, Zhang M, Browne W, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626CrossRefGoogle Scholar
  46. Zhang Y, Zhang E, Chen W (2016) Deep neural network for halftone image classification based on sparse auto-encoder. Eng Appl Artif Intell 50:245–255CrossRefGoogle Scholar
  47. Zuniga A, Mora M, Oyarce M, Fredes C (2014) Grape maturity estimation based on seed images and neural networks. Eng Appl Artif Intell 35:95–104CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations