Abstract
Genetic programming (GP) is a popular and powerful optimization algorithm that has a wide range of applications, such as time series prediction, classification, data mining, and knowledge discovery. Despite the great success it enjoyed, selecting the proper primitives from high-dimension primitive set for GP to construct solutions is still a time-consuming and challenging issue that limits the efficacy of GP in real-world applications. In this paper, we propose a multi-population GP framework with adaptively weighted primitives to address the above issues. In the proposed framework, the entire population consists of several sub-populations and each has a different vector of primitive weights to determine the probability of using the corresponding primitives in a sub-population. By adaptively adjusting the weights of the primitives and periodically sharing information between sub-populations, the proposed framework can efficiently identify important primitives to assist the search. Furthermore, based on the proposed framework and the graphics processing unit computing technique, a high-performance self-learning gene expression programming algorithm (HSL-GEP) is developed. The HSL-GEP is tested on fifteen problems, including four real-world problems. The experimental results have demonstrated that the proposed HSL-GEP outperforms several state-of-the-art GPs, in terms of both solution quality and search efficiency.
Similar content being viewed by others
Notes
The dataset of \(F_{11}\) can be downloaded from http://symbolicregression.com/sites/default/files/DataSets/towerData.txt.
The dataset of \(F_{12,13}\) can be downloaded from http://archive.ics.uci.edu/ml/datasets/Communities+and+Crime+Unnormalized.
References
Ahmad F, Isa NAM, Hussain Z, Osman MK, Sulaiman SN (2015) A ga-based feature selection and parameter optimization of an ann in diagnosing breast cancer. Pattern Anal Appl 18(4):861–870
Ahmed S, Zhang M, Peng L (2013) Enhanced feature selection for biomarker discovery in LC-MS data using GP. In: IEEE congress on evolutionary computation (CEC), pp 584–591
Antonio LM, Coello CCA (2018) Coevolutionary multiobjective evolutionary algorithms: survey of the state-of-the-art. IEEE Trans Evol Comput 22(6):851–865
Banzhaf W, Harding S, Langdon WB, Wilson G (2008) Accelerating genetic programming through graphics processing units. In: Genetic programming theory and practice VI, pp 1–19
Brameier MF, Banzhaf W (2007) Linear genetic programming. Springer, Berlin
Cano A, Ventura S (2014) Gpu-parallel subtree interpreter for genetic programming. In: Conference on genetic and evolutionary computation, pp 887–894
Chen B, Chen B, Liu H, Zhang X (2015) A fast parallel genetic algorithm for graph coloring problem based on CUDA. In: International conference on cyber-enabled distributed computing and knowledge discovery, pp 145–148
Chen Q, Xue B, Niu B, Zhang M (2016) Improving generalisation of genetic programming for high-dimensional symbolic regression with feature selection. In Congress on evolutionary computation (CEC), pp 3793–3800
Chen Q, Zhang M, Xue B (2017) Feature selection to improve generalization of genetic programming for high-dimensional symbolic regression. IEEE Trans Evol Comput 21(5):792–806
Chitty DM (2016a) Faster GPU based genetic programming using A two dimensional stack. In: CoRR. arXiv:1601.00221
Chitty DM (2016b) Improving the performance of gpu-based genetic programming through exploitation of on-chip memory. Soft Comput 20(2):661–680
Cook S (2012) CUDA programming: a developer’s guide to parallel computing with GPUs. Elsevier, London
Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21(15):4387–4398
Deng W, Yao R, Zhao H, Yang X, Li G (2019a) A novel intelligent diagnosis method using optimal LS-SVM with improved pso algorithm. Soft Comput 23(7):2445–2462
Deng W, Xu J, Zhao H (2019b) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292
Dick G, Rimoni AP, Whigham PA (2015) A re-examination of the use of genetic programming on the oral bioavailability problem. In: Proceedings of the genetic and evolutionary computation conference (GECCO)
Espejo PG, Ventura S, Herrera F (2010) A survey on the application of genetic programming to classification. IEEE Trans Syst Man Cybern C Appl Rev 40(2):121–144
Ferreira C (2006) Gene expression programming. Springer, Berlin
Ffrancon R, Schoenauer M (2015) Memetic semantic genetic programming. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation. ACM, pp 1023–1030
Gandomi AH, Sajedi S, Kiani B, Huang Q (2016) Genetic programming for experimental big data mining: a case study on concrete creep formulation. Autom Constr 70:89–97
Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822
Hancer E, Xue B, Zhang M, Karaboga D, Akay B (2018) Pareto front feature selection based on artificial bee colony optimization. Inf Sci 422:462–479
Harding S, Banzhaf W (2007) Fast genetic programming and artificial developmental systems on GPUs. In: 21st International symposium on high performance computing systems and applications, 2007. HPCS 2007, p 2
Harding S, Banzhaf W (2007) Fast genetic programming on GPUs. In: European conference on genetic programming, pp 90–101
Harvey DY, Todd MD (2015) Automated feature design for numeric sequence classification by genetic programming. IEEE Trans Evol Comput 19(4):474–489
Keijzer M (2003) Improving symbolic regression with interval arithmetic and linear scaling. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 2610. Essex, UK, pp 70–82
Koza JR, Poli R (2005) Genetic programming
Langdon WB (2010) A many threaded CUDA interpreter for genetic programming. Springer, Berlin
Langdon WB (2011) Graphics processing units and genetic programming: an overview. Soft Comput 15(8):1657–1669
McDermott J, White DR., Luke S, Manzoni L, Castelli M, Vanneschi L, Jaskowski W, Krawiec K, Harper R, De Jong K, O’Reilly U-M (2012) Genetic programming needs better benchmarks. In: Proceedings of the 14th international conference on genetic and evolutionary computation, GECCO’12, pp 791–798
Mei Y, Omidvar MN, Li X, Yao X (2016a) A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization. ACM Trans Math Softw 42(2):13
Mei Y, Zhang M, Nguyen S (2016b) Feature selection in evolving job shop dispatching rules with genetic programming. In: Proceedings of the genetic and evolutionary computation conference (GECCO). ACM, pp 365–372
Mei Y, Su N, Xue B, Zhang M (2017) An efficient feature selection algorithm for evolving job shop scheduling rules with genetic programming. IEEE Trans Emerg Top Comput Intell 1(5):339–353
Miller JF, Thomson P (2000) Cartesian genetic programming. In: Genetic programming. Springer, Berlin, pp 121–132
Moore JH, Hill DP, Saykin A, Shen L (2013) Exploring interestingness in a computational evolution system for the genome-wide genetic analysis of Alzheimer’s Disease. Springer, New York
Moraglio A, Krawiec K, Johnson CG (2012) Geometric semantic genetic programming. In: International conference on parallel problem solving from nature. Springer, Berlin, pp 21–31
Neshatian K, Zhang M (2009) Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the genetic and evolutionary computation conference, GECCO 2009. Montreal, Québec, Canada, pp 1027–1034
O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358
Riley M, Mei Y, Zhang M (2016) Improving job shop dispatchingrules via terminal weighting and adaptive mutation in genetic programming. Vancouver, BC, Canada, pp 3362 – 3369
Rojas F, Meza F (2015) A parallel distributed genetic algorithm for the prize collecting steiner tree problem. In: International conference on computational science and computational intelligence (CSCI), pp. 643–646
Rosenwald A, Wright G, Chan WC, Connors JM, Campo E, Fisher RI, Gascoyne RD, Muller-Hermelink HK, Smeland EB, Giltnane JM (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. New Engl J Med 346(25):1937–1947
Sandin I, Andrade G, Viegas F, Madeira D (2012) Aggressive and effective feature selection using genetic programming. In: IEEE congress on evolutionary computation (CEC), pp 1–8
Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324(5923):81–85
Shao S, Liu X, Zhou M, Zhan J, Liu X, Chu Y, Chen H (2012) A gpu-based implementation of an enhanced GEP algorithm. In: Conference on genetic and evolutionary computation, pp 999–1006
Vladislavleva E, Smits G, Den Hertog D (2009) Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans Evol Comput 13(2):333–349
Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Syst Man Cybern 43(6):1656–1671
Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626
Yang Z, Tang K, Yao X (2008) Large scale evolutionary optimization using cooperative coevolution. Inf Sci 178(15):2985–2999
Yao X, Liu Y, Lin GM (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102
Zhai Y, Ong YS, Tsang IW (2014) The emerging “big dimensionality”. IEEE Comput Intell Mag 9(3):14–26
Zhong J, Cai W (2015) Differential evolution with sensitivity analysis and the powell’s method for crowd model calibration. J Comput Sci 9:26–32
Zhong J, Ong YS, Cai W (2016) Self-learning gene expression programming. IEEE Trans Evol Comput 20(1):65–80
Zhong J, Cai W, Lees M, Luo L (2017a) Automatic model construction for the behavior of human crowds. Appl Soft Comput 56:368–378
Zhong J, Feng L, Ong Y-S (2017b) Gene expression programming: a survey. IEEE Comput Intell Mag 12(3):54–72
Zhou C, Xiao W, Tirpak TM, Nelson PC (2003) Evolving accurate and compact classification rules with gene expression programming. IEEE Trans Evol Comput 7(6):519–531
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Grant No. 61602181), Program for Guangdong Introducing Innovative and Entrepreneurial Teams (Grant No. 2017ZT07X183), Guangdong Natural Science Foundation Research Team (Grant No. 2018B030312003), and the Fundamental Research Funds for the Central Universities (Grant No. D2191200).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, Z., Zhong, J., Feng, L. et al. A fast parallel genetic programming framework with adaptively weighted primitives for symbolic regression. Soft Comput 24, 7523–7539 (2020). https://doi.org/10.1007/s00500-019-04379-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-019-04379-4