Advertisement

Tackling Overfitting in Evolutionary-Driven Financial Model Induction

  • Clíodhna Tuite
  • Alexandros Agapitos
  • Michael O’Neill
  • Anthony Brabazon
Part of the Studies in Computational Intelligence book series (SCI, volume 380)

Summary

This chapter explores the issue of overfitting in grammar-based Genetic Programming. Tools such as Genetic Programming are well suited to problems in finance where we seek to learn or induce a model from data. Models that overfit the data upon which they are trained prevent model generalisation, which is an important goal of learning algorithms.

Early stopping is a technique that is frequently used to counteract overfitting, but this technique often fails to identify the optimal point at which to stop training. In this chapter, we implement four classes of stopping criteria, which attempt to stop training when the generalisation of the evolved model is maximised.

We show promising results using, in particular, one novel class of criteria, which measured the correlation between the training and validation fitness at each generation. These criteria determined whether or not to stop training depending on the measurement of this correlation - they had a high probability of being the best among a suite of potential criteria to be used during a run. This meant that they often found the lowest validation set error for the entire run faster than other criteria.

Keywords

Genetic Programming Excess Return Generalisation Error Good Criterion Trading Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agapitos, A., O’Neill, M., Brabazon, A.: Evolutionary Learning of Technical Trading Rules without Data-Mining Bias. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 294–303. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Banzhaf, W., Nordin, P., Keller, R., Francone, F.: Genetic programming: An introduction. Morgan Kaufmann, San Francisco (1999)Google Scholar
  3. 3.
    Becker, L.A., Seshadri, M.: Comprehensibility and overfitting avoidance in genetic programming for technical trading rules. Worcester Polytechnic Institute, Computer Science Technical Report (2003)Google Scholar
  4. 4.
    Brabazon, A., Dang, J., Dempsey, I., O’Neill, M., Edelman, D.: Natural computing in finance: a review. In: Handbook of Natural Computing: Theory, Springer, Heidelberg (2010)Google Scholar
  5. 5.
    Brabazon, A., O’Neill, M.: Biologically inspired algorithms for financial modelling. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  6. 6.
    Costelloe, D., Ryan, C.: On improving generalisation in genetic programming. In: Vanneschi, L., Gustafson, S., Moraglio, A., De Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 61–72. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Dempsey, I., O’Neill, M., Brabazon, A.: Foundations in Grammatical Evolution for Dynamic Environments. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Domingos, P.: The role of Occam’s razor in knowledge discovery. Data Mining and Knowledge Discovery 3(4), 409–425 (1999)CrossRefGoogle Scholar
  9. 9.
    Gagné, C., Schoenauer, M., Parizeau, M., Tomassini, M.: Genetic programming, validation sets, and parsimony pressure. In: Collet, P., Tomassini, M., Ebner, M., Gustafson, S., Ekárt, A. (eds.) EuroGP 2006. LNCS, vol. 3905, pp. 109–120. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Gencay, R., Qi, M.: Pricing and hedging derivative securities with neural networks: Bayesian regularization, early stopping, and bagging. IEEE Transactions on Neural Networks 12(4), 726–734 (2002)CrossRefGoogle Scholar
  11. 11.
    Kushchu, I.: Genetic programming and evolutionary generalization. IEEE Transactions on Evolutionary Computation 6(5), 431–442 (2002)CrossRefGoogle Scholar
  12. 12.
    Luke, S., Panait, L.: Lexicographic parsimony pressure. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 829–836. Morgan Kaufmann, San Francisco (2002)Google Scholar
  13. 13.
    Luke, S., Panait, L.: A comparison of bloat control methods for genetic programming. IEEE Transactions on Evolutionary Computation 14(3), 309–344 (2006)Google Scholar
  14. 14.
    Mckay, R.I., Hoai, N.X., Whigham, P.A., Shan, Y., O’Neill, M.: Grammar-based Genetic Programming: a survey. Genetic Programming and Evolvable Machines 11(3-4), 365–396 (2010)CrossRefGoogle Scholar
  15. 15.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  16. 16.
    O’Neill, M., Hemberg, E., Gilligan, C., Bartley, E., McDermott, J., Brabazon, A.: GEVA: grammatical evolution in Java. ACM SIGEVOlution 3(2), 17–22 (2008)CrossRefGoogle Scholar
  17. 17.
    O’Neill, M., Ryan, C.: Grammatical Evolution: Evolutionary automatic programming in an arbitrary language. Kluwer, Dordrecht (2003)zbMATHGoogle Scholar
  18. 18.
    O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genetic Programming and Evolvable Machines 11(3-4), 339–363 (2010)CrossRefGoogle Scholar
  19. 19.
    Paris, G., Robilliard, D., Fonlupt, C.: Exploring overfitting in genetic programming. In: Liardet, P., Collet, P., Fonlupt, C., Lutton, E., Schoenauer, M. (eds.) EA 2003. LNCS, vol. 2936, pp. 267–277. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  20. 20.
    Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming. Lulu Enterprises, UK (2008)Google Scholar
  21. 21.
    Prechelt, L.: Early stopping - but when? In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, p. 55. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  22. 22.
    Thomas, J.D., Sycara, K.: The importance of simplicity and validation in genetic programming for data mining in financial data. In: Proceedings of the Joint GECCO 1999 and AAAI 1999 Workshop on Data Mining with Evolutionary Algorithms: Research Directions, pp. 7–11 (1999)Google Scholar
  23. 23.
    Tuite, C., Agapitos, A., O’Neill, M., Brabazon, A.: A Preliminary Investigation of Overfitting in Evolutionary Driven Model Induction: Implications for Financial Modelling. In: Di Chio, C., Brabazon, A., Di Caro, G.A., Drechsler, R., Farooq, M., Grahl, J., Greenfield, G., Prins, C., Romero, J., Squillero, G., Tarantino, E., Tettamanzi, A.G.B., Urquhart, N., Uyar, A.Ş. (eds.) EvoApplications 2011, Part II. LNCS, vol. 6625, pp. 120–130. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  24. 24.
    Tuite, C., Agapitos, A., O’Neill, M., Brabazon, A.: Early Stopping Criteria to Counteract Overfitting in Genetic Programming. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation. ACM, New York (2011) (forthcoming)Google Scholar
  25. 25.
    Vanneschi, L., Gustafson, S.: Using crossover based similarity measure to improve genetic programming generalization ability. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 1139–1146. ACM, New York (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Clíodhna Tuite
    • 1
    • 2
  • Alexandros Agapitos
    • 1
    • 2
  • Michael O’Neill
    • 1
    • 2
  • Anthony Brabazon
    • 1
    • 3
  1. 1.Natural Computing Research and Applications Group & Financial Mathematics and Computation Research Cluster, Complex and Adaptive Systems LaboratoryUniversity College DublinIreland
  2. 2.School of Computer Science and InformaticsUniversity College DublinIreland
  3. 3.School of BusinessUniversity College DublinIreland

Personalised recommendations