Fitness Function Comparison for GA-Based Feature Construction
When primitive data representation yields attribute interactions, learning requires feature construction. MFE2/GA, a GA-based feature construction has been shown to learn more accurately than others when there exist several complex attribute interactions. A new fitness function, based on the principle of Minimum Description Length (MDL), is proposed and implemented as part of the MFE3/GA system. Since the individuals of the GA population are collections of new features constructed to change the representation of data, an MDL-based fitness considers not only the part of data left unexplained by the constructed features (errors), but also the complexity of the constructed features as a new representation (theory). An empirical study shows the advantage of the new fitness over other fitness not based on MDL, and both are compared to the performance baselines provided by relevant systems.
KeywordsMachine learning attribute interaction feature construction feature selection genetic algorithms MDL principle Entropy
Unable to display preview. Download preview PDF.
- 3.Jakulin, A., Bratko, I.: Testing the significance of attribute interactions. In: Brodley, C.E. (ed.) Proc. of the Twenty-first International Conference on Machine Learning, pp. 409–416. ACM Press, New York, USA (2004)Google Scholar
- 4.Pérez, E., Rendell, L.A.: Using multidimensional projection to find relations. In: Proc. of the Twelfth International Conference on Machine Learning, Tahoe City, California, pp. 447–455. Morgan Kaufmann, San Francisco (1995)Google Scholar
- 5.Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, New York, Inc (1999)Google Scholar
- 6.Larsen, O., Freitas, A.A., Nievola, J.C.: Constructing X-of-N attributes with a genetic algorithm. In: Proc. of the GECCO, p. 1268. Morgan Kaufmann, San Francisco (2002)Google Scholar
- 9.Ritthoff, O., Klinkenberg, R., Fischer, S., Mierswa, I.: A hybrid approach to feature selection and generation using an evolutionary algorithm. In: UK Workshop on Computational Intelligence (September 2002)Google Scholar
- 10.Shafti, L.S., Pérez, E.: Reducing complex attribute interaction through non-algebraic feature construction. In: Proc. of the IASTED International Conference on AIA, Innsbruck, Austria, pp. 359–365. Acta Press (February 2007)Google Scholar
- 11.Grunwald, P.D.: The Minimum Description Length Principle. MIT Press, Cambridge (2007)Google Scholar
- 14.Quinlan, R.J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, California (1993)Google Scholar
- 15.Shannon, C.E.: A mathematical theory of communication. Bell System Tech. Journal 27, 379–423, 623–656 (1948)Google Scholar