Fastfood Elastic Net: Combining Variable Selection with Kernel Expansion Approximations
As the complexity of a prediction problem grows, simple linear approaches tend to fail which has led to the development of algorithms to make complicated, nonlinear problems solvable both quickly and inexpensively. Fastfood, one of such algorithms, has been shown to generate reliable models, but its current state does not offer feature selection that is useful in solving a wide array of complex real-world problems that spans from cancer prediction to financial analysis.
The aim of this research is to extend Fastfood with variable importance by integrating with Elastic net. Elastic net offers feature selection, but is only capable of producing linear models. We show that in combining the two, it is possible to retain the feature selection offered by the Elastic net and the nonlinearity produced by Fastfood. Models constructed with the Fastfood enhanced Elastic net are relatively quick and inexpensive to compute and are also quite powerful in their ability to make accurate predictions.
KeywordsKernel methods Data mining Algorithms and programming techniques for big data processing
The authors would like to thank the College of Charleston for hosting the NSF Omics REU which is funded by the National Science Foundation DBI Award 1359301 as well as the UCI machine learning repository .
- 1.Bowick, M., Neiswanger, W.: Learning fastfood feature transforms for scalable neural networks. In: Proceedings of the International conference on..., pp. 1–15 (2013)Google Scholar
- 2.Freund, Y., Schapire, R.R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
- 3.Le, Q., Sarlós, T., Smola, A.J.: Fastfood – approximating kernel expansions in loglinear time. Int. Conf. Mach. Learn. 28(1), 1–29 (2013)Google Scholar
- 4.Lichman, M.: UCI machine learning repository (2013)Google Scholar
- 5.Rahimi, A., Recht, B.: Weighted sums of random kitchen sinks: replacing minimization with randomization in learning. Adv. Neural Inf. Process. Syst. 1(1), 1–8 (2009)Google Scholar
- 6.Zou, H., Hastie, T.: Journal of the Royal Statistical Society. Series B: Statistical Methodology 28 (2013)Google Scholar