Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems
- 239 Downloads
In this paper, we propose a new incremental linear discriminant analysis (ILDA) for multitask pattern recognition (MTPR) problems in which a chunk of training data for a particular task are given sequentially and the task is switched to another related task one after another. The Pang et al.’s ILDA is extended such that a discriminant space of the current task is augmented with effective discriminant vectors that are selected from other tasks based on the class separability. We call this selective augmentation of discriminant vectors knowledge transfer of feature space. In the experiments, the proposed ILDA is evaluated for seven MTPR problems, each of which consists of three recognition tasks. The results demonstrate that the proposed ILDA with knowledge transfer outperforms the conventional ILDA and its naive extension to MTPR problems with regard to both class separability and recognition accuracy. We confirm that the proposed knowledge transfer works well to evolve effective feature spaces online in MTPR problems.
KeywordsLinear discriminant analysis Multitask learning Incremental learning Feature extraction
The authors would like to thank Professor Shigeo Abe for his helpful comments and discussions. This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Scientific Research (C) 205002205.
- Argyriou A, Evgeniou T, Pontil M (2007) Multi-task feature learning. In: Schölkopf B et al (eds) Advances in neural information processing systems 19, MIT Press, pp 41–48Google Scholar
- Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html
- Caruana R (1995) Learning many related tasks at the same time with backpropagation. In: Tesauro G et al (eds) Advances in neural information processing systems 7, MIT Press, pp 657–664Google Scholar
- Carpenter GA, Grossberg S (1998) The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21:77–88Google Scholar
- Cauwenberghs G, Poggio T (2000) Incremental and decremental support vector machine learning. In: Leen T et al (eds) Advances in neural information processing systems 13, MIT PressGoogle Scholar
- Kasabov N (2002) Evolving connectionist systems: methods and applications in bioinformatics, brain study and intelligent machines. Springer, LondonGoogle Scholar
- Roy A (2003) Neural networks: how do we make a widely used technology out of it? IEEE Neural Netw Soc Newsl 1:8–12Google Scholar
- Silver D and Mercer R (2001) Selective functional transfer: inductive bias from related tasks. In: Proc IASTED Int Conf on Artif Intell and Soft Comput, pp 182–189Google Scholar
- Takeuchi Y, Ozawa S, Abe S (2007) An efficient incremental kernel principal component analysis for online feature selection. In: Proc Int Joint Conf on Neural Netwo 2007, pp 1603–1608Google Scholar
- Thrun S, Pratt L (1998) Learning to learn, Kluwer, BostonGoogle Scholar