Incremental Boolean Combination of Classifiers
The incremental Boolean combination (incrBC) technique is a new learn-and-combine approach that is proposed to adapt ensemble-based pattern classification systems over time, in response to new data acquired during operations. When a new block of training data becomes available, this technique generates a diversified pool of base classifiers from the data by varying training hyperparameters and random initializations. The responses of these classifiers are then combined with those of previously-trained classifiers through Boolean combination in the ROC space. Through this process, an ensemble is selected from the pool, where Boolean fusion functions and thresholds are adapted for improved accuracy, while redundant base classifiers are pruned. Results of computer simulations conducted using Hidden Markov Models (HMMs) on synthetic and real-world host-based intrusion detection data indicate that incrBC can sustain a significantly higher level of accuracy than when the parameters of a single best HMM are re-estimated for each new block of data, using reference batch and incremental learning techniques. It also outperforms static fusion techniques such as majority voting for combining the responses of new and previously-generated pools of HMMs. Pruning prevents pool sizes from increasing indefinitely over time, without adversely affecting the overall ensemble performance.
KeywordsBoolean Function Anomaly Detection Decision Threshold Incremental Learning Boolean Combination
Unable to display preview. Download preview PDF.
- 1.Connolly, J.F., Granger, E., Sabourin, R.: An adaptive classification system for video-based face recognition. Information Sciences (2010) (in Press)Google Scholar
- 2.Khreich, W., Granger, E., Miri, A., Sabourin, R.: A comparison of techniques for on-line incremental learning of HMM parameters in anomaly detection. In: Proc. 2nd IEEE Int’l Conf. on Computational Intelligence for Security and Defense Applications, Ottawa, Canada, July 2009, pp. 1–8 (2009)Google Scholar
- 5.Mizuno, J., Watanabe, T., Ueki, K., Amano, K., Takimoto, E., Maruoka, A.: On-line estimation of hidden Markov model parameters. In: Proc. 3rd Int’l Conf. on Discovery Science, vol. 1967, pp. 155–169 (2000)Google Scholar
- 8.Tao, Q., Veldhuis, R.: Threshold-optimized decision-level fusion and its application to biometrics. Pattern Recognition 41(5), 852–867 (2008)Google Scholar
- 10.Tulyakov, S., Jaeger, S., Govindaraju, V., Doermann, D.: Review of classifier combination methods. In: Marinai, S., H.F. (eds.) Studies in Comp. Intelligence: ML in Document Analysis and Recognition, pp. 361–386. Springer, Heidelberg (2008)Google Scholar
- 11.Warrender, C., Forrest, S., Pearlmutter, B.: Detecting intrusions using system calls: Alternative data models. In: Proc. IEEE Computer Society Symposium on Research in Security and Privacy, Oakland, CA, pp. 133–145 (1999)Google Scholar