Multi-label Feature Selection Method Based on Multivariate Mutual Information and Particle Swarm Optimization

Abstract

Multi-label feature selection has become an indispensable pre-processing step to deal with possible irrelevant and redundant features, to decrease computational burdens, improve classification performance and enhance model interpretability, in multi-label learning. Mutual information (MI) between two random variables is widely used to describe feature-label relevance and feature-feature redundancy. Furthermore, multivariate mutual information (MMI) is approximated via limiting three-degree interactions to speed up its computation, and then is used to characterize relevance between selected feature subset and label subset. In this paper, we combine MMI-based relevance with MI-based redundancy to define a new max-relevance and min-redundancy feature selection criterion (simply MMI). To search for a globally optimal solution, we add an auxiliary mutation operation to existing binary particle swarm optimization with mutation to control the number of selected features strictly to form a new PSO variant: M2BPSO. Integrating MMI with M2BPSO builds a novel multi-label feature selection method: MMI-PSO. The experiments on four benchmark data sets demonstrate the effectiveness of our proposed algorithm, according to four instance-based classification evaluation metrics, compared with three state-of-the-art feature selection approaches.