Maximum relevance minimum redundancy-based feature selection using rough mutual information in adaptive neighborhood rough sets

Abstract

Feature selection based on neighborhood rough sets (NRSs) has become a popular area of research in data mining. However, the limitation that NRSs inherently ignore the differences between different classes hinders their performance and development, and feature evaluation functions for NRSs cannot reflect the relevance between features and decisions effectively. The above two points restrict the effect of NRSs on feature selection. Consequently, a feature selection algorithm using the rough mutual information in adaptive neighborhood rough sets (ANRSs) is presented in this paper. First, we propose the boundaries of samples to granulate all samples and build ANRSs model by combining the boundaries of all samples in the same class, and the model discovers the characteristics of each class from the given data and overcomes the inherent limitation of NRSs. Second, inspired by combining information and algebraic views, we naturally extend the mutual information that represents the information view to ANRSs and combine it with the roughness that represents the algebraic view to propose the rough mutual information to quantitatively compute the relevance between features and decisions. Third, a maximum relevance minimum redundancy-based feature selection algorithm using the rough mutual information is studied. Additionally, to decrease the time consumption of the algorithm in processing high-dimensional datasets and improve the classification accuracy of the algorithm, the Fisher score dimensionality reduction method is introduced into the designed algorithm. Finally, the experimental results of the designed algorithm based on ANRSs and various algorithms based on NRSs are compared on eleven datasets to demonstrate the efficiency of our algorithm.

Appendix

Proof Proof of Proposition

1 From (13), (18), and (19), it is easy to know $\left \lvert {{{\left [ {{a_{i}}} \right ]}_{{S_{1}}}}} \right \rvert \ge \left \lvert {{{\left [ {{a_{i}}} \right ]}_{{S_{2}}}}} \right \rvert$, $\left \lvert {\underline {{R_{{S_{1}}}}} \left (D \right )} \right \rvert \le \left \lvert {\underline {{R_{{S_{2}}}}} \left (D \right )} \right \rvert$, and $\left \lvert {\overline {{R_{{S_{1}}}}} \left (D \right )} \right \rvert \ge \left \lvert {\overline {{R_{{S_{2}}}}} \left (D \right )} \right \rvert$. Thus, it is clear that ${\alpha _{{S_{1}}}}\left (D \right ) \le {\alpha _{{S_{2}}}}\left (D \right )$, $1 - {\alpha _{{S_{1}}}}\left (D \right ) \ge 1 - {\alpha _{{S_{2}}}}\left (D \right )$, and ${\gamma _{{S_{1}}}}\left (D \right ) \ge {\gamma _{{S_{2}}}}\left (D \right )$. Therefore, Proposition 1 is established. □

Proof Proof of Proposition

2

$$ \begin{array}{@{}rcl@{}} RM\left( {D;S} \right) & = & - \frac{{\left\lvert {\overline {{R_{S}}} \left( D \right)} \right\rvert - \left\lvert {\underline {{R_{S}}} \left( D \right)} \right\rvert}}{{\left\lvert U \right\rvert\left\lvert {\overline {{R_{S}}} \left( D \right)} \right\rvert}} \sum \limits_{i = 1}^{\left\lvert U \right\rvert} \log \frac{{\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}}} \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}}{{\left\lvert U \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}} \cap {{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}}\\ & = & - \frac{{\left( {\left\lvert {\overline {{R_{S}}} \left( D \right)} \right\rvert\! -\! \left\lvert {\underline {{R_{S}}} \left( D \right)} \right\rvert} \right)/\left\lvert {\overline {{R_{S}}} \left( D \right)} \right\rvert}}{{\left\lvert U \right\rvert}} \sum \limits_{i = 1}^{\left\lvert U \right\rvert} \log \frac{{\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}}} \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}}{{\left\lvert U \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}} \cap {{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}}\\ & = & - \frac{{1 - \left\lvert {\underline {{R_{S}}} \left( D \right)} \right\rvert/\left\lvert {\overline {{R_{S}}} \left( D \right)} \right\rvert}}{{\left\lvert U \right\rvert}} \sum \limits_{i = 1}^{\left\lvert U \right\rvert} \log \frac{{\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}}} \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}}{{\left\lvert U \right\rvert\left\lvert {{{\left[ {{a_{i}}} \right]}_{D}} \cap {{\left[ {{a_{i}}} \right]}_{S}}} \right\rvert}} \end{array} $$

From (21) and (26), Proposition 2 is established. □

Proof Proof of Proposition

3 $\left \lvert {{{\left [ {{a_{i}}} \right ]}_{{S_{1}}}}} \right \rvert \ge \left \lvert {{{\left [ {{a_{i}}} \right ]}_{{S_{2}}}}} \right \rvert$ and $\left \lvert {{{\left [ {{a_{i}}} \right ]}_{D}} \cap {{\left [ {{a_{i}}} \right ]}_{{S_{1}}}}} \right \rvert \ge \left \lvert {{{\left [ {{a_{i}}} \right ]}_{D}} \cap {{\left [ {{a_{i}}} \right ]}_{{S_{2}}}}} \right \rvert$ are known from (13). Therefore, the relationship between $M\left ({D;{S_{1}}} \right )$ and $M\left ({D;{S_{2}}} \right )$ is fuzzy. There is ${\gamma _{{S_{1}}}}\left (D \right ) \ge {\gamma _{{S_{2}}}}\left (D \right )$ according to Proposition 1. Thus, the numerical relationship between ${\gamma _{{S_{1}}}}\left (D \right )*M\left ({D;{S_{1}}} \right )$ and ${\gamma _{{S_{2}}}}\left (D \right )*M\left ({D;{S_{2}}} \right )$ is unclear, Proposition 3 is established. □