Efficient redundancy reduced subgroup discovery via quadratic programming

Abstract

Subgroup discovery is a task at the intersection of predictive and descriptive induction, aiming at identifying subgroups that have the most unusual statistical (distributional) characteristics with respect to a property of interest. Although a great deal of work has been devoted to the topic, one remaining problem concerns the redundancy of subgroup descriptions, which often effectively convey very similar information. In this paper, we propose a quadratic programming based approach to reduce the amount of redundancy in the subgroup rules. Experimental results on 12 datasets show that the resulting subgroups are in fact less redundant compared to standard methods. In addition, our experiments show that the computational costs are significantly lower than the costs of other methods compared in the paper.

Appendix: mutual information matrix is not positive definite

Proposition 2

Mutual information matrix is not positive definite.

Proof

: linear algebra states that a symmetric matrix H is positive definite if x ^T Hx > 0, $\forall x\ne 0$. The mutual information matrix H ∈ ℝ^n×n is symmetric and its entry is denoted as H _ij . Let x be a n × 1 column vector, x ∈ ℝ^n×1. Then, the matrix form of x ^T Hx can be written as:

$$ x^T H x = \begin{bmatrix} x_1 & x_2& \ldots & x_n \end{bmatrix} \begin{bmatrix}H_{11}& H_{12} & \ldots & H_{1n}\\ H_{21} & H_{22} & \ldots & H_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ H_{n1} & H_{n2} & \ldots & H_{nn}\\ \end{bmatrix} \begin{bmatrix} x_1 \\ x_2\\ \ldots\\ x_n \end{bmatrix}$$

(15)

$$ = \begin{bmatrix} x_1 & x_2& \ldots & x_n \end{bmatrix}\begin{bmatrix} \displaystyle\mathop{\sum}\nolimits_{i=1}^n H_{1i}x_i \\[3pt] \displaystyle\mathop{\sum}\nolimits_{i=1}^n H_{2i}x_i\\ \ldots\\ \displaystyle\mathop{\sum}\nolimits_{i=1}^n H_{ni}x_i \end{bmatrix} $$

(16)

$$ = H_{11}x_1^2+H_{12}x_1x_2+\ldots+H_{1n}x_1x_n $$

(17)

$$\begin{array}{lll} && + H_{21}x_2x_1+H_{22}x_2^2+\ldots+H_{2n}x_2x_n\\ && + \ldots\end{array}$$

(18)

$$ + H_{n1}x_nx_1+H_{n2}x_nx_2+\ldots+H_{nn}x_n^2 $$

(19)

$$ = \displaystyle\mathop{\sum}\nolimits_{i,j=1}^n H_{ij}x_ix_j. $$

(20)

If n = 4, then x ^T Hx can be expanded as $H_{11}x_1^2+H_{22}x_2^2+H_{33}x_3^2+H_{44}x_4^2+2H_{12}x_1x_2+2H_{13}x_1x_3+2H_{14}x_1x_4 +2H_{23}x_2x_3+2H_{24}x_2x_4+2H_{34}x_3x_4$, since H _ij  = H _ji for symmetric matrix. It is sufficient to show a counter example to complete the proof. For example, if a data set is $\begin{bmatrix}3& 4 & 2 & 5\\ 1 & 4 & 4 & 4\\ 5& 4 & 4 & 3\\ 1 & 3 & 2 & 2\\ \end{bmatrix}$ and the corresponding mutual information matrix H is $\begin{bmatrix}1.5& 0.3 & 0.5 & 1.5\\ 0.3 & 0.8 & 0.3 & 0.8\\ 0.5& 0.3 & 1.0 & 1.0\\ 1.5 & 0.8 & 1.0 & 2.0\\ \end{bmatrix}$ and the column vector $x=\begin{bmatrix} 7.2 & 5.4& 4.9 & -9.9 \end{bmatrix}^T$, then x ^T H x = − 0.19 < 0, which violates the necessary condition for a matrix being positive definite. Therefore the mutual information matrix is not positive definite. □