Semi-supervised Non-negative Local Coordinate Factorization

Abstract

Non-negative matrix factorization (NMF) is a popular matrix decomposition technique that has attracted extensive attentions from data mining community. However, NMF suffers from the following deficiencies: (1) it is non-trivial to guarantee the representation of the data points to be sparse, and (2) NMF often achieves unsatisfactory clustering results because it completely neglects the labels of the dataset. Thus, this paper proposes a semi-supervised non-negative local coordinate factorization (SNLCF) to overcome the above deficiencies. Particularly, SNLCF induces the sparse coefficients by imposing the local coordinate constraint and propagates the labels of the labeled data to the unlabeled ones by indicating the coefficients of the labeled examples to be the class indicator. Benefit from the labeled data, SNLCF can boost NMF in clustering the unlabeled data. Experimental results on UCI datasets and two popular face image datasets suggest that SNLCF outperforms the representative methods in terms of both average accuracy and average normalized mutual information.

Appendix A: Proof of Theorem 2.1

For any entry ${{u}_{ab}}$ in U , we use ${{F}_{ab}}$ to denote the related terms of the objective O with ${{u}_{ab}}$. The partial derivatives of O over ${{u}_{ab}}$ is:

$$\begin{aligned} F'_{ab} = \frac{{\partial O}}{{\partial {U_{ab}}}} = {((2UV{V^T} - 2X{V^T}) + \beta \sum \limits _{i = 1}^{{n_u}} {( - 2x^u_i{1^T}{\varLambda _i} + 2U{\varLambda _i})} )_{ab}}, \end{aligned}$$

(7)

where ${\varLambda _i} = diag({v^u_i}) \in {R^{k \times k}}$.

According to (7) and [25], we construct the auxiliary function of ${F_{ab}}$ as follows

$$\begin{aligned} G\left( {u,u_{ab}^{\left( t \right) }} \right)&= {F_{ab}}\left( {u_{ab}^{\left( t \right) }} \right) + F_{ab}'\left( {u_{ab}^{\left( t \right) }} \right) \left( {u - u_{ab}^{\left( t \right) }} \right) \nonumber \\&\quad +\frac{{{{\left( {UV{V^T}} \right) }_{ab}} + \beta \sum \limits _{i = 1}^{{n_u}} {{{\left( {U{\varLambda _i}} \right) }_{ab}}} }}{{u_{ab}^{\left( t \right) }}}{\left( {u - u_{ab}^{\left( t \right) }} \right) ^2}. \end{aligned}$$

(8)

By setting the derivatives of (8) to zero, we can obtain the update rule (4). Similarly, we can obtain the update rules (5) and (6) for $V_u$ and $V_l$, respectively.

According to the property of the auxiliary function [25], we can obtain

$$\begin{aligned} O\left( {{U^{\left( {t + 1} \right) }},{V^{\left( {t + 1} \right) }}} \right) \le O\left( {{U^{\left( t \right) }},{V^{\left( t \right) }}} \right) \le O\left( {{U^{\left( {t - 1} \right) }},{V^{\left( {t - 1} \right) }}} \right) \!. \end{aligned}$$

(9)

Based on (9), this MUR can guarantee the objective to be non-increased. Meanwhile, the equalition (9) holds when their sub-gradients of objective (3) are zeros. Thus, this MUR can converge to local minima of SNLCF. This completes the proof.    $\square$