Non-Outlier Pseudo-Labeling for Short Text Clustering

Abstract

Instance-level correlation and cluster-level discrepancy of data are two crucial aspects of short text clustering. Current deep clustering methods, however, suffer from inaccurate estimation of either instance-level correlation or cluster-level discrepancy of data and strongly relay on the quality of the initial text representation. In this paper, we propose a Non-outlier Pseudo-labeling-based Short Text Clustering (NPLC) method, which consists of two parts. In the first part, we use Mask Language Model (MLM) to pre-train the feature model on a given dataset to enhance the initial text representation. The second part based on non-outlier pseudo-labeling is a joint training in which we first cluster the dataset and select cluster labels of outlier-free data in each cluster as pseudo labels for the next joint training based on a novel framework. The novel framework makes use of a contrastive loss to gain excellent inter-cluster separation by minimizing similarity between outlier-free and outlier data and a clustering loss to narrow intra-cluster distances by maximizing similarity among outlier-free data. Extensive experimental results demonstrate that NPLC achieves significant improvements over existing methods and advances the state-of-the-art results on most benchmark datasets with 1%–12% improvement on Accuracy and 1%–6% improvement on Normalized Mutual Information.